2024-09-17T07:54:26.532 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T07:54:26.549 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T07:54:26.624 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191 branch: reef description: orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} email: ceph-qa@ceph.com first_in_suite: false job_id: '7908191' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 5234 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 46/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3Gac62gk08//5LBAYr9Cl1gdj+6dKc2GAWuPWiFE0Tjb1x8erC/4t+z6AJR7pjD1bXB28ankovYpK5g495df8= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFc7r+1tcVGIkkLNelnqr08P5oPZGD1dG/LZasg8yJ2Y8NVkh02QsyV+SBcOo9ZMo+cWTN/T/QLpLtmerrz5u8s= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T07:54:26.624 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa; will attempt to use it 2024-09-17T07:54:26.624 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa/tasks 2024-09-17T07:54:26.625 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T07:54:26.626 INFO:teuthology.task.internal:Checking packages... 2024-09-17T07:54:26.647 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-17T07:54:26.647 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T07:54:26.647 INFO:teuthology.packaging:ref: None 2024-09-17T07:54:26.648 INFO:teuthology.packaging:tag: None 2024-09-17T07:54:26.648 INFO:teuthology.packaging:branch: reef 2024-09-17T07:54:26.648 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T07:54:26.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-17T07:54:26.892 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-17T07:54:26.894 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T07:54:26.900 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T07:54:26.900 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T07:54:26.905 INFO:teuthology.task.internal:Saving configuration 2024-09-17T07:54:26.918 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T07:54:26.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T07:54:26.951 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 07:49:34.745974', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3Gac62gk08//5LBAYr9Cl1gdj+6dKc2GAWuPWiFE0Tjb1x8erC/4t+z6AJR7pjD1bXB28ankovYpK5g495df8='} 2024-09-17T07:54:26.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 07:49:34.747349', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFc7r+1tcVGIkkLNelnqr08P5oPZGD1dG/LZasg8yJ2Y8NVkh02QsyV+SBcOo9ZMo+cWTN/T/QLpLtmerrz5u8s='} 2024-09-17T07:54:26.973 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T07:54:26.979 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T07:54:26.979 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-17T07:54:26.979 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T07:54:27.041 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe8f09e3ac0>, signals=[15]) 2024-09-17T07:54:27.042 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T07:54:27.047 INFO:teuthology.task.internal:Opening connections... 2024-09-17T07:54:27.047 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-09-17T07:54:27.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:54:27.127 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T07:54:27.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:54:27.203 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T07:54:27.209 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-09-17T07:54:27.227 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-09-17T07:54:27.227 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-09-17T07:54:27.285 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-09-17T07:54:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T07:54:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-09-17T07:54:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T07:54:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T07:54:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T07:54:27.287 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-09-17T07:54:27.316 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-09-17T07:54:27.334 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-09-17T07:54:27.335 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-09-17T07:54:27.393 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-09-17T07:54:27.393 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-09-17T07:54:27.393 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T07:54:27.394 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T07:54:27.395 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-09-17T07:54:27.416 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T07:54:27.424 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T07:54:27.429 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T07:54:27.429 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-09-17T07:54:27.431 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-09-17T07:54:27.451 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T07:54:27.457 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T07:54:27.457 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-09-17T07:54:27.488 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-09-17T07:54:27.516 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T07:54:27.569 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T07:54:27.584 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T07:54:27.584 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T07:54:27.584 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T07:54:27.585 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T07:54:27.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:54:27.602 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-17T07:54:27.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:54:27.603 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-17T07:54:27.659 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-503.el9.x86_64 2024-09-17T07:54:27.660 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-503.el9.x86_64 2024-09-17T07:54:27.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2024-09-17T07:54:27.662 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-503.el9.x86_64 2024-09-17T07:54:27.662 INFO:teuthology.task.kernel:Running kernel on smithi173: 5.14.0-503.el9.x86_64 2024-09-17T07:54:27.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum install -y kernel 2024-09-17T07:54:31.116 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-17T07:54:31.993 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-17T07:54:32.900 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-17T07:54:34.027 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-17T07:54:39.209 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-17T07:54:39.544 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-17T07:54:40.909 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-17T07:54:41.128 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-17T07:54:42.189 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-17T07:54:42.199 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-17T07:54:47.661 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-09-17T07:54:47.680 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-17T07:54:49.550 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T07:54:49.551 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T07:54:49.611 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T07:54:49.611 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T07:54:49.635 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T07:54:49.640 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T07:54:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-17T07:54:49.644 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-09-17T07:54:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:54:49.646 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 79 M 2024-09-17T07:54:49.646 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 126 M 2024-09-17T07:54:49.646 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T07:54:49.706 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T07:54:49.714 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout:Install 4 Packages 2024-09-17T07:54:49.715 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:54:49.716 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 79 M 2024-09-17T07:54:49.716 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 126 M 2024-09-17T07:54:49.716 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T07:54:49.854 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 433 kB/s | 36 kB 00:00 2024-09-17T07:54:49.940 INFO:teuthology.orchestra.run.smithi173.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 290 kB/s | 36 kB 00:00 2024-09-17T07:54:50.504 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-17T07:54:51.945 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-17T07:54:52.204 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-17T07:54:52.205 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:54:52.205 INFO:teuthology.orchestra.run.smithi079.stdout:Total 31 MB/s | 79 MB 00:02 2024-09-17T07:54:52.887 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T07:54:53.238 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T07:54:53.238 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T07:54:53.289 INFO:teuthology.orchestra.run.smithi173.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-09-17T07:54:54.225 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T07:54:54.225 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T07:54:55.331 INFO:teuthology.orchestra.run.smithi173.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.3 MB/s | 29 MB 00:05 2024-09-17T07:54:56.256 INFO:teuthology.orchestra.run.smithi173.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 5.4 MB/s | 35 MB 00:06 2024-09-17T07:54:56.256 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:54:56.257 INFO:teuthology.orchestra.run.smithi173.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-17T07:54:56.634 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T07:54:56.944 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T07:54:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:54:57.311 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T07:54:57.312 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T07:54:57.354 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:54:58.295 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T07:54:58.295 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T07:55:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:55:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:00.714 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T07:55:01.431 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:55:01.514 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:55:04.274 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:55:04.413 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:09.221 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:09.255 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:09.982 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:14.116 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:48.188 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:48.227 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:55:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:55:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:49.368 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:49.368 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:55:49.368 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T07:55:49.368 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.369 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.369 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.369 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.369 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:55:49.369 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T07:55:49.720 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:55:49.812 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.812 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:55:49.812 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T07:55:49.812 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T07:55:49.812 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T07:55:49.813 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T07:55:49.813 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T07:55:49.813 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:55:49.872 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:49.873 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:55:49.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 82.el9 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8274670 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-09-17T07:55:49.920 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-09-17T07:55:49.921 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T07:55:49.922 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T07:55:49.922 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T07:55:49.922 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 5.14.0-508.el9.x86_64 2024-09-17T07:55:49.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T07:55:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-09-17T07:55:51.956 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T07:55:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-09-17T07:55:51.977 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-17T07:55:51.992 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.8AzO07AJkV 2024-09-17T07:55:51.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8AzO07AJkV 2024-09-17T07:55:52.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.8AzO07AJkV 2024-09-17T07:55:52.204 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.8AzO07AJkV is 6KB 2024-09-17T07:55:52.253 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.8AzO07AJkV 2024-09-17T07:55:52.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T07:55:52.340 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T07:55:52.340 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T07:55:52.340 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T07:55:52.340 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T07:55:52.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T07:55:52.460 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:52.510 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:52.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-09-17T07:55:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:55:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:55:52.904 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:55:53.775 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:55:53.776 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T07:55:54.117 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:55:54.205 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:54.205 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:55:54.205 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T07:55:54.205 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T07:55:54.205 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T07:55:54.205 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T07:55:54.206 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T07:55:54.206 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:55:54.265 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:55:54.265 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:55:54.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -qi grub2-tools 2024-09-17T07:55:54.297 INFO:teuthology.orchestra.run.smithi173.stdout:Name : grub2-tools 2024-09-17T07:55:54.297 INFO:teuthology.orchestra.run.smithi173.stdout:Epoch : 1 2024-09-17T07:55:54.297 INFO:teuthology.orchestra.run.smithi173.stdout:Version : 2.06 2024-09-17T07:55:54.297 INFO:teuthology.orchestra.run.smithi173.stdout:Release : 82.el9 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: x86_64 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Group : Unspecified 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Size : 8274670 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:License : GPLv3+ 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Packager : builder@centos.org 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Vendor : CentOS 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Summary : Support tools for GRUB. 2024-09-17T07:55:54.298 INFO:teuthology.orchestra.run.smithi173.stdout:Description : 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout:hardware devices. 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T07:55:54.299 INFO:teuthology.orchestra.run.smithi173.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T07:55:54.300 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T07:55:54.300 INFO:teuthology.task.kernel:Updating grub on smithi173 to boot 5.14.0-508.el9.x86_64 2024-09-17T07:55:54.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T07:55:54.897 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-09-17T07:55:56.244 INFO:teuthology.orchestra.run.smithi173.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T07:55:56.275 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-09-17T07:55:56.277 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-09-17T07:55:56.293 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.TWP7cKEtGY 2024-09-17T07:55:56.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TWP7cKEtGY 2024-09-17T07:55:56.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.TWP7cKEtGY 2024-09-17T07:55:56.496 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.TWP7cKEtGY is 6KB 2024-09-17T07:55:56.545 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.TWP7cKEtGY 2024-09-17T07:55:56.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T07:55:56.623 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T07:55:56.623 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T07:55:56.623 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T07:55:56.623 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T07:55:56.625 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T07:55:56.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2024-09-17T07:56:22.549 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T07:56:22.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T07:56:22.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:56:26.781 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T07:56:26.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T07:56:26.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:56:40.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-17T07:56:45.248 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T07:56:49.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T07:56:49.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:56:52.957 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-17T07:56:54.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T07:56:54.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:56:57.309 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T07:57:04.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T07:57:04.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:57:09.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T07:57:09.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:57:27.741 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T07:57:36.966 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-17T07:57:37.365 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T07:57:37.365 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T07:57:37.366 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-17T07:57:37.397 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-508.el9.x86_64 2024-09-17T07:57:37.397 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T07:57:37.398 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T07:57:37.398 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T07:57:38.398 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T07:57:38.399 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T07:57:38.540 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-09-17T07:57:38.562 DEBUG:teuthology.parallel:result is None 2024-09-17T07:57:42.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T07:57:42.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:57:43.138 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-17T07:57:43.502 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T07:57:43.503 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T07:57:43.503 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-17T07:57:43.517 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-508.el9.x86_64 2024-09-17T07:57:43.518 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T07:57:43.518 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T07:57:43.518 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T07:57:44.519 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T07:57:44.519 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T07:57:44.612 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-09-17T07:57:44.637 DEBUG:teuthology.parallel:result is None 2024-09-17T07:57:44.638 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T07:57:44.647 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T07:57:44.648 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T07:57:44.651 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T07:57:44.682 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T07:57:44.689 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T07:57:44.696 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T07:57:44.696 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T07:57:44.710 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T07:57:44.797 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T07:57:44.803 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T07:57:44.803 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T07:57:44.806 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T07:57:44.842 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:57:44.853 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:57:45.023 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:57:45.039 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:57:45.041 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T07:57:45.048 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T07:57:45.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T07:57:45.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T07:57:45.108 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T07:57:45.117 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T07:57:45.117 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T07:57:45.120 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T07:57:45.153 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:57:45.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:57:45.269 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:57:45.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:57:45.395 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T07:57:45.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T07:57:45.462 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:57:45.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:57:45.585 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:57:45.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:57:45.716 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T07:57:45.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T07:57:45.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-09-17T07:57:45.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-09-17T07:57:45.840 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:57:46.050 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:57:46.643 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T07:57:46.651 INFO:teuthology.task.internal:Starting timer... 2024-09-17T07:57:46.651 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T07:57:46.663 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T07:57:46.673 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T07:57:46.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-09-17T07:57:46.729 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-09-17T07:57:46.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-09-17T07:57:46.786 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-09-17T07:57:46.787 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T07:57:46.788 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-09-17T07:57:46.815 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-09-17T07:57:46.815 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-09-17T07:57:46.838 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-09-17T07:57:46.838 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-09-17T07:57:46.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:57:46.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:57:46.883 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:57:46.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:57:46.908 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T07:57:46.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-09-17T07:57:46.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-09-17T07:57:46.993 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T07:57:47.002 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T07:57:47.014 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T07:57:47.015 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T08:01:30.467 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-09-17T08:01:30.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T08:01:30.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:01:30.549 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-17T08:01:30.632 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-17T08:01:30.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T08:01:30.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T08:01:30.710 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-17T08:01:30.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T08:01:30.794 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T08:01:30.804 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T08:01:30.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:01:30.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:01:30.807 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T08:01:30.807 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:01:30.833 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:01:30.847 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:01:30.866 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T08:01:30.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-09-17T08:01:30.881 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T08:01:30.882 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-09-17T08:01:30.897 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:01:30.907 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-09-17T08:01:30.911 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:01:30.918 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-09-17T08:01:30.933 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T08:01:30.948 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T08:01:30.966 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:01:30.969 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:01:30.969 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-17T08:01:30.969 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:30.969 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:30.970 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:30.970 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:31.001 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:01:31.004 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:01:31.004 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-17T08:01:31.004 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:31.004 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:31.005 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:31.005 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:01:31.005 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T08:01:31.015 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T08:01:31.016 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi079.front.sepia.ceph.com 2024-09-17T08:01:31.016 DEBUG:teuthology.orchestra.run.smithi079:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:01:31.016 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T08:01:31.017 DEBUG:teuthology.orchestra.run.smithi173:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T08:01:31.365 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:01:31.385 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:Removing: 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:Removing unused dependencies: 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:Remove 2 Packages 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:31.386 INFO:teuthology.orchestra.run.smithi079.stdout:Freed space: 5.5 M 2024-09-17T08:01:31.387 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:01:31.390 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:01:31.390 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:01:31.394 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:01:31.396 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:31.396 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Removing: 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Removing unused dependencies: 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Remove 2 Packages 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Freed space: 5.5 M 2024-09-17T08:01:31.397 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:01:31.401 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:01:31.401 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:01:31.450 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:01:31.450 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:01:31.482 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:01:31.482 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:01:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:01:31.620 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:01:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:01:31.724 INFO:teuthology.orchestra.run.smithi173.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T08:01:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:01:31.791 INFO:teuthology.orchestra.run.smithi173.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:01:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:01:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:01:32.319 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T08:01:32.320 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T08:01:32.476 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:01:32.477 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:32.477 INFO:teuthology.orchestra.run.smithi079.stdout:Removed: 2024-09-17T08:01:32.477 INFO:teuthology.orchestra.run.smithi079.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:01:32.477 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:32.477 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout:Removed: 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:32.634 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:01:33.311 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Sep 2024 08:00:45 AM UTC. 2024-09-17T08:01:33.447 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:01:33.448 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:33.448 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:01:33.448 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:33.448 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:01:33.448 INFO:teuthology.orchestra.run.smithi079.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout:Install 7 Packages 2024-09-17T08:01:33.449 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:33.455 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 5.2 M 2024-09-17T08:01:33.456 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 19 M 2024-09-17T08:01:33.456 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:01:33.495 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Sep 2024 08:00:45 AM UTC. 2024-09-17T08:01:33.633 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T08:01:33.634 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T08:01:33.635 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:33.635 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:01:33.635 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:01:33.635 INFO:teuthology.orchestra.run.smithi173.stdout:Install 7 Packages 2024-09-17T08:01:33.635 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:33.647 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 5.2 M 2024-09-17T08:01:33.647 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 19 M 2024-09-17T08:01:33.647 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:01:34.327 INFO:teuthology.orchestra.run.smithi079.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 520 kB/s | 44 kB 00:00 2024-09-17T08:01:34.364 INFO:teuthology.orchestra.run.smithi079.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 866 kB/s | 106 kB 00:00 2024-09-17T08:01:34.387 INFO:teuthology.orchestra.run.smithi079.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-17T08:01:34.412 INFO:teuthology.orchestra.run.smithi079.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.7 MB/s | 84 kB 00:00 2024-09-17T08:01:34.488 INFO:teuthology.orchestra.run.smithi079.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.6 MB/s | 906 kB 00:00 2024-09-17T08:01:34.563 INFO:teuthology.orchestra.run.smithi079.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.7 MB/s | 837 kB 00:00 2024-09-17T08:01:35.305 INFO:teuthology.orchestra.run.smithi079.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.5 MB/s | 3.2 MB 00:00 2024-09-17T08:01:35.305 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:01:35.305 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.8 MB/s | 5.2 MB 00:01 2024-09-17T08:01:35.415 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:01:35.430 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:01:35.430 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:01:35.745 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:01:35.746 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:01:36.217 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:01:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:01:36.352 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:01:36.387 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:01:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:01:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:01:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:01:37.228 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:37.504 INFO:teuthology.orchestra.run.smithi173.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 110 kB/s | 44 kB 00:00 2024-09-17T08:01:37.529 INFO:teuthology.orchestra.run.smithi173.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 246 kB/s | 106 kB 00:00 2024-09-17T08:01:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 574 kB/s | 72 kB 00:00 2024-09-17T08:01:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 673 kB/s | 84 kB 00:00 2024-09-17T08:01:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.4 MB/s | 906 kB 00:00 2024-09-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi173.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:01:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:01:38.328 INFO:teuthology.orchestra.run.smithi173.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.7 MB/s | 3.2 MB 00:00 2024-09-17T08:01:38.328 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:01:38.328 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.1 MB/s | 5.2 MB 00:04 2024-09-17T08:01:38.445 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:01:38.702 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:38.702 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:38.702 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:01:38.703 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:01:38.785 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:01:38.786 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:01:39.084 DEBUG:teuthology.parallel:result is None 2024-09-17T08:01:39.263 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:01:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T08:01:39.393 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T08:01:39.421 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T08:01:39.637 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T08:01:39.669 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T08:01:40.141 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:01:40.249 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T08:01:40.317 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:41.353 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:41.353 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T08:01:41.353 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T08:01:41.353 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T08:01:41.353 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T08:01:41.354 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T08:01:41.354 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T08:01:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T08:01:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:01:41.701 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:01:42.126 DEBUG:teuthology.parallel:result is None 2024-09-17T08:01:42.126 INFO:teuthology.run_tasks:Running task install... 2024-09-17T08:01:42.136 DEBUG:teuthology.task.install:project ceph 2024-09-17T08:01:42.136 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-17T08:01:42.136 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-17T08:01:42.136 INFO:teuthology.task.install:Using flavor: default 2024-09-17T08:01:42.142 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T08:01:42.142 INFO:teuthology.task.install:extra packages: [] 2024-09-17T08:01:42.142 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-17T08:01:42.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:01:42.145 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-17T08:01:42.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:01:42.295 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-17T08:01:42.295 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-17T08:01:42.301 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-17T08:01:42.301 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-17T08:01:42.411 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T08:01:42.411 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:01:42.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T08:01:42.415 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T08:01:42.415 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:01:42.415 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T08:01:42.442 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T08:01:42.443 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T08:01:42.444 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T08:01:42.444 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T08:01:42.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T08:01:42.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T08:01:42.586 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T08:01:42.594 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T08:01:42.652 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-09-17T08:01:42.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-09-17T08:01:42.659 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-09-17T08:01:42.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-09-17T08:01:42.879 INFO:teuthology.orchestra.run.smithi173.stdout:45 files removed 2024-09-17T08:01:42.900 INFO:teuthology.orchestra.run.smithi079.stdout:45 files removed 2024-09-17T08:01:42.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw 2024-09-17T08:01:42.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw 2024-09-17T08:01:43.454 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-17T08:01:43.490 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 276 kB/s | 77 kB 00:00 2024-09-17T08:01:43.722 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 51 kB/s | 11 kB 00:00 2024-09-17T08:01:43.783 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-17T08:01:43.989 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T08:01:44.051 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T08:01:44.676 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-17T08:01:45.839 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.3 MB 00:01 2024-09-17T08:01:50.926 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 20 MB 00:05 2024-09-17T08:01:56.927 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-17T08:01:58.686 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-17T08:01:59.640 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-17T08:02:03.830 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:16 2024-09-17T08:02:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-17T08:02:07.365 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:02:07.367 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T08:02:07.368 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T08:02:07.369 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout:Install 28 Packages 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 2 Packages 2024-09-17T08:02:07.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:07.371 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 53 M 2024-09-17T08:02:07.371 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:08.030 INFO:teuthology.orchestra.run.smithi079.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 9.9 MB/s | 5.1 MB 00:00 2024-09-17T08:02:08.089 INFO:teuthology.orchestra.run.smithi079.stdout:(2/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 430 kB/s | 25 kB 00:00 2024-09-17T08:02:08.190 INFO:teuthology.orchestra.run.smithi079.stdout:(3/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-17T08:02:08.216 INFO:teuthology.orchestra.run.smithi079.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 5.5 MB/s | 712 kB 00:00 2024-09-17T08:02:08.249 INFO:teuthology.orchestra.run.smithi079.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 7.0 MB/s | 477 kB 00:00 2024-09-17T08:02:08.308 INFO:teuthology.orchestra.run.smithi079.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 768 kB/s | 45 kB 00:00 2024-09-17T08:02:08.416 INFO:teuthology.orchestra.run.smithi079.stdout:(7/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 22 MB/s | 4.5 MB 00:00 2024-09-17T08:02:08.466 INFO:teuthology.orchestra.run.smithi079.stdout:(8/30): python3-ceph-common-18.2.4-855.g19e2a2b 815 kB/s | 129 kB 00:00 2024-09-17T08:02:08.491 INFO:teuthology.orchestra.run.smithi079.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 2.1 MB/s | 162 kB 00:00 2024-09-17T08:02:08.517 INFO:teuthology.orchestra.run.smithi079.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 6.2 MB/s | 321 kB 00:00 2024-09-17T08:02:08.551 INFO:teuthology.orchestra.run.smithi079.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-17T08:02:08.576 INFO:teuthology.orchestra.run.smithi079.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-17T08:02:08.768 INFO:teuthology.orchestra.run.smithi079.stdout:(13/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 15 MB/s | 18 MB 00:01 2024-09-17T08:02:08.818 INFO:teuthology.orchestra.run.smithi079.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 400 kB/s | 107 kB 00:00 2024-09-17T08:02:08.952 INFO:teuthology.orchestra.run.smithi079.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 426 kB/s | 160 kB 00:00 2024-09-17T08:02:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 216 kB/s | 45 kB 00:00 2024-09-17T08:02:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-17T08:02:09.070 INFO:teuthology.orchestra.run.smithi079.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 459 kB/s | 42 kB 00:00 2024-09-17T08:02:09.095 INFO:teuthology.orchestra.run.smithi079.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-17T08:02:09.130 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-17T08:02:09.145 INFO:teuthology.orchestra.run.smithi079.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-17T08:02:09.171 INFO:teuthology.orchestra.run.smithi079.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-17T08:02:09.196 INFO:teuthology.orchestra.run.smithi079.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T08:02:09.222 INFO:teuthology.orchestra.run.smithi079.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-17T08:02:09.247 INFO:teuthology.orchestra.run.smithi079.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T08:02:09.289 INFO:teuthology.orchestra.run.smithi079.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T08:02:09.323 INFO:teuthology.orchestra.run.smithi079.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-17T08:02:09.390 INFO:teuthology.orchestra.run.smithi079.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-17T08:02:09.507 INFO:teuthology.orchestra.run.smithi079.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T08:02:09.565 INFO:teuthology.orchestra.run.smithi079.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-17T08:02:09.624 INFO:teuthology.orchestra.run.smithi079.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-17T08:02:09.628 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:09.628 INFO:teuthology.orchestra.run.smithi079.stdout:Total 23 MB/s | 53 MB 00:02 2024-09-17T08:02:09.697 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:09.731 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:09.731 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:10.175 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:10.176 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:10.631 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-17T08:02:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:02:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-17T08:02:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-17T08:02:11.713 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-17T08:02:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-17T08:02:12.154 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:12.246 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:02:12.381 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:12.427 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:12.456 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-17T08:02:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-17T08:02:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:02:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:12.675 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:12.780 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-17T08:02:12.816 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-17T08:02:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-17T08:02:12.889 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-17T08:02:12.917 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-17T08:02:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-17T08:02:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-17T08:02:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-17T08:02:13.349 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:02:13.446 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:02:13.485 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-17T08:02:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-17T08:02:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-17T08:02:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:02:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-17T08:02:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T08:02:19.436 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:02:19.438 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T08:02:19.439 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-17T08:02:19.440 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:Install 28 Packages 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 2 Packages 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 53 M 2024-09-17T08:02:19.441 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:02:20.509 INFO:teuthology.orchestra.run.smithi173.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 8.4 MB/s | 5.1 MB 00:00 2024-09-17T08:02:20.609 INFO:teuthology.orchestra.run.smithi173.stdout:(2/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-17T08:02:20.635 INFO:teuthology.orchestra.run.smithi173.stdout:(3/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 201 kB/s | 25 kB 00:00 2024-09-17T08:02:20.668 INFO:teuthology.orchestra.run.smithi173.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 12 MB/s | 712 kB 00:00 2024-09-17T08:02:20.694 INFO:teuthology.orchestra.run.smithi173.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 7.9 MB/s | 477 kB 00:00 2024-09-17T08:02:20.744 INFO:teuthology.orchestra.run.smithi173.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 903 kB/s | 45 kB 00:00 2024-09-17T08:02:20.810 INFO:teuthology.orchestra.run.smithi173.stdout:(7/30): python3-ceph-common-18.2.4-855.g19e2a2b 1.9 MB/s | 129 kB 00:00 2024-09-17T08:02:20.869 INFO:teuthology.orchestra.run.smithi173.stdout:(8/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 22 MB/s | 4.5 MB 00:00 2024-09-17T08:02:20.894 INFO:teuthology.orchestra.run.smithi173.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 1.9 MB/s | 162 kB 00:00 2024-09-17T08:02:20.928 INFO:teuthology.orchestra.run.smithi173.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 5.4 MB/s | 321 kB 00:00 2024-09-17T08:02:20.972 INFO:teuthology.orchestra.run.smithi173.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 3.8 MB/s | 300 kB 00:00 2024-09-17T08:02:20.995 INFO:teuthology.orchestra.run.smithi173.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.4 MB/s | 99 kB 00:00 2024-09-17T08:02:21.163 INFO:teuthology.orchestra.run.smithi173.stdout:(13/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 15 MB/s | 18 MB 00:01 2024-09-17T08:02:21.221 INFO:teuthology.orchestra.run.smithi173.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 429 kB/s | 107 kB 00:00 2024-09-17T08:02:21.246 INFO:teuthology.orchestra.run.smithi173.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-09-17T08:02:21.271 INFO:teuthology.orchestra.run.smithi173.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 580 kB/s | 160 kB 00:00 2024-09-17T08:02:21.322 INFO:teuthology.orchestra.run.smithi173.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-17T08:02:21.339 INFO:teuthology.orchestra.run.smithi173.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 631 kB/s | 42 kB 00:00 2024-09-17T08:02:21.364 INFO:teuthology.orchestra.run.smithi173.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-17T08:02:21.414 INFO:teuthology.orchestra.run.smithi173.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.2 MB/s | 111 kB 00:00 2024-09-17T08:02:21.448 INFO:teuthology.orchestra.run.smithi173.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 824 kB/s | 26 kB 00:00 2024-09-17T08:02:21.473 INFO:teuthology.orchestra.run.smithi173.stdout:(22/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-17T08:02:21.499 INFO:teuthology.orchestra.run.smithi173.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-17T08:02:21.524 INFO:teuthology.orchestra.run.smithi173.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T08:02:21.566 INFO:teuthology.orchestra.run.smithi173.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T08:02:21.608 INFO:teuthology.orchestra.run.smithi173.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-17T08:02:21.700 INFO:teuthology.orchestra.run.smithi173.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-17T08:02:21.792 INFO:teuthology.orchestra.run.smithi173.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T08:02:21.850 INFO:teuthology.orchestra.run.smithi173.stdout:(29/30): librados2-18.2.4-855.g19e2a2be.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-17T08:02:21.967 INFO:teuthology.orchestra.run.smithi173.stdout:(30/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 11 MB/s | 3.0 MB 00:00 2024-09-17T08:02:21.972 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:21.972 INFO:teuthology.orchestra.run.smithi173.stdout:Total 21 MB/s | 53 MB 00:02 2024-09-17T08:02:22.039 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:02:22.074 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:02:22.074 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:02:22.530 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:02:22.530 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:02:23.870 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:02:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-17T08:02:23.998 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-17T08:02:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-17T08:02:24.373 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-17T08:02:24.480 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:24.564 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:24.648 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:02:24.690 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-17T08:02:24.808 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-17T08:02:24.878 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:02:24.909 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:25.082 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-17T08:02:25.117 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-17T08:02:25.165 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-17T08:02:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-17T08:02:25.233 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-17T08:02:25.269 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-17T08:02:25.302 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-17T08:02:25.614 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-17T08:02:25.657 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:02:25.748 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-17T08:02:25.779 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-17T08:02:25.822 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-17T08:02:26.228 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-17T08:02:26.259 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:02:26.330 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-17T08:02:26.398 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-17T08:02:26.428 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-09-17T08:02:26.998 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-09-17T08:02:26.999 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:02:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:02:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:02:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:28.526 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:28.566 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-17T08:02:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-17T08:02:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-17T08:02:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-17T08:02:29.104 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-17T08:02:29.145 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:02:29.239 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-17T08:02:29.239 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T08:02:29.239 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T08:02:29.239 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:02:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T08:02:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T08:02:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T08:02:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:30.255 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:30.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-test 2024-09-17T08:02:31.229 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:26 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:31.360 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T08:02:31.361 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:31.362 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:31.362 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:31.362 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-09-17T08:02:31.362 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:31.362 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 40 M 2024-09-17T08:02:31.363 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 164 M 2024-09-17T08:02:31.363 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:31.609 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 739 kB/s | 166 kB 00:00 2024-09-17T08:02:31.684 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.0 MB/s | 304 kB 00:00 2024-09-17T08:02:31.793 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-09-17T08:02:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 29 MB/s | 40 MB 00:01 2024-09-17T08:02:32.745 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:32.746 INFO:teuthology.orchestra.run.smithi079.stdout:Total 29 MB/s | 40 MB 00:01 2024-09-17T08:02:32.773 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:32.793 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:32.793 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:33.123 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:33.123 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:02:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T08:02:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T08:02:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:02:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:02:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-09-17T08:02:39.140 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-09-17T08:02:39.141 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-09-17T08:02:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:02:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-17T08:02:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-17T08:02:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T08:02:39.819 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:39.820 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:39.870 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:02:39.894 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-17T08:02:39.894 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:39.894 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T08:02:39.894 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:02:39.895 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T08:02:39.895 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:39.896 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:02:39.971 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-17T08:02:39.972 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:40.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph 2024-09-17T08:02:40.734 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:40.884 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-17T08:02:40.886 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T08:02:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T08:02:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout:Install 33 Packages 2024-09-17T08:02:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:40.890 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 29 M 2024-09-17T08:02:40.890 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 104 M 2024-09-17T08:02:40.890 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:41.147 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-17T08:02:41.148 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-17T08:02:41.149 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-17T08:02:41.150 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-17T08:02:41.276 INFO:teuthology.orchestra.run.smithi079.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 51 kB/s | 6.5 kB 00:00 2024-09-17T08:02:41.484 INFO:teuthology.orchestra.run.smithi079.stdout:(2/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 6.2 MB/s | 2.1 MB 00:00 2024-09-17T08:02:41.526 INFO:teuthology.orchestra.run.smithi079.stdout:(3/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 3.9 MB/s | 1.5 MB 00:00 2024-09-17T08:02:41.610 INFO:teuthology.orchestra.run.smithi079.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.9 MB/s | 246 kB 00:00 2024-09-17T08:02:41.735 INFO:teuthology.orchestra.run.smithi079.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-17T08:02:41.852 INFO:teuthology.orchestra.run.smithi079.stdout:(6/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 8.1 MB/s | 4.7 MB 00:00 2024-09-17T08:02:41.903 INFO:teuthology.orchestra.run.smithi079.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 7.3 MB/s | 1.2 MB 00:00 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:02:41.989 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T08:02:41.990 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T08:02:41.991 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T08:02:41.992 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:41.992 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi079.stdout:(8/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi079.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 425 kB/s | 106 kB 00:00 2024-09-17T08:02:42.129 INFO:teuthology.orchestra.run.smithi079.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 599 kB/s | 135 kB 00:00 2024-09-17T08:02:42.187 INFO:teuthology.orchestra.run.smithi079.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T08:02:42.213 INFO:teuthology.orchestra.run.smithi079.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 930 kB/s | 125 kB 00:00 2024-09-17T08:02:42.246 INFO:teuthology.orchestra.run.smithi079.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T08:02:42.305 INFO:teuthology.orchestra.run.smithi079.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 296 kB/s | 35 kB 00:00 2024-09-17T08:02:42.330 INFO:teuthology.orchestra.run.smithi079.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-09-17T08:02:42.355 INFO:teuthology.orchestra.run.smithi079.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 358 kB/s | 51 kB 00:00 2024-09-17T08:02:42.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-test 2024-09-17T08:02:42.406 INFO:teuthology.orchestra.run.smithi079.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 431 kB/s | 43 kB 00:00 2024-09-17T08:02:42.440 INFO:teuthology.orchestra.run.smithi079.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 340 kB/s | 11 kB 00:00 2024-09-17T08:02:42.465 INFO:teuthology.orchestra.run.smithi079.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-17T08:02:42.490 INFO:teuthology.orchestra.run.smithi079.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 349 kB/s | 18 kB 00:00 2024-09-17T08:02:42.516 INFO:teuthology.orchestra.run.smithi079.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 457 kB/s | 23 kB 00:00 2024-09-17T08:02:42.541 INFO:teuthology.orchestra.run.smithi079.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 385 kB/s | 19 kB 00:00 2024-09-17T08:02:42.566 INFO:teuthology.orchestra.run.smithi079.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 389 kB/s | 20 kB 00:00 2024-09-17T08:02:42.592 INFO:teuthology.orchestra.run.smithi079.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 916 kB/s | 46 kB 00:00 2024-09-17T08:02:42.618 INFO:teuthology.orchestra.run.smithi079.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T08:02:42.643 INFO:teuthology.orchestra.run.smithi079.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T08:02:42.668 INFO:teuthology.orchestra.run.smithi079.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-17T08:02:42.693 INFO:teuthology.orchestra.run.smithi079.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T08:02:42.719 INFO:teuthology.orchestra.run.smithi079.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T08:02:42.744 INFO:teuthology.orchestra.run.smithi079.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-17T08:02:42.769 INFO:teuthology.orchestra.run.smithi079.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-17T08:02:42.794 INFO:teuthology.orchestra.run.smithi079.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T08:02:42.828 INFO:teuthology.orchestra.run.smithi079.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-17T08:02:42.833 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:42.833 INFO:teuthology.orchestra.run.smithi079.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-17T08:02:42.955 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:42.961 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:25 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:02:42.994 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:42.994 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:43.093 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:02:43.093 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:43.093 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:02:43.093 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:43.093 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout:Install 4 Packages 2024-09-17T08:02:43.094 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:43.095 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 40 M 2024-09-17T08:02:43.095 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 164 M 2024-09-17T08:02:43.095 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:02:43.237 INFO:teuthology.orchestra.run.smithi173.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-17T08:02:43.279 INFO:teuthology.orchestra.run.smithi173.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T08:02:43.345 INFO:teuthology.orchestra.run.smithi173.stdout:(3/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 729 kB/s | 166 kB 00:00 2024-09-17T08:02:43.382 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:43.383 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:44.379 INFO:teuthology.orchestra.run.smithi173.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-17T08:02:44.381 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:44.381 INFO:teuthology.orchestra.run.smithi173.stdout:Total 31 MB/s | 40 MB 00:01 2024-09-17T08:02:44.408 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:02:44.427 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:02:44.427 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:02:44.623 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:02:44.717 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-17T08:02:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-17T08:02:44.755 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:02:44.756 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:02:44.787 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-17T08:02:44.879 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-17T08:02:44.968 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-17T08:02:45.016 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-17T08:02:45.052 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-17T08:02:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-17T08:02:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-17T08:02:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-17T08:02:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-17T08:02:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-17T08:02:45.242 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:02:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-17T08:02:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-17T08:02:45.324 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T08:02:45.370 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T08:02:45.398 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:02:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-17T08:02:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-17T08:02:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-17T08:02:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-17T08:02:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-17T08:02:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-17T08:02:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-17T08:02:46.032 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-17T08:02:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-17T08:02:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-17T08:02:46.254 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-17T08:02:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-17T08:02:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-17T08:02:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-17T08:02:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:02:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:02:46.640 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:46.640 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T08:02:46.641 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:02:46.641 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:02:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:48.608 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:02:48.633 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:02:48.634 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:48.634 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T08:02:48.634 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:02:48.634 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:02:48.634 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:48.843 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:02:48.867 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:48.930 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:02:49.093 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-17T08:02:49.162 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-17T08:02:50.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-17T08:02:50.691 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-17T08:02:50.692 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-17T08:02:51.135 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-17T08:02:51.136 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-17T08:02:51.136 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-17T08:02:51.136 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:51.430 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:02:51.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T08:02:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T08:02:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:51.766 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:52.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-base 2024-09-17T08:02:52.349 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:35 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:02:52.484 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:02:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T08:02:52.487 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T08:02:52.488 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:02:52.489 INFO:teuthology.orchestra.run.smithi173.stdout:Install 33 Packages 2024-09-17T08:02:52.490 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:52.490 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 29 M 2024-09-17T08:02:52.490 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 104 M 2024-09-17T08:02:52.490 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:02:52.735 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:47 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:52.824 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:02:52.866 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:52.867 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:02:52.867 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:52.947 INFO:teuthology.orchestra.run.smithi173.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-17T08:02:52.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install cephadm 2024-09-17T08:02:53.148 INFO:teuthology.orchestra.run.smithi173.stdout:(2/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.5 MB/s | 1.5 MB 00:00 2024-09-17T08:02:53.190 INFO:teuthology.orchestra.run.smithi173.stdout:(3/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 5.8 MB/s | 2.1 MB 00:00 2024-09-17T08:02:53.290 INFO:teuthology.orchestra.run.smithi173.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.4 MB/s | 246 kB 00:00 2024-09-17T08:02:53.365 INFO:teuthology.orchestra.run.smithi173.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-17T08:02:53.528 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:53.532 INFO:teuthology.orchestra.run.smithi173.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-17T08:02:53.583 INFO:teuthology.orchestra.run.smithi173.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-17T08:02:53.659 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:02:53.660 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:53.661 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 226 k 2024-09-17T08:02:53.661 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 217 k 2024-09-17T08:02:53.661 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:53.750 INFO:teuthology.orchestra.run.smithi173.stdout:(8/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 27 MB/s | 17 MB 00:00 2024-09-17T08:02:53.800 INFO:teuthology.orchestra.run.smithi173.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 623 kB/s | 135 kB 00:00 2024-09-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi079.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi173.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi173.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 786 kB/s | 125 kB 00:00 2024-09-17T08:02:53.951 INFO:teuthology.orchestra.run.smithi173.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-09-17T08:02:53.956 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:02:53.976 INFO:teuthology.orchestra.run.smithi173.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 520 kB/s | 35 kB 00:00 2024-09-17T08:02:54.001 INFO:teuthology.orchestra.run.smithi173.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T08:02:54.026 INFO:teuthology.orchestra.run.smithi173.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-17T08:02:54.052 INFO:teuthology.orchestra.run.smithi173.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-17T08:02:54.102 INFO:teuthology.orchestra.run.smithi173.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-17T08:02:54.128 INFO:teuthology.orchestra.run.smithi173.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 462 kB/s | 11 kB 00:00 2024-09-17T08:02:54.152 INFO:teuthology.orchestra.run.smithi173.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-17T08:02:54.178 INFO:teuthology.orchestra.run.smithi173.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-17T08:02:54.203 INFO:teuthology.orchestra.run.smithi173.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-17T08:02:54.228 INFO:teuthology.orchestra.run.smithi173.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-17T08:02:54.254 INFO:teuthology.orchestra.run.smithi173.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-17T08:02:54.279 INFO:teuthology.orchestra.run.smithi173.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-17T08:02:54.304 INFO:teuthology.orchestra.run.smithi173.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T08:02:54.338 INFO:teuthology.orchestra.run.smithi173.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T08:02:54.363 INFO:teuthology.orchestra.run.smithi173.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-09-17T08:02:54.389 INFO:teuthology.orchestra.run.smithi173.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T08:02:54.413 INFO:teuthology.orchestra.run.smithi173.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-17T08:02:54.416 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:02:54.456 INFO:teuthology.orchestra.run.smithi173.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-17T08:02:54.481 INFO:teuthology.orchestra.run.smithi173.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 795 kB/s | 20 kB 00:00 2024-09-17T08:02:54.506 INFO:teuthology.orchestra.run.smithi173.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-17T08:02:54.508 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:02:54.531 INFO:teuthology.orchestra.run.smithi173.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.5 MB/s | 427 kB 00:00 2024-09-17T08:02:54.535 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:54.536 INFO:teuthology.orchestra.run.smithi173.stdout:Total 14 MB/s | 29 MB 00:02 2024-09-17T08:02:54.652 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:02:54.688 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:02:54.688 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:02:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:02:55.094 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:02:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:02:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:55.094 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:02:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:02:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:55.095 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:55.095 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:02:55.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T08:02:55.881 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:56.017 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:56.018 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-17T08:02:56.018 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:56.018 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 143 k 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 395 k 2024-09-17T08:02:56.019 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:56.228 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 682 kB/s | 143 kB 00:00 2024-09-17T08:02:56.229 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:56.229 INFO:teuthology.orchestra.run.smithi079.stdout:Total 678 kB/s | 143 kB 00:00 2024-09-17T08:02:56.229 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:56.237 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:56.237 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:56.297 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:56.297 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:56.322 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:02:56.392 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-17T08:02:56.424 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-17T08:02:56.455 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:02:56.466 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-17T08:02:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:02:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:02:56.554 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:56.554 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T08:02:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-17T08:02:56.643 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-17T08:02:56.684 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-17T08:02:56.728 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-17T08:02:56.758 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-17T08:02:56.793 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-17T08:02:56.826 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-17T08:02:56.866 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-17T08:02:56.909 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-17T08:02:56.946 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-17T08:02:56.994 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-17T08:02:57.076 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-17T08:02:57.223 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-17T08:02:57.319 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-17T08:02:57.372 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-17T08:02:57.410 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:57.454 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:57.468 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-17T08:02:57.523 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-17T08:02:57.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr 2024-09-17T08:02:57.722 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-17T08:02:57.773 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-17T08:02:57.852 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-17T08:02:57.941 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-17T08:02:57.981 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-17T08:02:58.107 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-17T08:02:58.247 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-17T08:02:58.310 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:02:58.336 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-17T08:02:58.336 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:58.336 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T08:02:58.337 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:02:58.337 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T08:02:58.337 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:58.340 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:02:58.385 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:58.386 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:02:58.386 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:02:58.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T08:02:59.038 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:02:59.170 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:02:59.170 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:02:59.170 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:59.170 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout:Install 5 Packages 2024-09-17T08:02:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:02:59.172 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 3.8 M 2024-09-17T08:02:59.172 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 87 M 2024-09-17T08:02:59.172 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:02:59.384 INFO:teuthology.orchestra.run.smithi079.stdout:(1/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 108 kB/s | 16 kB 00:00 2024-09-17T08:02:59.409 INFO:teuthology.orchestra.run.smithi079.stdout:(2/5): ceph-grafana-dashboards-18.2.4-855.g19e2 137 kB/s | 24 kB 00:00 2024-09-17T08:02:59.452 INFO:teuthology.orchestra.run.smithi079.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 459 kB/s | 31 kB 00:00 2024-09-17T08:02:59.476 INFO:teuthology.orchestra.run.smithi079.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T08:02:59.602 INFO:teuthology.orchestra.run.smithi079.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 9.5 MB/s | 3.5 MB 00:00 2024-09-17T08:02:59.606 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:02:59.606 INFO:teuthology.orchestra.run.smithi079.stdout:Total 8.7 MB/s | 3.8 MB 00:00 2024-09-17T08:02:59.606 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:02:59.613 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:02:59.613 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:02:59.708 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:02:59.708 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:02:59.760 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:02:59.788 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-17T08:02:59.788 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:02:59.789 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T08:02:59.789 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:02:59.789 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T08:02:59.789 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:02:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:00.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-17T08:03:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-17T08:03:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:03:00.237 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T08:03:00.261 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:00.470 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T08:03:00.494 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:00.559 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:03:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-17T08:03:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:03:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-17T08:03:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-17T08:03:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-17T08:03:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-17T08:03:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-17T08:03:02.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-17T08:03:02.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-17T08:03:02.262 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-17T08:03:02.262 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-17T08:03:02.262 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-17T08:03:02.263 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-17T08:03:02.263 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-17T08:03:02.263 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-17T08:03:02.263 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-17T08:03:02.353 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:03:02.354 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-17T08:03:02.354 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-17T08:03:02.354 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:03:02.354 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T08:03:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T08:03:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:02.617 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:02.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T08:03:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-17T08:03:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:03.188 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T08:03:03.189 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T08:03:03.190 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:03.191 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:03.411 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:03.544 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:03.545 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-17T08:03:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T08:03:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout:Install 12 Packages 2024-09-17T08:03:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:03.548 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 38 M 2024-09-17T08:03:03.548 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 212 M 2024-09-17T08:03:03.548 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:03.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-base 2024-09-17T08:03:04.152 INFO:teuthology.orchestra.run.smithi079.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 912 kB/s | 191 kB 00:00 2024-09-17T08:03:04.210 INFO:teuthology.orchestra.run.smithi079.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 563 kB/s | 33 kB 00:00 2024-09-17T08:03:04.225 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:47 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:04.235 INFO:teuthology.orchestra.run.smithi079.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-17T08:03:04.294 INFO:teuthology.orchestra.run.smithi079.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-09-17T08:03:04.315 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:04.357 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:04.358 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:04.358 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:04.378 INFO:teuthology.orchestra.run.smithi079.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-09-17T08:03:04.403 INFO:teuthology.orchestra.run.smithi079.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-17T08:03:04.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install cephadm 2024-09-17T08:03:04.504 INFO:teuthology.orchestra.run.smithi079.stdout:(7/12): ceph-mgr-diskprediction-local-18.2.4-85 13 MB/s | 7.4 MB 00:00 2024-09-17T08:03:04.529 INFO:teuthology.orchestra.run.smithi079.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-17T08:03:04.629 INFO:teuthology.orchestra.run.smithi079.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 20 MB/s | 4.9 MB 00:00 2024-09-17T08:03:04.654 INFO:teuthology.orchestra.run.smithi079.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-17T08:03:04.995 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:47 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:05.049 INFO:teuthology.orchestra.run.smithi079.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-17T08:03:05.125 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:05.126 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:05.126 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:05.126 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:05.126 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:05.126 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 226 k 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 217 k 2024-09-17T08:03:05.127 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:05.314 INFO:teuthology.orchestra.run.smithi079.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-09-17T08:03:05.314 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:05.315 INFO:teuthology.orchestra.run.smithi079.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-17T08:03:05.345 INFO:teuthology.orchestra.run.smithi173.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-17T08:03:05.345 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-17T08:03:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:05.351 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:05.351 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:05.354 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:05.354 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:05.668 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:05.695 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:05.695 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:05.807 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:03:05.897 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:03:06.210 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:06.211 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:06.257 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:03:06.513 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-17T08:03:06.513 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:06.514 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:06.514 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:06.514 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:06.514 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:06.765 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T08:03:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:07.045 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T08:03:07.100 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:03:07.204 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:03:07.238 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T08:03:07.353 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:07.423 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T08:03:07.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T08:03:07.487 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:07.488 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:07.488 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-17T08:03:07.488 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:07.488 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:07.488 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 143 k 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 395 k 2024-09-17T08:03:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T08:03:07.701 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 676 kB/s | 143 kB 00:00 2024-09-17T08:03:07.701 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:07.702 INFO:teuthology.orchestra.run.smithi173.stdout:Total 670 kB/s | 143 kB 00:00 2024-09-17T08:03:07.702 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:07.710 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:07.710 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:07.770 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:07.770 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:07.914 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:07.968 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:03:07.993 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:03:07.993 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:07.994 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T08:03:07.994 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:08.184 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T08:03:08.310 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T08:03:08.946 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-17T08:03:08.947 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:08.947 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:08.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:08.947 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:08.947 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:09.186 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr 2024-09-17T08:03:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:03:09.778 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:09.869 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:09.911 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:09.912 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:09.912 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:09.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T08:03:10.563 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:10.606 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T08:03:10.632 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:03:10.697 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:10.698 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout:Install 5 Packages 2024-09-17T08:03:10.699 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:10.700 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 3.8 M 2024-09-17T08:03:10.700 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 87 M 2024-09-17T08:03:10.700 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:10.912 INFO:teuthology.orchestra.run.smithi173.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 160 kB/s | 24 kB 00:00 2024-09-17T08:03:10.938 INFO:teuthology.orchestra.run.smithi173.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 93 kB/s | 16 kB 00:00 2024-09-17T08:03:10.980 INFO:teuthology.orchestra.run.smithi173.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-17T08:03:11.005 INFO:teuthology.orchestra.run.smithi173.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T08:03:11.213 INFO:teuthology.orchestra.run.smithi173.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 7.8 MB/s | 3.5 MB 00:00 2024-09-17T08:03:11.217 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:11.217 INFO:teuthology.orchestra.run.smithi173.stdout:Total 7.3 MB/s | 3.8 MB 00:00 2024-09-17T08:03:11.218 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:11.224 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:11.224 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:11.302 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:11.302 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:11.553 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:11.637 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-17T08:03:11.676 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-17T08:03:11.705 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:03:12.756 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-17T08:03:12.801 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:03:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:03:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-17T08:03:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:03:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:03:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T08:03:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T08:03:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:13.771 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:13.772 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:13.957 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-17T08:03:13.957 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-17T08:03:13.957 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-17T08:03:13.957 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-17T08:03:13.957 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T08:03:14.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T08:03:14.237 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:14.237 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:14.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-rook 2024-09-17T08:03:14.437 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T08:03:14.830 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:14.962 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T08:03:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:14.964 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:Install 13 Packages 2024-09-17T08:03:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 2.2 M 2024-09-17T08:03:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 26 M 2024-09-17T08:03:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:15.015 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:15.147 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T08:03:15.148 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:15.149 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-17T08:03:15.150 INFO:teuthology.orchestra.run.smithi173.stdout:Install 12 Packages 2024-09-17T08:03:15.150 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:15.150 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 38 M 2024-09-17T08:03:15.150 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 212 M 2024-09-17T08:03:15.150 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:15.468 INFO:teuthology.orchestra.run.smithi079.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 301 kB/s | 26 kB 00:00 2024-09-17T08:03:15.494 INFO:teuthology.orchestra.run.smithi079.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 173 kB/s | 19 kB 00:00 2024-09-17T08:03:15.553 INFO:teuthology.orchestra.run.smithi079.stdout:(3/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 291 kB/s | 49 kB 00:00 2024-09-17T08:03:15.603 INFO:teuthology.orchestra.run.smithi079.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-17T08:03:15.628 INFO:teuthology.orchestra.run.smithi079.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-17T08:03:15.662 INFO:teuthology.orchestra.run.smithi079.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 910 kB/s | 54 kB 00:00 2024-09-17T08:03:15.696 INFO:teuthology.orchestra.run.smithi079.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-17T08:03:15.721 INFO:teuthology.orchestra.run.smithi079.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 246 kB/s | 14 kB 00:00 2024-09-17T08:03:15.746 INFO:teuthology.orchestra.run.smithi079.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-17T08:03:15.797 INFO:teuthology.orchestra.run.smithi079.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T08:03:15.822 INFO:teuthology.orchestra.run.smithi079.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-09-17T08:03:15.847 INFO:teuthology.orchestra.run.smithi079.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T08:03:15.881 INFO:teuthology.orchestra.run.smithi079.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-17T08:03:15.883 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:15.883 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-17T08:03:15.948 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:15.962 INFO:teuthology.orchestra.run.smithi173.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-85 16 MB/s | 7.4 MB 00:00 2024-09-17T08:03:15.962 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:15.963 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:16.046 INFO:teuthology.orchestra.run.smithi173.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-09-17T08:03:16.050 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:16.052 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:16.071 INFO:teuthology.orchestra.run.smithi173.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 340 kB/s | 191 kB 00:00 2024-09-17T08:03:16.138 INFO:teuthology.orchestra.run.smithi173.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-17T08:03:16.188 INFO:teuthology.orchestra.run.smithi173.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-17T08:03:16.255 INFO:teuthology.orchestra.run.smithi173.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-17T08:03:16.314 INFO:teuthology.orchestra.run.smithi173.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-17T08:03:16.456 INFO:teuthology.orchestra.run.smithi173.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-17T08:03:16.489 INFO:teuthology.orchestra.run.smithi173.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-17T08:03:16.514 INFO:teuthology.orchestra.run.smithi173.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-09-17T08:03:16.568 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:16.631 INFO:teuthology.orchestra.run.smithi173.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 11 MB/s | 4.9 MB 00:00 2024-09-17T08:03:16.744 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T08:03:16.807 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T08:03:16.863 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T08:03:16.908 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T08:03:16.950 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T08:03:17.060 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T08:03:17.161 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T08:03:17.211 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T08:03:17.923 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T08:03:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T08:03:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T08:03:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T08:03:18.071 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:03:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:03:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-17T08:03:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T08:03:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T08:03:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T08:03:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T08:03:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T08:03:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:19.189 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:19.190 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:19.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T08:03:20.089 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:15 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:20.221 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T08:03:20.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout:Install 6 Packages 2024-09-17T08:03:20.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:20.224 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 7.0 M 2024-09-17T08:03:20.224 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 33 M 2024-09-17T08:03:20.224 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:20.720 INFO:teuthology.orchestra.run.smithi079.stdout:(1/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 621 kB/s | 139 kB 00:00 2024-09-17T08:03:20.773 INFO:teuthology.orchestra.run.smithi173.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-17T08:03:20.774 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:20.774 INFO:teuthology.orchestra.run.smithi173.stdout:Total 6.8 MB/s | 38 MB 00:05 2024-09-17T08:03:20.795 INFO:teuthology.orchestra.run.smithi079.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.7 MB/s | 548 kB 00:00 2024-09-17T08:03:20.820 INFO:teuthology.orchestra.run.smithi079.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-17T08:03:20.846 INFO:teuthology.orchestra.run.smithi079.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.0 MB/s | 74 kB 00:00 2024-09-17T08:03:21.136 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:21.162 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:21.162 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:21.288 INFO:teuthology.orchestra.run.smithi079.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 313 kB/s | 248 kB 00:00 2024-09-17T08:03:21.680 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:21.680 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:21.755 INFO:teuthology.orchestra.run.smithi079.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-17T08:03:21.757 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:21.757 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-17T08:03:21.829 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:21.841 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:21.841 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:21.940 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:21.941 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T08:03:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T08:03:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T08:03:22.568 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:03:22.672 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:03:22.714 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T08:03:22.911 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T08:03:22.938 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T08:03:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T08:03:23.021 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T08:03:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T08:03:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T08:03:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:03:23.675 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T08:03:23.793 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T08:03:24.043 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:03:24.044 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-17T08:03:24.044 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T08:03:24.044 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T08:03:24.044 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T08:03:24.044 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T08:03:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T08:03:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T08:03:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T08:03:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:24.321 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:24.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-fuse 2024-09-17T08:03:25.027 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:03:25.179 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:20 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:03:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout:Install 2 Packages 2024-09-17T08:03:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:25.314 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 932 k 2024-09-17T08:03:25.314 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.7 M 2024-09-17T08:03:25.314 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:25.825 INFO:teuthology.orchestra.run.smithi079.stdout:(1/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 3.2 MB/s | 852 kB 00:00 2024-09-17T08:03:26.086 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T08:03:26.129 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:03:26.192 INFO:teuthology.orchestra.run.smithi079.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 127 kB/s | 80 kB 00:00 2024-09-17T08:03:26.193 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:26.193 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.0 MB/s | 932 kB 00:00 2024-09-17T08:03:26.206 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:26.230 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:26.230 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:26.314 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:26.314 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:26.524 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:26.665 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T08:03:26.749 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:03:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:03:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:27.695 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:28.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-volume 2024-09-17T08:03:28.598 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:23 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:28.736 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-17T08:03:28.737 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:Install 2 Packages 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 342 k 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.5 M 2024-09-17T08:03:28.738 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:28.823 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-17T08:03:28.823 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T08:03:28.824 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T08:03:28.900 INFO:teuthology.orchestra.run.smithi079.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 671 kB/s | 77 kB 00:00 2024-09-17T08:03:29.025 INFO:teuthology.orchestra.run.smithi079.stdout:(2/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-17T08:03:29.026 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:29.026 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.2 MB/s | 342 kB 00:00 2024-09-17T08:03:29.037 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:29.046 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:29.046 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:29.085 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:29.085 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:29.294 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T08:03:29.294 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:29.294 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:29.294 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:29.294 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:29.295 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:29.424 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-17T08:03:29.461 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:03:29.477 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:03:29.477 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:29.477 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T08:03:29.477 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:29.657 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-rook 2024-09-17T08:03:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-17T08:03:30.253 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:13 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:30.386 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:30.387 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:30.387 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:30.387 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:30.387 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:30.387 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T08:03:30.388 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:Install 13 Packages 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 2.2 M 2024-09-17T08:03:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 26 M 2024-09-17T08:03:30.390 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:30.584 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:30.671 INFO:teuthology.orchestra.run.smithi173.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 242 kB/s | 19 kB 00:00 2024-09-17T08:03:30.688 INFO:teuthology.orchestra.run.smithi173.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 267 kB/s | 26 kB 00:00 2024-09-17T08:03:30.749 INFO:teuthology.orchestra.run.smithi173.stdout:(3/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 318 kB/s | 49 kB 00:00 2024-09-17T08:03:30.772 INFO:teuthology.orchestra.run.smithi173.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-17T08:03:30.797 INFO:teuthology.orchestra.run.smithi173.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-17T08:03:30.823 INFO:teuthology.orchestra.run.smithi173.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T08:03:30.865 INFO:teuthology.orchestra.run.smithi173.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 479 kB/s | 32 kB 00:00 2024-09-17T08:03:30.890 INFO:teuthology.orchestra.run.smithi173.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-17T08:03:30.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados-devel 2024-09-17T08:03:30.924 INFO:teuthology.orchestra.run.smithi173.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-17T08:03:30.957 INFO:teuthology.orchestra.run.smithi173.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.4 MB/s | 220 kB 00:00 2024-09-17T08:03:30.982 INFO:teuthology.orchestra.run.smithi173.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.7 MB/s | 90 kB 00:00 2024-09-17T08:03:31.007 INFO:teuthology.orchestra.run.smithi173.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-17T08:03:31.041 INFO:teuthology.orchestra.run.smithi173.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-17T08:03:31.044 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:31.044 INFO:teuthology.orchestra.run.smithi173.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-17T08:03:31.108 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:31.124 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:31.124 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:31.212 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:31.213 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:31.498 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:31.630 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:31.631 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:31.632 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:03:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:31.632 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 126 k 2024-09-17T08:03:31.632 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 456 k 2024-09-17T08:03:31.632 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:31.719 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:31.842 INFO:teuthology.orchestra.run.smithi079.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 605 kB/s | 126 kB 00:00 2024-09-17T08:03:31.843 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:31.843 INFO:teuthology.orchestra.run.smithi079.stdout:Total 600 kB/s | 126 kB 00:00 2024-09-17T08:03:31.843 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:31.848 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:31.848 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:31.889 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T08:03:31.911 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:31.912 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T08:03:32.007 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T08:03:32.052 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T08:03:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:32.093 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T08:03:32.196 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T08:03:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:32.298 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T08:03:32.345 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T08:03:32.639 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:32.887 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:32.888 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:32.888 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:32.888 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:32.888 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:32.888 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:33.073 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T08:03:33.134 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T08:03:33.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs2 2024-09-17T08:03:33.167 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T08:03:33.215 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T08:03:33.237 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:03:33.763 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:33.859 INFO:teuthology.orchestra.run.smithi079.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:33.904 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:33.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs-devel 2024-09-17T08:03:34.017 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T08:03:34.018 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T08:03:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T08:03:34.019 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T08:03:34.349 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T08:03:34.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:34.351 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:34.574 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:34.605 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T08:03:34.705 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:34.706 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:34.706 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-17T08:03:34.706 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:34.706 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 31 k 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 139 k 2024-09-17T08:03:34.707 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:34.856 INFO:teuthology.orchestra.run.smithi079.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 208 kB/s | 31 kB 00:00 2024-09-17T08:03:34.857 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:34.857 INFO:teuthology.orchestra.run.smithi079.stdout:Total 206 kB/s | 31 kB 00:00 2024-09-17T08:03:34.857 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:34.860 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:34.861 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:34.877 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:34.878 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:34.970 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:35.078 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:35.191 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:18 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:35.325 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:35.325 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:35.326 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:35.327 INFO:teuthology.orchestra.run.smithi173.stdout:Install 6 Packages 2024-09-17T08:03:35.327 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:35.327 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 7.0 M 2024-09-17T08:03:35.327 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 33 M 2024-09-17T08:03:35.327 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:35.437 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:35.666 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:35.710 INFO:teuthology.orchestra.run.smithi173.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-17T08:03:35.735 INFO:teuthology.orchestra.run.smithi173.stdout:(2/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 641 kB/s | 139 kB 00:00 2024-09-17T08:03:35.803 INFO:teuthology.orchestra.run.smithi173.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 866 kB/s | 58 kB 00:00 2024-09-17T08:03:35.845 INFO:teuthology.orchestra.run.smithi173.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-17T08:03:35.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados2 2024-09-17T08:03:35.870 INFO:teuthology.orchestra.run.smithi173.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-17T08:03:35.945 INFO:teuthology.orchestra.run.smithi173.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-17T08:03:35.947 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:35.948 INFO:teuthology.orchestra.run.smithi173.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-17T08:03:36.019 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:36.029 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:36.030 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:36.129 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:36.129 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:36.429 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:36.434 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:36.523 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:36.566 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:36.566 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:36.567 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:36.601 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T08:03:36.646 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T08:03:36.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librbd1 2024-09-17T08:03:37.166 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T08:03:37.231 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T08:03:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T08:03:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:03:37.363 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:37.364 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:37.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rados 2024-09-17T08:03:38.030 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:38.122 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:38.166 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:38.167 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:38.167 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T08:03:38.169 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T08:03:38.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rgw 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T08:03:38.442 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T08:03:38.443 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T08:03:38.443 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:38.443 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:38.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-fuse 2024-09-17T08:03:38.837 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:38.928 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:38.972 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:38.973 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:38.973 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:39.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-cephfs 2024-09-17T08:03:39.291 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:39.424 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:39.424 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:39.424 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:39.424 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:Install 2 Packages 2024-09-17T08:03:39.425 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:39.426 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 932 k 2024-09-17T08:03:39.426 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 2.7 M 2024-09-17T08:03:39.426 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:39.563 INFO:teuthology.orchestra.run.smithi173.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 908 kB/s | 80 kB 00:00 2024-09-17T08:03:39.636 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:39.724 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:39.747 INFO:teuthology.orchestra.run.smithi173.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 3.1 MB/s | 852 kB 00:00 2024-09-17T08:03:39.747 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:39.748 INFO:teuthology.orchestra.run.smithi173.stdout:Total 2.8 MB/s | 932 kB 00:00 2024-09-17T08:03:39.760 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:39.766 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:39.767 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:39.768 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:39.781 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:39.781 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:39.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rbd 2024-09-17T08:03:39.861 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:39.861 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:40.056 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:40.179 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T08:03:40.254 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:03:40.435 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:40.526 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:40.569 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:40.570 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-17T08:03:40.571 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:40.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-fuse 2024-09-17T08:03:40.933 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-17T08:03:40.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:41.225 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:41.237 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:41.369 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:41.370 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 87 k 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 227 k 2024-09-17T08:03:41.371 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:41.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-volume 2024-09-17T08:03:41.709 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 256 kB/s | 87 kB 00:00 2024-09-17T08:03:41.710 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:41.710 INFO:teuthology.orchestra.run.smithi079.stdout:Total 255 kB/s | 87 kB 00:00 2024-09-17T08:03:41.710 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:41.717 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:41.717 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:41.780 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:41.780 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:41.934 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:42.031 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:42.125 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:25 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:42.264 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T08:03:42.265 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:Install 2 Packages 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 342 k 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 1.5 M 2024-09-17T08:03:42.266 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:42.518 INFO:teuthology.orchestra.run.smithi173.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 608 kB/s | 77 kB 00:00 2024-09-17T08:03:42.635 INFO:teuthology.orchestra.run.smithi173.stdout:(2/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-17T08:03:42.635 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:42.635 INFO:teuthology.orchestra.run.smithi173.stdout:Total 923 kB/s | 342 kB 00:00 2024-09-17T08:03:42.647 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:42.657 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:42.657 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:42.669 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:42.696 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:42.696 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:42.842 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:42.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-mirror 2024-09-17T08:03:43.033 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-17T08:03:43.070 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:03:43.086 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-17T08:03:43.086 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:43.086 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T08:03:43.086 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:43.480 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:43.617 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 3.0 M 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 12 M 2024-09-17T08:03:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:43.861 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-17T08:03:43.962 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2024-09-17T08:03:43.963 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:43.963 INFO:teuthology.orchestra.run.smithi079.stdout:Total 8.7 MB/s | 3.0 MB 00:00 2024-09-17T08:03:43.964 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:43.972 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:43.972 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:44.057 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:44.057 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:44.181 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:44.182 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:44.481 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados-devel 2024-09-17T08:03:44.485 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:44.555 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:03:44.580 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:45.077 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:45.211 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:45.211 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-17T08:03:45.211 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:45.213 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 126 k 2024-09-17T08:03:45.213 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 456 k 2024-09-17T08:03:45.213 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:45.433 INFO:teuthology.orchestra.run.smithi173.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 576 kB/s | 126 kB 00:00 2024-09-17T08:03:45.433 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:45.433 INFO:teuthology.orchestra.run.smithi173.stdout:Total 572 kB/s | 126 kB 00:00 2024-09-17T08:03:45.433 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:45.439 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:45.439 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:45.502 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:45.502 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:45.662 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:45.736 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:45.737 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:45.763 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:45.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-nbd 2024-09-17T08:03:46.187 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:46.474 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:46.585 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 08:02:05 AM UTC. 2024-09-17T08:03:46.717 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:46.718 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 172 k 2024-09-17T08:03:46.719 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 494 k 2024-09-17T08:03:46.719 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-17T08:03:46.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs2 2024-09-17T08:03:46.933 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 803 kB/s | 172 kB 00:00 2024-09-17T08:03:46.933 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:46.934 INFO:teuthology.orchestra.run.smithi079.stdout:Total 797 kB/s | 172 kB 00:00 2024-09-17T08:03:46.934 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-17T08:03:46.942 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-17T08:03:46.942 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-17T08:03:47.006 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-17T08:03:47.006 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-17T08:03:47.155 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-17T08:03:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:47.319 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:47.413 INFO:teuthology.orchestra.run.smithi173.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:47.459 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:47.460 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:47.460 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:47.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs-devel 2024-09-17T08:03:47.687 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:03:47.943 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-17T08:03:48.102 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:48.171 DEBUG:teuthology.parallel:result is None 2024-09-17T08:03:48.234 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 31 k 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 139 k 2024-09-17T08:03:48.235 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:48.387 INFO:teuthology.orchestra.run.smithi173.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 205 kB/s | 31 kB 00:00 2024-09-17T08:03:48.387 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:48.388 INFO:teuthology.orchestra.run.smithi173.stdout:Total 204 kB/s | 31 kB 00:00 2024-09-17T08:03:48.388 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:48.391 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:48.391 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:48.409 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:48.410 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:48.503 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:48.601 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:48.950 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:49.172 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:49.353 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados2 2024-09-17T08:03:49.929 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:50.017 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:50.060 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:50.061 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:50.061 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:50.136 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librbd1 2024-09-17T08:03:50.713 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:50.801 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:50.843 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:50.845 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:50.845 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:50.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rados 2024-09-17T08:03:51.494 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:51.585 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:51.628 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:51.629 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:51.629 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:51.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rgw 2024-09-17T08:03:52.269 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:52.360 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:52.404 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:52.405 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:52.405 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:52.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-cephfs 2024-09-17T08:03:53.052 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:53.140 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:53.183 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:53.184 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:53.184 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:53.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rbd 2024-09-17T08:03:53.827 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:53.917 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-17T08:03:53.960 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:53.961 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-17T08:03:53.961 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:54.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-fuse 2024-09-17T08:03:54.613 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:54.744 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:54.745 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 87 k 2024-09-17T08:03:54.746 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 227 k 2024-09-17T08:03:54.746 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:54.940 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 446 kB/s | 87 kB 00:00 2024-09-17T08:03:54.941 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:54.941 INFO:teuthology.orchestra.run.smithi173.stdout:Total 443 kB/s | 87 kB 00:00 2024-09-17T08:03:54.941 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:54.947 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:54.947 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:55.010 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:55.010 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:55.170 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:55.269 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:55.671 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:55.963 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:55.963 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:55.963 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:55.963 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:55.963 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:55.964 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:56.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-mirror 2024-09-17T08:03:56.843 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:56.987 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:56.988 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:56.989 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 3.0 M 2024-09-17T08:03:56.989 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 12 M 2024-09-17T08:03:56.989 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:03:57.293 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 9.8 MB/s | 3.0 MB 00:00 2024-09-17T08:03:57.294 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:03:57.294 INFO:teuthology.orchestra.run.smithi173.stdout:Total 9.8 MB/s | 3.0 MB 00:00 2024-09-17T08:03:57.294 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:03:57.304 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:03:57.304 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:03:57.388 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:03:57.389 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:03:57.823 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:03:57.886 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T08:03:57.913 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:59.013 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:03:59.013 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:59.013 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:03:59.013 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:03:59.014 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:59.014 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:03:59.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-nbd 2024-09-17T08:03:59.859 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 08:02:17 AM UTC. 2024-09-17T08:03:59.993 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-17T08:03:59.994 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:59.994 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-17T08:03:59.994 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:59.994 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 172 k 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 494 k 2024-09-17T08:03:59.995 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-17T08:04:00.189 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 890 kB/s | 172 kB 00:00 2024-09-17T08:04:00.189 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:04:00.190 INFO:teuthology.orchestra.run.smithi173.stdout:Total 883 kB/s | 172 kB 00:00 2024-09-17T08:04:00.190 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-17T08:04:00.197 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-17T08:04:00.197 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-17T08:04:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-17T08:04:00.261 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-17T08:04:00.416 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-17T08:04:00.519 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:04:00.957 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:04:01.246 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-17T08:04:01.246 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:04:01.246 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-17T08:04:01.247 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-17T08:04:01.247 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:04:01.247 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-17T08:04:01.484 DEBUG:teuthology.parallel:result is None 2024-09-17T08:04:01.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:01.627 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T08:04:01.653 INFO:teuthology.orchestra.run.smithi079.stdout:18.2.4-855.g19e2a2be.el9 2024-09-17T08:04:01.654 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-17T08:04:01.654 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-17T08:04:01.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:01.797 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T08:04:01.820 INFO:teuthology.orchestra.run.smithi173.stdout:18.2.4-855.g19e2a2be.el9 2024-09-17T08:04:01.820 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-17T08:04:01.820 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-17T08:04:01.822 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T08:04:01.822 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:04:01.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T08:04:01.854 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:04:01.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T08:04:01.887 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T08:04:01.887 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:04:01.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T08:04:01.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T08:04:01.990 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:04:01.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T08:04:02.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T08:04:02.082 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T08:04:02.082 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:04:02.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T08:04:02.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T08:04:02.175 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:04:02.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T08:04:02.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T08:04:02.264 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T08:04:02.265 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:04:02.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T08:04:02.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T08:04:02.355 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:04:02.356 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T08:04:02.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T08:04:02.448 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T08:04:02.546 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'cephadm_mode': 'root'} 2024-09-17T08:04:02.547 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:02.547 INFO:tasks.cephadm:Cluster fsid is 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:02.547 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T08:04:02.548 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.79', 'mon.c': '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]', 'mon.b': '172.21.15.173'} 2024-09-17T08:04:02.548 INFO:tasks.cephadm:First mon is mon.a on smithi079 2024-09-17T08:04:02.548 INFO:tasks.cephadm:First mgr is y 2024-09-17T08:04:02.548 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T08:04:02.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-09-17T08:04:02.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-09-17T08:04:02.599 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-17T08:04:02.599 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:02.746 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56094/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-855-g19e2a2be', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-855.g19e2a2be'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-16 16:29:21.739533', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-17T08:04:02.892 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:02.894 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-17T08:04:02.894 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.205 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 08:04 /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.206 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.519 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 08:04 /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.520 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.539 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T08:04:03.569 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 on all hosts... 2024-09-17T08:04:03.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:04:03.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T08:04:05.121 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:04:05.283 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:04:31.881 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:04:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:04:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:04:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-09-17T08:04:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:04:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:04:31.882 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:04:31.918 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:04:31.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-09-17T08:04:31.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-09-17T08:04:31.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-09-17T08:04:32.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-09-17T08:04:32.061 INFO:tasks.cephadm:Writing seed config... 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-17T08:04:32.063 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T08:04:32.064 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-17T08:04:32.065 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:04:32.065 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T08:04:32.087 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 66e8930a-74cb-11ef-bceb-c7b262605968 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-17T08:04:32.087 DEBUG:teuthology.orchestra.run.smithi079:mon.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T08:04:32.130 DEBUG:teuthology.orchestra.run.smithi079:mgr.y> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T08:04:32.173 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T08:04:32.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 -v bootstrap --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.79 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:04:32.325 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-17T08:04:32.325 INFO:teuthology.orchestra.run.smithi079.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', '-v', 'bootstrap', '--fsid', '66e8930a-74cb-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.79', '--skip-admin-label'] 2024-09-17T08:04:32.349 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:04:32.350 INFO:teuthology.orchestra.run.smithi079.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T08:04:32.350 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying podman|docker is present... 2024-09-17T08:04:32.371 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:04:32.371 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying lvm2 is present... 2024-09-17T08:04:32.372 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying time synchronization is in place... 2024-09-17T08:04:32.379 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:04:32.379 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:04:32.385 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:04:32.386 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:04:32.392 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout enabled 2024-09-17T08:04:32.398 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout active 2024-09-17T08:04:32.398 INFO:teuthology.orchestra.run.smithi079.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:04:32.399 INFO:teuthology.orchestra.run.smithi079.stdout:Repeating the final host check... 2024-09-17T08:04:32.420 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 5.2.2 2024-09-17T08:04:32.420 INFO:teuthology.orchestra.run.smithi079.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T08:04:32.420 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl is present 2024-09-17T08:04:32.421 INFO:teuthology.orchestra.run.smithi079.stdout:lvcreate is present 2024-09-17T08:04:32.427 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T08:04:32.427 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T08:04:32.433 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T08:04:32.433 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:04:32.439 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout enabled 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout active 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:Unit chronyd.service is enabled and running 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:Host looks OK 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:Cluster fsid: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:Acquiring lock 139947189857872 on /run/cephadm/66e8930a-74cb-11ef-bceb-c7b262605968.lock 2024-09-17T08:04:32.446 INFO:teuthology.orchestra.run.smithi079.stdout:Lock 139947189857872 acquired on /run/cephadm/66e8930a-74cb-11ef-bceb-c7b262605968.lock 2024-09-17T08:04:32.447 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying IP 172.21.15.79 port 3300 ... 2024-09-17T08:04:32.447 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying IP 172.21.15.79 port 6789 ... 2024-09-17T08:04:32.447 INFO:teuthology.orchestra.run.smithi079.stdout:Base mon IP(s) is [172.21.15.79:3300, 172.21.15.79:6789], mon addrv is [v2:172.21.15.79:3300,v1:172.21.15.79:6789] 2024-09-17T08:04:32.450 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.79 metric 100 2024-09-17T08:04:32.450 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.79 metric 100 2024-09-17T08:04:32.452 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T08:04:32.452 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-17T08:04:32.452 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce87/64 scope link noprefixroute 2024-09-17T08:04:32.455 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T08:04:32.456 INFO:teuthology.orchestra.run.smithi079.stdout:Mon IP `172.21.15.79` is in CIDR network `172.21.0.0/20` 2024-09-17T08:04:32.456 INFO:teuthology.orchestra.run.smithi079.stdout:Mon IP `172.21.15.79` is in CIDR network `172.21.0.0/20` 2024-09-17T08:04:32.456 INFO:teuthology.orchestra.run.smithi079.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T08:04:32.456 INFO:teuthology.orchestra.run.smithi079.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T08:04:32.457 INFO:teuthology.orchestra.run.smithi079.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stdout 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying blob sha256:d8a707c21dc06a64315160cb2d8e524fa91943a344731b2a930ee3a716bebbdd 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Copying config sha256:980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T08:04:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T08:04:33.837 INFO:teuthology.orchestra.run.smithi079.stdout:ceph: stdout ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:04:33.837 INFO:teuthology.orchestra.run.smithi079.stdout:Ceph version: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T08:04:33.838 INFO:teuthology.orchestra.run.smithi079.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T08:04:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:stat: stdout 167 167 2024-09-17T08:04:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial keys... 2024-09-17T08:04:35.640 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQATOOlmZ0cTAxAAetJqS/prGvydVQbbExFGlg== 2024-09-17T08:04:36.646 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQAUOOlmK+XpABAAkI7SjsEWXh0JBGYyDy8Ozg== 2024-09-17T08:04:37.727 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph-authtool: stdout AQAVOOlm8sJOAxAAMqTySYmtY+D/fMLK16lRHA== 2024-09-17T08:04:37.728 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial monmap... 2024-09-17T08:04:38.707 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool for a [v2:172.21.15.79:3300,v1:172.21.15.79:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = pacific 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: set fsid to 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:04:38.708 INFO:teuthology.orchestra.run.smithi079.stdout:Creating mon... 2024-09-17T08:04:39.646 INFO:teuthology.orchestra.run.smithi079.stdout:create mon.a on 2024-09-17T08:04:39.860 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-17T08:04:40.006 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T08:04:40.199 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target. 2024-09-17T08:04:40.200 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target. 2024-09-17T08:04:40.414 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:04:40.414 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T08:04:40.582 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target.wants/ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968@.service. 2024-09-17T08:04:41.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:41 smithi079 ceph-mon[29098]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:04:41.089 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:41 smithi079 bash[29060]: de6f9a94e4af8d989423da6040c99ca45dd45fd78ad78d8eb00a30ff65b7773c 2024-09-17T08:04:41.103 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:04:41.103 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-09-17T08:04:41.109 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:04:41.109 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:04:41.109 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-09-17T08:04:41.110 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:04:41.110 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mon to start... 2024-09-17T08:04:41.110 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mon... 2024-09-17T08:04:41.342 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:41 smithi079 systemd[1]: Started Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: monmap e1: 1 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: fsmap 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: mgrmap e1: no daemons active 2024-09-17T08:04:42.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:42 smithi079 ceph-mon[29098]: from='client.? 172.21.15.79:0/2106901453' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-17T08:04:42.231 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout id: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout services: 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.571292s) 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T08:04:42.233 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout data: 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:mon is available 2024-09-17T08:04:42.234 INFO:teuthology.orchestra.run.smithi079.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T08:04:43.310 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:43 smithi079 ceph-mon[29098]: from='client.? 172.21.15.79:0/3910384146' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:04:43.310 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:43 smithi079 ceph-mon[29098]: from='client.? 172.21.15.79:0/3910384146' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T08:04:43.313 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:43.313 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:04:43.313 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout fsid = 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:43.313 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.79:3300,v1:172.21.15.79:6789] 2024-09-17T08:04:43.313 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:04:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:Generating new minimal ceph.conf... 2024-09-17T08:04:44.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:44 smithi079 ceph-mon[29098]: from='client.? 172.21.15.79:0/1682206555' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:04:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:Restarting the monitor... 2024-09-17T08:04:44.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:44 smithi079 systemd[1]: Stopping Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:04:44.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:44 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29075]: 2024-09-17T08:04:44.662+0000 7faac0f6a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:04:44.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:44 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29075]: 2024-09-17T08:04:44.662+0000 7faac0f6a640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T08:04:45.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:44 smithi079 podman[29413]: 2024-09-17 08:04:44.847909107 +0000 UTC m=+0.287944290 container died de6f9a94e4af8d989423da6040c99ca45dd45fd78ad78d8eb00a30ff65b7773c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:04:45.479 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 podman[29413]: 2024-09-17 08:04:45.203220953 +0000 UTC m=+0.643256130 container cleanup de6f9a94e4af8d989423da6040c99ca45dd45fd78ad78d8eb00a30ff65b7773c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:04:45.479 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 bash[29413]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:04:45.821 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 podman[29430]: 2024-09-17 08:04:45.478902198 +0000 UTC m=+0.634100235 container remove de6f9a94e4af8d989423da6040c99ca45dd45fd78ad78d8eb00a30ff65b7773c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-17T08:04:45.821 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T08:04:45.821 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 systemd[1]: Stopped Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:04:45.822 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 systemd[1]: Starting Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:04:46.071 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 podman[29527]: 2024-09-17 08:04:45.723302063 +0000 UTC m=+0.015915963 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:46.071 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:45 smithi079 podman[29527]: 2024-09-17 08:04:45.821210329 +0000 UTC m=+0.113824210 container create d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 podman[29527]: 2024-09-17 08:04:46.040344242 +0000 UTC m=+0.332958133 container init d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 podman[29527]: 2024-09-17 08:04:46.045315931 +0000 UTC m=+0.337929814 container start d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b) 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: pidfile_write: ignore empty --pid-file 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: load: jerasure load: lrc 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Git sha 0 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: DB SUMMARY 2024-09-17T08:04:46.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: DB Session ID: TAQGZA4ES45QA6MDWY6G 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89930 ; 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.env: 0x559fcea0fbc0 2024-09-17T08:04:46.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.info_log: 0x559fd0d1abe0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.statistics: (nil) 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.use_fsync: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:04:46.074 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.db_log_dir: 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.wal_dir: 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:04:46.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.write_buffer_manager: 0x559fd0d28320 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.unordered_write: 0 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:04:46.076 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:04:46.077 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.row_cache: None 2024-09-17T08:04:46.077 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.wal_filter: None 2024-09-17T08:04:46.077 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:04:46.077 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:04:46.077 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:04:46.139 INFO:teuthology.orchestra.run.smithi079.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.wal_compression: 0 2024-09-17T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:04:46.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:04:46.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:04:46.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_open_files: -1 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:04:46.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Compression algorithms supported: 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kZSTD supported: 0 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:04:46.363 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.merge_operator: 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_filter: None 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:04:46.364 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559fd0d1a7e0) 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_type: 0 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: data_block_index_type: 0 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_shortening: 1 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:04:46.365 INFO:journalctl@ceph.mon.a.smithi079.stdout: checksum: 4 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: no_block_cache: 0 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache: 0x559fd0d0f350 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_options: 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: capacity : 536870912 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: num_shard_bits : 4 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: strict_capacity_limit : 0 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_cache_compressed: (nil) 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: persistent_cache: (nil) 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_size: 4096 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_size_deviation: 10 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_restart_interval: 16 2024-09-17T08:04:46.366 INFO:journalctl@ceph.mon.a.smithi079.stdout: index_block_restart_interval: 1 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: metadata_block_size: 4096 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: partition_filters: 0 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: use_delta_encoding: 1 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: filter_policy: bloomfilter 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: whole_key_filtering: 1 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: verify_compression: 0 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: format_version: 5 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: enable_index_compression: 1 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: block_align: 0 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: max_auto_readahead_size: 262144 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: prepopulate_block_cache: 0 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:04:46.367 INFO:journalctl@ceph.mon.a.smithi079.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:04:46.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:04:46.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression: NoCompression 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.num_levels: 7 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:04:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:04:46.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:04:46.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:04:46.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:04:46.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:04:46.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:04:46.375 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.ttl: 2592000 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:04:46.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c712dffe-03b5-4487-a515-65866a2c8714 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560286071327, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560286072712, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 86009, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 276, "table_properties": {"data_size": 84087, "index_size": 242, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11449, "raw_average_key_size": 48, "raw_value_size": 77749, "raw_average_value_size": 330, "num_data_blocks": 10, "num_entries": 235, "num_filter_entries": 235, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726560286, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c712dffe-03b5-4487-a515-65866a2c8714", "db_session_id": "TAQGZA4ES45QA6MDWY6G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560286072864, "job": 1, "event": "recovery_finished"} 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559fd0d3ae00 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: DB pointer 0x559fd0e2e000 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:04:46.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** DB Stats ** 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: L0 2/0 85.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:04:46.378 INFO:journalctl@ceph.mon.a.smithi079.stdout: Sum 2/0 85.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:04:46.379 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: Cumulative compaction: 0.00 GB write, 18.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: Interval compaction: 0.00 GB write, 18.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: Block cache BinnedLRUCache@0x559fd0d0f350#7 capacity: 512.00 MB usage: 1.27 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: starting mon.a rank 0 at public addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] at bind addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???) e1 preinit fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).mds e1 new map 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).mds e1 print_map 2024-09-17T08:04:46.380 INFO:journalctl@ceph.mon.a.smithi079.stdout: e1 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout: legacy client fscid: -1 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout: 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout: No filesystems configured 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: monmap e1: 1 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:04:46.381 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: fsmap 2024-09-17T08:04:46.382 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 bash[29527]: d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede 2024-09-17T08:04:46.382 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T08:04:46.382 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 ceph-mon[29565]: mgrmap e1: no daemons active 2024-09-17T08:04:46.382 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:46 smithi079 systemd[1]: Started Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:04:47.224 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T08:04:47.246 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:04:47.246 INFO:teuthology.orchestra.run.smithi079.stdout:Creating mgr... 2024-09-17T08:04:47.246 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T08:04:47.246 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T08:04:47.492 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:04:47.492 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T08:04:47.639 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968.target.wants/ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-66e8930a-74cb-11ef-bceb-c7b262605968@.service. 2024-09-17T08:04:47.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1668421114' entity='client.admin' 2024-09-17T08:04:47.865 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:47 smithi079 systemd[1]: Starting Ceph mgr.y for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:04:48.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:47 smithi079 podman[29805]: 2024-09-17 08:04:47.864629657 +0000 UTC m=+0.108374000 container create 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:04:48.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:47 smithi079 podman[29805]: 2024-09-17 08:04:47.78122816 +0000 UTC m=+0.024972499 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T08:04:48.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 podman[29805]: 2024-09-17 08:04:48.058760145 +0000 UTC m=+0.302504488 container init 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-17T08:04:48.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 podman[29805]: 2024-09-17 08:04:48.063609406 +0000 UTC m=+0.307353753 container start 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-17T08:04:48.182 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:04:48.182 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-09-17T08:04:48.189 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:04:48.189 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:04:48.189 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-09-17T08:04:48.189 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T08:04:48.197 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:04:48.197 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr to start... 2024-09-17T08:04:48.205 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr... 2024-09-17T08:04:48.395 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 bash[29805]: 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 2024-09-17T08:04:48.395 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 systemd[1]: Started Ceph mgr.y for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:04:48.395 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.181+0000 7f21889b0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:04:48.395 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.243+0000 7f21889b0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:04:48.395 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.393+0000 7f21889b0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:04:48.712 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.463+0000 7f21889b0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:04:48.712 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.613+0000 7f21889b0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:04:49.056 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3848237228' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:04:49.056 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:48.758+0000 7f21889b0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:04:49.321 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:49 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:49.075+0000 7f21889b0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:04:49.321 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:49 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:49.136+0000 7f21889b0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:04:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:04:49.346 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:04:49.347 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:04:49.350 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:04:49.350 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:04:49.350 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.350 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:04:49.351 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.352 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:04:41.066691+0000", 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:04:49.353 INFO:teuthology.orchestra.run.smithi079.stdout:mgr not available, waiting (1/15)... 2024-09-17T08:04:49.619 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:49 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:49.319+0000 7f21889b0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:04:50.619 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:50 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:50.207+0000 7f21889b0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:04:50.619 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:50 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:50.277+0000 7f21889b0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:04:51.275 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.018+0000 7f21889b0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:04:51.275 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.133+0000 7f21889b0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:04:51.276 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.274+0000 7f21889b0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:04:51.558 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.334+0000 7f21889b0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:04:51.559 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.488+0000 7f21889b0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:04:51.559 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.557+0000 7f21889b0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:04:51.848 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.618+0000 7f21889b0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:04:51.848 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.779+0000 7f21889b0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:04:51.848 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.847+0000 7f21889b0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:04:52.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:52 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1545758109' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:04:52.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:51 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:51.942+0000 7f21889b0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:04:52.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:52 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:52.030+0000 7f21889b0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:04:52.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:52 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:52.089+0000 7f21889b0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:04:52.495 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:52 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:52.150+0000 7f21889b0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:04:52.503 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:04:52.503 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:04:52.503 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:04:52.503 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:04:52.504 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:04:52.506 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:04:41.066691+0000", 2024-09-17T08:04:52.507 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:04:52.508 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:04:52.508 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:04:52.508 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:04:52.508 INFO:teuthology.orchestra.run.smithi079.stdout:mgr not available, waiting (2/15)... 2024-09-17T08:04:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:04:52 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:04:52.923+0000 7f21889b0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: Activating manager daemon y 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: mgrmap e2: y(active, starting, since 0.00286175s) 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: Manager daemon y is now available 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:04:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' 2024-09-17T08:04:53.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' 2024-09-17T08:04:53.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:53 smithi079 ceph-mon[29565]: from='mgr.14100 172.21.15.79:0/917014018' entity='mgr.y' 2024-09-17T08:04:55.168 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:54 smithi079 ceph-mon[29565]: mgrmap e3: y(active, since 1.00606s) 2024-09-17T08:04:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:55 smithi079 ceph-mon[29565]: mgrmap e4: y(active, since 2s) 2024-09-17T08:04:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:04:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/468030644' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T08:05:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 0 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:05:00.917 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T08:05:00.918 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T08:05:00.919 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ], 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T08:04:41.066691+0000", 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout }, 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:05:00.920 INFO:teuthology.orchestra.run.smithi079.stdout:mgr is available 2024-09-17T08:05:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3027811146' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout fsid = 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T08:05:06.542 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T08:05:06.543 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling cephadm module... 2024-09-17T08:05:07.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/767064594' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-17T08:05:07.535 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setuser ceph since I am not root 2024-09-17T08:05:07.535 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setgroup ceph since I am not root 2024-09-17T08:05:07.535 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.364+0000 7f0244b87040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:05:07.535 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.419+0000 7f0244b87040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:05:07.811 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.567+0000 7f0244b87040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:05:07.811 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.631+0000 7f0244b87040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:05:07.811 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.776+0000 7f0244b87040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:05:08.062 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:07.919+0000 7f0244b87040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:05:08.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/767064594' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T08:05:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:08 smithi079 ceph-mon[29565]: mgrmap e5: y(active, since 14s) 2024-09-17T08:05:08.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:08 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:08.229+0000 7f0244b87040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:05:08.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:08 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:08.290+0000 7f0244b87040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:05:08.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:08 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:08.469+0000 7f0244b87040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:05:09.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2087276585' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:05:09.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:09 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:09.355+0000 7f0244b87040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:05:09.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:09 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:09.423+0000 7f0244b87040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:05:09.920 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:05:09.921 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for the mgr to restart... 2024-09-17T08:05:09.921 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr epoch 5... 2024-09-17T08:05:10.244 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.136+0000 7f0244b87040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:05:10.600 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.252+0000 7f0244b87040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:05:10.600 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.395+0000 7f0244b87040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:05:10.600 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.455+0000 7f0244b87040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:05:10.600 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.598+0000 7f0244b87040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:05:10.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.665+0000 7f0244b87040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:05:10.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.722+0000 7f0244b87040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:05:11.141 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.874+0000 7f0244b87040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:05:11.141 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:10 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:10.938+0000 7f0244b87040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:05:11.141 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:11.002+0000 7f0244b87040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:05:11.142 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:11.082+0000 7f0244b87040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:05:11.142 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:11.140+0000 7f0244b87040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:05:11.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:11.198+0000 7f0244b87040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:05:12.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:11.955+0000 7f0244b87040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: Active manager daemon y restarted 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: Activating manager daemon y 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: mgrmap e6: y(active, starting, since 0.00325313s) 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: Manager daemon y is now available 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:12.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:12.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:12.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:12.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:05:12.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:12 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:05:13.305 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-mon[29565]: Found migration_current of "None". Setting to last migration. 2024-09-17T08:05:13.305 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:13.305 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:13.305 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-mon[29565]: mgrmap e7: y(active, since 1.00723s) 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:mgr epoch 5 is available 2024-09-17T08:05:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:Setting orchestrator backend to cephadm... 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:13.567+0000 7f01b20b2640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:05:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Traceback (most recent call last): 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._connections.run(self.expiration_interval) 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._run(expiration_interval) 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:05:13.822 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s = self.context.wrap_socket( 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: return self.sslsocket_class._create( 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self.do_handshake() 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._sslobj.do_handshake() 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:05:13.823 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:14.203 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:14 smithi079 ceph-mon[29565]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T08:05:14.203 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:14 smithi079 ceph-mon[29565]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T08:05:14.203 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:14 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:15.097 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:13] ENGINE Bus STARTING 2024-09-17T08:05:15.097 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:13] ENGINE Serving on https://172.21.15.79:7150 2024-09-17T08:05:15.097 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:13] ENGINE Error in HTTPServer.serve 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: Traceback (most recent call last): 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._run(expiration_interval) 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: s = self.context.wrap_socket( 2024-09-17T08:05:15.098 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: return self.sslsocket_class._create( 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: self.do_handshake() 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._sslobj.do_handshake() 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:13] ENGINE Serving on http://172.21.15.79:8765 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:13] ENGINE Bus STARTED 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: mgrmap e8: y(active, since 2s) 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:15.099 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:15 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:15.794 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T08:05:15.794 INFO:teuthology.orchestra.run.smithi079.stdout:Generating ssh key... 2024-09-17T08:05:16.194 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-mon[29565]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Generating public/private rsa key pair. 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Your identification has been saved in /tmp/tmpzykfdt9e/key 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Your public key has been saved in /tmp/tmpzykfdt9e/key.pub 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: The key fingerprint is: 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: SHA256:qSd2yHypzqHOZihFrcYelLbPfVM/tudnMc+tVe3corc ceph-66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: The key's randomart image is: 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: +---[RSA 3072]----+ 2024-09-17T08:05:16.620 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | | 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | | 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | o | 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | = . . .| 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | = o S o| 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | B o o .. =o| 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | + = .O =. . ..X| 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: |. o.=+.Bo +.oo*| 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: | . ++.+. . .o*E= | 2024-09-17T08:05:16.621 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: +----[SHA256]-----+ 2024-09-17T08:05:17.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:17 smithi079 ceph-mon[29565]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:17.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:17 smithi079 ceph-mon[29565]: Generating ssh key... 2024-09-17T08:05:17.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:17 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:17.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:17 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:18.123 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaWVPg5TVTEUdbpTsWfu+K8FhJ93sb4fgDrnMOvmxXqYw5b0vOMea8oMfoB/S10mJW/nGa/QSoD6OlCdGxjX4Awfm19gwnCBfdoN2/VT9RZ0Pviv9yu3L66qqvM2ExtqkAaa+5dnri5kvTpnZ5RllAu666w0UgWEXff2QkrJapcaSOTjEBW+NhxJo8FhS405SOUmllnK7oLzJVaX4RI3/zu2dZtDIKiqBBEWa43PdYgC9vUm1dr26kZ3GbrQ9huHx/qkhdLGFEjCux1FK+UHHYoPthhTlJf1zNVGjWHXDHqoqE4G3OnEzTqXW1+KwX2pcRgOu7J7UhtDdFemphhf2WRM9Cl9kiBW3wU9HlznSCC38m+d5D94H5W8bSHuWwcc+PqSImvWW78UA9DY6QIEA1S1K3zLj5PXU+qFKBHxlipKBZjB9C0DrzNhdQJqgdkmrW8sOIvMqlSwEmuLpJO2tIEhfWN8Pd7y39svYPxCarjLCwV8Bi2kjK+f6JMen2tG8= ceph-66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:05:18.123 INFO:teuthology.orchestra.run.smithi079.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:05:18.123 INFO:teuthology.orchestra.run.smithi079.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T08:05:18.132 INFO:teuthology.orchestra.run.smithi079.stdout:Adding host smithi079... 2024-09-17T08:05:18.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:18 smithi079 ceph-mon[29565]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:19.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:19 smithi079 ceph-mon[29565]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "addr": "172.21.15.79", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:20.530 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:20 smithi079 ceph-mon[29565]: Deploying cephadm binary to smithi079 2024-09-17T08:05:21.955 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Added host 'smithi079' with addr '172.21.15.79' 2024-09-17T08:05:21.955 INFO:teuthology.orchestra.run.smithi079.stdout:Deploying unmanaged mon service... 2024-09-17T08:05:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:22 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:22 smithi079 ceph-mon[29565]: Added host smithi079 2024-09-17T08:05:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:22 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:23.572 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T08:05:23.572 INFO:teuthology.orchestra.run.smithi079.stdout:Deploying unmanaged mgr service... 2024-09-17T08:05:23.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:23 smithi079 ceph-mon[29565]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:23.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:23 smithi079 ceph-mon[29565]: Saving service mon spec with placement count:5 2024-09-17T08:05:23.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:23 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:25 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:25 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:25 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:25.441 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T08:05:26.302 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:26 smithi079 ceph-mon[29565]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:26.302 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:26 smithi079 ceph-mon[29565]: Saving service mgr spec with placement count:2 2024-09-17T08:05:27.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:27 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:27.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:27 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1739035796' entity='client.admin' 2024-09-17T08:05:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:28 smithi079 ceph-mon[29565]: from='mgr.14118 172.21.15.79:0/3709944099' entity='mgr.y' 2024-09-17T08:05:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2162888479' entity='client.admin' 2024-09-17T08:05:28.892 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling the dashboard module... 2024-09-17T08:05:29.746 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1609022668' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-17T08:05:29.746 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setuser ceph since I am not root 2024-09-17T08:05:29.746 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setgroup ceph since I am not root 2024-09-17T08:05:29.746 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:29.743+0000 7f69f8090040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:05:30.022 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:29.804+0000 7f69f8090040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:05:30.023 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:29.952+0000 7f69f8090040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:05:30.023 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.020+0000 7f69f8090040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:05:30.309 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.166+0000 7f69f8090040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:05:30.309 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.307+0000 7f69f8090040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:05:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1609022668' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T08:05:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-mon[29565]: mgrmap e9: y(active, since 17s) 2024-09-17T08:05:30.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.629+0000 7f69f8090040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:05:30.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.691+0000 7f69f8090040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:05:31.354 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:30.881+0000 7f69f8090040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:05:31.661 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:05:31.661 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T08:05:31.661 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T08:05:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T08:05:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T08:05:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:05:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for the mgr to restart... 2024-09-17T08:05:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:Waiting for mgr epoch 9... 2024-09-17T08:05:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:31 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2219654170' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T08:05:31.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:31.767+0000 7f69f8090040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:05:31.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:31.835+0000 7f69f8090040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:05:32.803 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:32 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:32.549+0000 7f69f8090040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:05:32.803 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:32 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:32.662+0000 7f69f8090040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:05:33.074 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:32 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:32.801+0000 7f69f8090040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:05:33.074 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:32 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:32.860+0000 7f69f8090040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:05:33.074 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.005+0000 7f69f8090040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:05:33.074 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.072+0000 7f69f8090040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:05:33.349 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.130+0000 7f69f8090040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:05:33.350 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.283+0000 7f69f8090040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:05:33.350 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.348+0000 7f69f8090040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:05:33.612 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.413+0000 7f69f8090040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:05:33.612 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.494+0000 7f69f8090040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:05:33.612 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.552+0000 7f69f8090040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:05:33.612 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:33.610+0000 7f69f8090040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:05:34.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:34.380+0000 7f69f8090040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: Active manager daemon y restarted 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: Activating manager daemon y 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: mgrmap e10: y(active, starting, since 0.00311595s) 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: Manager daemon y is now available 2024-09-17T08:05:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:05:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:35.717+0000 7f695e51a640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:05:35] ENGINE Error in HTTPServer.serve 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Traceback (most recent call last): 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._connections.run(self.expiration_interval) 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._run(expiration_interval) 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:05:35.999 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s = self.context.wrap_socket( 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: return self.sslsocket_class._create( 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self.do_handshake() 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._sslobj.do_handshake() 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:05:36.000 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:05:35 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout { 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout } 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:mgr epoch 9 is available 2024-09-17T08:05:36.001 INFO:teuthology.orchestra.run.smithi079.stdout:Generating a dashboard self-signed certificate... 2024-09-17T08:05:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:36 smithi079 ceph-mon[29565]: mgrmap e11: y(active, since 1.00559s) 2024-09-17T08:05:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:36 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:35] ENGINE Bus STARTING 2024-09-17T08:05:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:36 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:35] ENGINE Serving on http://172.21.15.79:8765 2024-09-17T08:05:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-17T08:05:37.365 INFO:teuthology.orchestra.run.smithi079.stdout:Creating initial admin user... 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:35] ENGINE Serving on https://172.21.15.79:7150 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:35] ENGINE Bus STARTED 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: [17/Sep/2024:08:05:35] ENGINE Error in HTTPServer.serve 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout: Traceback (most recent call last): 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:05:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._run(expiration_interval) 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:05:37.621 INFO:journalctl@ceph.mon.a.smithi079.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: s = self.context.wrap_socket( 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: return self.sslsocket_class._create( 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: self.do_handshake() 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:05:37.622 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._sslobj.do_handshake() 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:37.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:37 smithi079 ceph-mon[29565]: mgrmap e12: y(active, since 2s) 2024-09-17T08:05:38.941 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$G73pPLokrzgibTn56HPTl.2uiaNjt5ywQZvZ.hvlZSmnN8P.Ws0dK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726560338, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T08:05:38.942 INFO:teuthology.orchestra.run.smithi079.stdout:Fetching dashboard port number... 2024-09-17T08:05:39.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:39 smithi079 ceph-mon[29565]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:39.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:40.255 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:40 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3729962843' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-17T08:05:40.582 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T08:05:40.592 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T08:05:40.592 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout disabled 2024-09-17T08:05:40.599 INFO:teuthology.orchestra.run.smithi079.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T08:05:40.599 INFO:teuthology.orchestra.run.smithi079.stdout:systemctl: stdout inactive 2024-09-17T08:05:40.599 INFO:teuthology.orchestra.run.smithi079.stdout:firewalld.service is not enabled 2024-09-17T08:05:40.599 INFO:teuthology.orchestra.run.smithi079.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout:Ceph Dashboard is now available at: 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: URL: https://smithi079.front.sepia.ceph.com:8443/ 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: User: admin 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: Password: 0w0ulxjkx0 2024-09-17T08:05:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:40.602 INFO:teuthology.orchestra.run.smithi079.stdout:Saving cluster configuration to /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config directory 2024-09-17T08:05:40.602 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling autotune for osd_memory_target 2024-09-17T08:05:42.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:42 smithi079 ceph-mon[29565]: mgrmap e13: y(active, since 6s) 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T08:05:43.094 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout: ceph telemetry on 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout:For more information see: 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:43.095 INFO:teuthology.orchestra.run.smithi079.stdout:Bootstrap complete. 2024-09-17T08:05:43.128 INFO:tasks.cephadm:Fetching config... 2024-09-17T08:05:43.129 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:05:43.129 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T08:05:43.145 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T08:05:43.145 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:05:43.145 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T08:05:43.200 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T08:05:43.201 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:05:43.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T08:05:43.264 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T08:05:43.265 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:05:43.265 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T08:05:43.319 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T08:05:43.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaWVPg5TVTEUdbpTsWfu+K8FhJ93sb4fgDrnMOvmxXqYw5b0vOMea8oMfoB/S10mJW/nGa/QSoD6OlCdGxjX4Awfm19gwnCBfdoN2/VT9RZ0Pviv9yu3L66qqvM2ExtqkAaa+5dnri5kvTpnZ5RllAu666w0UgWEXff2QkrJapcaSOTjEBW+NhxJo8FhS405SOUmllnK7oLzJVaX4RI3/zu2dZtDIKiqBBEWa43PdYgC9vUm1dr26kZ3GbrQ9huHx/qkhdLGFEjCux1FK+UHHYoPthhTlJf1zNVGjWHXDHqoqE4G3OnEzTqXW1+KwX2pcRgOu7J7UhtDdFemphhf2WRM9Cl9kiBW3wU9HlznSCC38m+d5D94H5W8bSHuWwcc+PqSImvWW78UA9DY6QIEA1S1K3zLj5PXU+qFKBHxlipKBZjB9C0DrzNhdQJqgdkmrW8sOIvMqlSwEmuLpJO2tIEhfWN8Pd7y39svYPxCarjLCwV8Bi2kjK+f6JMen2tG8= ceph-66e8930a-74cb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:05:43.396 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaWVPg5TVTEUdbpTsWfu+K8FhJ93sb4fgDrnMOvmxXqYw5b0vOMea8oMfoB/S10mJW/nGa/QSoD6OlCdGxjX4Awfm19gwnCBfdoN2/VT9RZ0Pviv9yu3L66qqvM2ExtqkAaa+5dnri5kvTpnZ5RllAu666w0UgWEXff2QkrJapcaSOTjEBW+NhxJo8FhS405SOUmllnK7oLzJVaX4RI3/zu2dZtDIKiqBBEWa43PdYgC9vUm1dr26kZ3GbrQ9huHx/qkhdLGFEjCux1FK+UHHYoPthhTlJf1zNVGjWHXDHqoqE4G3OnEzTqXW1+KwX2pcRgOu7J7UhtDdFemphhf2WRM9Cl9kiBW3wU9HlznSCC38m+d5D94H5W8bSHuWwcc+PqSImvWW78UA9DY6QIEA1S1K3zLj5PXU+qFKBHxlipKBZjB9C0DrzNhdQJqgdkmrW8sOIvMqlSwEmuLpJO2tIEhfWN8Pd7y39svYPxCarjLCwV8Bi2kjK+f6JMen2tG8= ceph-66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:05:43.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaWVPg5TVTEUdbpTsWfu+K8FhJ93sb4fgDrnMOvmxXqYw5b0vOMea8oMfoB/S10mJW/nGa/QSoD6OlCdGxjX4Awfm19gwnCBfdoN2/VT9RZ0Pviv9yu3L66qqvM2ExtqkAaa+5dnri5kvTpnZ5RllAu666w0UgWEXff2QkrJapcaSOTjEBW+NhxJo8FhS405SOUmllnK7oLzJVaX4RI3/zu2dZtDIKiqBBEWa43PdYgC9vUm1dr26kZ3GbrQ9huHx/qkhdLGFEjCux1FK+UHHYoPthhTlJf1zNVGjWHXDHqoqE4G3OnEzTqXW1+KwX2pcRgOu7J7UhtDdFemphhf2WRM9Cl9kiBW3wU9HlznSCC38m+d5D94H5W8bSHuWwcc+PqSImvWW78UA9DY6QIEA1S1K3zLj5PXU+qFKBHxlipKBZjB9C0DrzNhdQJqgdkmrW8sOIvMqlSwEmuLpJO2tIEhfWN8Pd7y39svYPxCarjLCwV8Bi2kjK+f6JMen2tG8= ceph-66e8930a-74cb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T08:05:43.520 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaWVPg5TVTEUdbpTsWfu+K8FhJ93sb4fgDrnMOvmxXqYw5b0vOMea8oMfoB/S10mJW/nGa/QSoD6OlCdGxjX4Awfm19gwnCBfdoN2/VT9RZ0Pviv9yu3L66qqvM2ExtqkAaa+5dnri5kvTpnZ5RllAu666w0UgWEXff2QkrJapcaSOTjEBW+NhxJo8FhS405SOUmllnK7oLzJVaX4RI3/zu2dZtDIKiqBBEWa43PdYgC9vUm1dr26kZ3GbrQ9huHx/qkhdLGFEjCux1FK+UHHYoPthhTlJf1zNVGjWHXDHqoqE4G3OnEzTqXW1+KwX2pcRgOu7J7UhtDdFemphhf2WRM9Cl9kiBW3wU9HlznSCC38m+d5D94H5W8bSHuWwcc+PqSImvWW78UA9DY6QIEA1S1K3zLj5PXU+qFKBHxlipKBZjB9C0DrzNhdQJqgdkmrW8sOIvMqlSwEmuLpJO2tIEhfWN8Pd7y39svYPxCarjLCwV8Bi2kjK+f6JMen2tG8= ceph-66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:05:43.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T08:05:43.729 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:05:43.849 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3273252524' entity='client.admin' 2024-09-17T08:05:46.302 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:46.303 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:46.303 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:05:46.303 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:46.303 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:46.303 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:47.286 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T08:05:47.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T08:05:47.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/308819418' entity='client.admin' 2024-09-17T08:05:47.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:47 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:47.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:47 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:47.905 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:05:50.545 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-09-17T08:05:50.546 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:05:50.546 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-09-17T08:05:50.565 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:05:50.565 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:05:50.622 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-09-17T08:05:50.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch host add smithi173 2024-09-17T08:05:50.813 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:05:50.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:53.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:52 smithi079 ceph-mon[29565]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:05:54.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:53 smithi079 ceph-mon[29565]: Deploying cephadm binary to smithi173 2024-09-17T08:05:55.258 INFO:teuthology.orchestra.run.smithi079.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-09-17T08:05:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:55 smithi079 ceph-mon[29565]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:05:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:05:55.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T08:05:56.129 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:05:56.506 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:56 smithi079 ceph-mon[29565]: Added host smithi173 2024-09-17T08:05:56.506 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:56.506 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:57.861 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:05:57.861 INFO:teuthology.orchestra.run.smithi079.stdout:[{"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-09-17T08:05:57.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:57 smithi079 ceph-mon[29565]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:05:57.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:58.464 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T08:05:58.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T08:05:58.658 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:05:59.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:05:59.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:05:59 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:00.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:06:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2361251075' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T08:06:01.238 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2024-09-17T08:06:01.845 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:01 smithi079 ceph-mon[29565]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:06:01.846 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:01 smithi079 ceph-mon[29565]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:06:01.846 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:01 smithi079 ceph-mon[29565]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:01.846 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2361251075' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:06:01.846 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:01 smithi079 ceph-mon[29565]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:01.865 INFO:tasks.cephadm:Adding mon.a on smithi079 2024-09-17T08:06:01.866 INFO:tasks.cephadm:Adding mon.c on smithi079 2024-09-17T08:06:01.866 INFO:tasks.cephadm:Adding mon.b on smithi173 2024-09-17T08:06:01.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b' 2024-09-17T08:06:02.062 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:03.799 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-09-17T08:06:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:03 smithi079 ceph-mon[29565]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:04.271 DEBUG:teuthology.orchestra.run.smithi079:mon.c> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T08:06:04.274 DEBUG:teuthology.orchestra.run.smithi173:mon.b> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T08:06:04.277 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:06:04.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:06:05.089 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:05.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:05.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: Saving service mon spec with placement smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b;count:3 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: Deploying daemon mon.b on smithi173 2024-09-17T08:06:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:04 smithi079 ceph-mon[29565]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:07 smithi079 ceph-mon[29565]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:08.415 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:06:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","modified":"2024-09-17T08:04:38.075362Z","created":"2024-09-17T08:04:38.075362Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T08:06:08.416 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-09-17T08:06:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:08 smithi173 systemd[1]: Starting Ceph mon.b for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:06:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/3415809056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 podman[33149]: 2024-09-17 08:06:09.270021238 +0000 UTC m=+0.102839498 container create de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 podman[33149]: 2024-09-17 08:06:09.183413135 +0000 UTC m=+0.016231410 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 podman[33149]: 2024-09-17 08:06:09.447648841 +0000 UTC m=+0.280467113 container init de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 podman[33149]: 2024-09-17 08:06:09.452611614 +0000 UTC m=+0.285429878 container start de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pidfile_write: ignore empty --pid-file 2024-09-17T08:06:09.532 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: load: jerasure load: lrc 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Git sha 0 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: DB SUMMARY 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: DB Session ID: ERX8R4EW8R1WJ4DTWD3P 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T08:06:09.533 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-17T08:06:09.536 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.env: 0x55c634cb1bc0 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.info_log: 0x55c636a84be0 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.statistics: (nil) 2024-09-17T08:06:09.537 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.use_fsync: 0 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:06:09.538 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.db_log_dir: 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.wal_dir: 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:06:09.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.write_buffer_manager: 0x55c636a92320 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:06:09.540 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.unordered_write: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.row_cache: None 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.wal_filter: None 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.wal_compression: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:06:09.541 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:06:09.542 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_open_files: -1 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:06:09.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Compression algorithms supported: 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kZSTD supported: 0 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.merge_operator: 2024-09-17T08:06:09.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_filter: None 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c636a847e0) 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: index_type: 0 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: data_block_index_type: 0 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: index_shortening: 1 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:06:09.545 INFO:journalctl@ceph.mon.b.smithi173.stdout: checksum: 4 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: no_block_cache: 0 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_cache: 0x55c636a79350 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_cache_options: 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: capacity : 536870912 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: num_shard_bits : 4 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: strict_capacity_limit : 0 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_cache_compressed: (nil) 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: persistent_cache: (nil) 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_size: 4096 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_size_deviation: 10 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_restart_interval: 16 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: index_block_restart_interval: 1 2024-09-17T08:06:09.546 INFO:journalctl@ceph.mon.b.smithi173.stdout: metadata_block_size: 4096 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: partition_filters: 0 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: use_delta_encoding: 1 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: filter_policy: bloomfilter 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: whole_key_filtering: 1 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: verify_compression: 0 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: format_version: 5 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: enable_index_compression: 1 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: block_align: 0 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: max_auto_readahead_size: 262144 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: prepopulate_block_cache: 0 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:06:09.547 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression: NoCompression 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.num_levels: 7 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:06:09.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:06:09.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:06:09.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:06:09.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:06:09.552 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.ttl: 2592000 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:06:09.553 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 95b3cb7f-82df-451d-89d3-28d9d066c543 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560369479692, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560369480116, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726560369, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "95b3cb7f-82df-451d-89d3-28d9d066c543", "db_session_id": "ERX8R4EW8R1WJ4DTWD3P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560369480210, "job": 1, "event": "recovery_finished"} 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c636aa4e00 2024-09-17T08:06:09.554 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: DB pointer 0x55c636b98000 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: ** DB Stats ** 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.555 INFO:journalctl@ceph.mon.b.smithi173.stdout: ** Compaction Stats [default] ** 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: ** Compaction Stats [default] ** 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:06:09.556 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Block cache BinnedLRUCache@0x55c636a79350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:06:09.557 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: using public_addr v2:172.21.15.173:0/0 -> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: starting mon.b rank -1 at public addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] at bind addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(???) e0 preinit fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: e1 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: legacy client fscid: -1 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout: No filesystems configured 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:06:09.558 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:06:09.559 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:06:09.559 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:06:09.559 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:06:09.559 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:09.559 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2361251075' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: Saving service mon spec with placement smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b;count:3 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:09.560 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: Deploying daemon mon.b on smithi173 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/3415809056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 ceph-mon[33187]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 bash[33149]: de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 2024-09-17T08:06:09.561 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:09 smithi173 systemd[1]: Started Ceph mon.b for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:06:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:09 smithi079 ceph-mon[29565]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:10.033 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:06:10.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:06:10.245 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:06:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: mon.a calling monitor election 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: mon.b calling monitor election 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: monmap e2: 2 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: fsmap 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: mgrmap e13: y(active, since 40s) 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: overall HEALTH_OK 2024-09-17T08:06:14.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:14.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:14.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:14.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: mon.a calling monitor election 2024-09-17T08:06:14.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:14.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: mon.b calling monitor election 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: monmap e2: 2 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: fsmap 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: mgrmap e13: y(active, since 40s) 2024-09-17T08:06:14.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: overall HEALTH_OK 2024-09-17T08:06:14.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:14.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:14.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:15.092 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:06:15.092 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","modified":"2024-09-17T08:06:09.502664Z","created":"2024-09-17T08:04:38.075362Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T08:06:15.093 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-09-17T08:06:15.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:15 smithi079 ceph-mon[29565]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:15.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:15 smithi079 ceph-mon[29565]: Deploying daemon mon.c on smithi079 2024-09-17T08:06:15.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/3882635089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:15.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:15 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:15.971 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:15 smithi173 ceph-mon[33187]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:15.971 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:15 smithi173 ceph-mon[33187]: Deploying daemon mon.c on smithi079 2024-09-17T08:06:15.971 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/3882635089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:15.971 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:15 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:16.698 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T08:06:16.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T08:06:16.907 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:06:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 systemd[1]: Starting Ceph mon.c for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[29565]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 podman[40611]: 2024-09-17 08:06:17.490578448 +0000 UTC m=+0.109158224 container create 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 podman[40611]: 2024-09-17 08:06:17.397271235 +0000 UTC m=+0.015851022 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 podman[40611]: 2024-09-17 08:06:17.726581181 +0000 UTC m=+0.345160963 container init 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 podman[40611]: 2024-09-17 08:06:17.731243468 +0000 UTC m=+0.349823246 container start 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T08:06:17.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T08:06:17.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 7 2024-09-17T08:06:17.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pidfile_write: ignore empty --pid-file 2024-09-17T08:06:17.950 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:17 smithi173 ceph-mon[33187]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: load: jerasure load: lrc 2024-09-17T08:06:18.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: RocksDB version: 7.9.2 2024-09-17T08:06:18.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Git sha 0 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: DB SUMMARY 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: DB Session ID: A5RB09Q44CRL2FTESIPO 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: CURRENT file: CURRENT 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.error_if_exists: 0 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.create_if_missing: 0 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.paranoid_checks: 1 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T08:06:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.env: 0x560704f61bc0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.info_log: 0x5607075c6be0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.statistics: (nil) 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.use_fsync: 0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_log_file_size: 0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_fallocate: 1 2024-09-17T08:06:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.use_direct_reads: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.db_log_dir: 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.wal_dir: 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T08:06:18.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.write_buffer_manager: 0x5607075d4320 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T08:06:18.125 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.unordered_write: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.row_cache: None 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.wal_filter: None 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.two_write_queues: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.wal_compression: 0 2024-09-17T08:06:18.126 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.atomic_flush: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.log_readahead_size: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_background_jobs: 2 2024-09-17T08:06:18.127 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_background_compactions: -1 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_subcompactions: 1 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_open_files: -1 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T08:06:18.128 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_background_flushes: -1 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Compression algorithms supported: 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kZSTD supported: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kXpressCompression supported: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kBZip2Compression supported: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kLZ4Compression supported: 1 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kZlibCompression supported: 1 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: kSnappyCompression supported: 1 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T08:06:18.129 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.merge_operator: 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_filter: None 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5607075c67e0) 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout: cache_index_and_filter_blocks: 1 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout: pin_top_level_index_and_filter: 1 2024-09-17T08:06:18.130 INFO:journalctl@ceph.mon.c.smithi079.stdout: index_type: 0 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: data_block_index_type: 0 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: index_shortening: 1 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: checksum: 4 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: no_block_cache: 0 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_cache: 0x5607075bb1f0 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_cache_name: BinnedLRUCache 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_cache_options: 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: capacity : 536870912 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: num_shard_bits : 4 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: strict_capacity_limit : 0 2024-09-17T08:06:18.131 INFO:journalctl@ceph.mon.c.smithi079.stdout: high_pri_pool_ratio: 0.000 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_cache_compressed: (nil) 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: persistent_cache: (nil) 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_size: 4096 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_size_deviation: 10 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_restart_interval: 16 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: index_block_restart_interval: 1 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: metadata_block_size: 4096 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: partition_filters: 0 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: use_delta_encoding: 1 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: filter_policy: bloomfilter 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: whole_key_filtering: 1 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: verify_compression: 0 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: read_amp_bytes_per_bit: 0 2024-09-17T08:06:18.132 INFO:journalctl@ceph.mon.c.smithi079.stdout: format_version: 5 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: enable_index_compression: 1 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: block_align: 0 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: max_auto_readahead_size: 262144 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: prepopulate_block_cache: 0 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: initial_auto_readahead_size: 8192 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T08:06:18.133 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression: NoCompression 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.num_levels: 7 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T08:06:18.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T08:06:18.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T08:06:18.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T08:06:18.139 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.inplace_update_support: 0 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T08:06:18.140 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.bloom_locality: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.max_successive_merges: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.ttl: 2592000 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enable_blob_files: false 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.min_blob_size: 0 2024-09-17T08:06:18.141 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: efc9aa08-82df-4166-b4a1-80fc13128833 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560377757810, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T08:06:18.142 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560377758187, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726560377, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "efc9aa08-82df-4166-b4a1-80fc13128833", "db_session_id": "A5RB09Q44CRL2FTESIPO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726560377758279, "job": 1, "event": "recovery_finished"} 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5607075e6e00 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: DB pointer 0x5607076da000 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: ** DB Stats ** 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T08:06:18.143 INFO:journalctl@ceph.mon.c.smithi079.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: ** Compaction Stats [default] ** 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T08:06:18.144 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Block cache BinnedLRUCache@0x5607075bb1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T08:06:18.145 INFO:journalctl@ceph.mon.c.smithi079.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: using public_addrv [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: starting mon.c rank -1 at public addrs [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] at bind addrs [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(???) e0 preinit fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).mds e1 new map 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: e1 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: legacy client fscid: -1 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: 2024-09-17T08:06:18.146 INFO:journalctl@ceph.mon.c.smithi079.stdout: No filesystems configured 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:06:18.147 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2361251075' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: Saving service mon spec with placement smithi079:172.21.15.79=a;smithi079:[v2:172.21.15.79:3301,v1:172.21.15.79:6790]=c;smithi173:172.21.15.173=b;count:3 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:18.148 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: Deploying daemon mon.b on smithi173 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/3415809056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.a calling monitor election 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.b calling monitor election 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.149 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: monmap e2: 2 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: fsmap 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mgrmap e13: y(active, since 40s) 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: overall HEALTH_OK 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:18.150 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: Deploying daemon mon.c on smithi079 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/3882635089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 ceph-mon[40649]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 bash[40611]: 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 2024-09-17T08:06:18.151 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:17 smithi079 systemd[1]: Started Ceph mon.c for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:06:23.089 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T08:06:23.089 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","modified":"2024-09-17T08:06:17.781257Z","created":"2024-09-17T08:04:38.075362Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-17T08:06:23.089 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: mon.a calling monitor election 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: mon.b calling monitor election 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: mon.c calling monitor election 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: monmap e3: 3 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0],c=[v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: fsmap 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: mgrmap e13: y(active, since 48s) 2024-09-17T08:06:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: overall HEALTH_OK 2024-09-17T08:06:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: mon.a calling monitor election 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: mon.b calling monitor election 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: mon.c calling monitor election 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: monmap e3: 3 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0],c=[v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: fsmap 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: mgrmap e13: y(active, since 48s) 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: overall HEALTH_OK 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: mon.a calling monitor election 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:06:23.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: mon.b calling monitor election 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: mon.c calling monitor election 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: monmap e3: 3 mons at {a=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],b=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0],c=[v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: fsmap 2024-09-17T08:06:23.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: mgrmap e13: y(active, since 48s) 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: overall HEALTH_OK 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:23.727 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T08:06:23.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T08:06:23.925 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:06:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/3228392053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/3228392053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/3228392053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:06:24.046 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:06:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:25.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:25.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:06:25.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[29565]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:06:25.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:24 smithi079 ceph-mon[40649]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:25.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:06:25.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:24 smithi173 ceph-mon[33187]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:06:25.812 INFO:teuthology.orchestra.run.smithi079.stdout:# minimal ceph.conf for 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:25.812 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-09-17T08:06:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:06:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: mon_host = [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] 2024-09-17T08:06:26.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: Reconfiguring daemon mon.a on smithi079 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/239540217' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: Reconfiguring daemon mon.a on smithi079 2024-09-17T08:06:26.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/239540217' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:26.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: Reconfiguring daemon mon.a on smithi079 2024-09-17T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/239540217' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:26.517 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T08:06:26.517 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:06:26.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:06:26.545 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:06:26.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:06:26.610 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:06:26.610 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T08:06:26.640 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:06:26.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:06:26.707 INFO:tasks.cephadm:Adding mgr.y on smithi079 2024-09-17T08:06:26.707 INFO:tasks.cephadm:Adding mgr.x on smithi173 2024-09-17T08:06:26.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi079=y;smithi173=x' 2024-09-17T08:06:26.933 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:06:27.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:27 smithi173 ceph-mon[33187]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:27.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:27 smithi079 ceph-mon[29565]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:27.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:27 smithi079 ceph-mon[40649]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:28.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:28 smithi173 ceph-mon[33187]: Reconfiguring daemon mon.c on smithi079 2024-09-17T08:06:28.579 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:28.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[29565]: Reconfiguring daemon mon.c on smithi079 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: Reconfiguring mon.c (monmap changed)... 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:28.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:28 smithi079 ceph-mon[40649]: Reconfiguring daemon mon.c on smithi079 2024-09-17T08:06:29.162 DEBUG:teuthology.orchestra.run.smithi173:mgr.x> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T08:06:29.164 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T08:06:29.165 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:06:29.165 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:06:29.181 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:06:29.181 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-09-17T08:06:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:06:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:29.237 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 959 Links: 1 2024-09-17T08:06:29.237 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:29.237 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:29.238 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-17 08:06:17.227810618 +0000 2024-09-17T08:06:29.238 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.238 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.238 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:06:29.302 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-17T08:06:29.302 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-17T08:06:29.302 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000289677 s, 1.8 MB/s 2024-09-17T08:06:29.303 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:06:29.360 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-17 08:06:17.227810618 +0000 2024-09-17T08:06:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.417 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.417 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:06:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-17T08:06:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-17T08:06:29.480 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000200486 s, 2.6 MB/s 2024-09-17T08:06:29.481 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:06:29.537 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-09-17T08:06:29.593 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:06:29.593 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:29.593 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-17 08:06:17.228810595 +0000 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.594 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-17 08:01:37.165241695 +0000 2024-09-17T08:06:29.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:06:29.656 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-17T08:06:29.657 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-17T08:06:29.657 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000229711 s, 2.2 MB/s 2024-09-17T08:06:29.658 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:06:29.715 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 955 Links: 1 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:29.770 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-17 08:06:17.228810595 +0000 2024-09-17T08:06:29.771 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.771 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-17 08:01:37.164241718 +0000 2024-09-17T08:06:29.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:06:29.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:29.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi079=y;smithi173=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: Saving service mgr spec with placement smithi079=y;smithi173=x;count:2 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:29.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi079=y;smithi173=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: Saving service mgr spec with placement smithi079=y;smithi173=x;count:2 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:29.834 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-17T08:06:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-17T08:06:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000173949 s, 2.9 MB/s 2024-09-17T08:06:29.836 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:06:29.893 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:06:29.893 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T08:06:29.909 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T08:06:29.910 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-09-17T08:06:29.964 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 08:06:08.559788936 +0000 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:29.965 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:29.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi079=y;smithi173=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: Saving service mgr spec with placement smithi079=y;smithi173=x;count:2 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T08:06:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:29.997 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T08:06:29.997 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T08:06:29.997 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000252521 s, 2.0 MB/s 2024-09-17T08:06:29.999 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T08:06:30.055 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 08:06:08.559788936 +0000 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.111 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.111 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T08:06:30.174 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T08:06:30.174 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T08:06:30.174 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000198147 s, 2.6 MB/s 2024-09-17T08:06:30.175 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T08:06:30.231 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 08:06:08.560788914 +0000 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 08:01:40.213729385 +0000 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 08:01:40.213729385 +0000 2024-09-17T08:06:30.287 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 08:01:40.213729385 +0000 2024-09-17T08:06:30.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T08:06:30.351 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T08:06:30.352 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T08:06:30.352 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000195987 s, 2.6 MB/s 2024-09-17T08:06:30.353 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T08:06:30.409 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T08:06:30.465 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 08:06:08.560788914 +0000 2024-09-17T08:06:30.466 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.466 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.466 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 08:01:40.212729408 +0000 2024-09-17T08:06:30.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T08:06:30.531 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T08:06:30.531 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T08:06:30.531 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000225373 s, 2.3 MB/s 2024-09-17T08:06:30.532 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T08:06:30.582 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:06:30.582 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: Reconfiguring daemon mon.b on smithi173 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:30.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.590 INFO:tasks.cephadm:Deploying osd.0 on smithi079 with /dev/vg_nvme/lv_4... 2024-09-17T08:06:30.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-17T08:06:30.780 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:06:30.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:30.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:06:30.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:30.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:30.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: Reconfiguring daemon mon.b on smithi173 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: Reconfiguring mon.b (monmap changed)... 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: Reconfiguring daemon mon.b on smithi173 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T08:06:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:31 smithi079 ceph-mon[29565]: Deploying daemon mgr.x on smithi173 2024-09-17T08:06:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:31 smithi079 ceph-mon[29565]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:31 smithi079 ceph-mon[40649]: Deploying daemon mgr.x on smithi173 2024-09-17T08:06:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:31 smithi079 ceph-mon[40649]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:31 smithi173 ceph-mon[33187]: Deploying daemon mgr.x on smithi173 2024-09-17T08:06:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:31 smithi173 ceph-mon[33187]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:32.279 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 podman[35223]: 2024-09-17 08:06:32.190348135 +0000 UTC m=+0.114413517 container create bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T08:06:32.497 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-17T08:06:32.592 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-17T08:06:32.606 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-09-17T08:06:32.606 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-09-17T08:06:32.606 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113985 s, 920 MB/s 2024-09-17T08:06:32.606 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 podman[35223]: 2024-09-17 08:06:32.384435865 +0000 UTC m=+0.308501266 container init bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 podman[35223]: 2024-09-17 08:06:32.389349915 +0000 UTC m=+0.313415300 container start bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True) 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-mgr[35261]: -- 172.21.15.173:0/754395668 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d7c2956d00 con 0x55d7c292b400 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 bash[35223]: bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 systemd[1]: Started Ceph mgr.x for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:32.506+0000 7f993f1c2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:32.566+0000 7f993f1c2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:06:32.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:32.722+0000 7f993f1c2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:06:32.974 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:32.792+0000 7f993f1c2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:06:32.974 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:32.953+0000 7f993f1c2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:06:33.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-09-17T08:06:33.381 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:33.092+0000 7f993f1c2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:06:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:33.417+0000 7f993f1c2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:33.477+0000 7f993f1c2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:33.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:33 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:33.974 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:33.668+0000 7f993f1c2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:06:34.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:34.525+0000 7f993f1c2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:34.589+0000 7f993f1c2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:35.546 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.294+0000 7f993f1c2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:06:35.547 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.406+0000 7f993f1c2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:06:35.547 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.545+0000 7f993f1c2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: Reconfiguring daemon mgr.y on smithi079 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:06:35.816 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:35.817 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.605+0000 7f993f1c2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:06:35.817 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.749+0000 7f993f1c2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:06:35.817 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.815+0000 7f993f1c2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: Reconfiguring daemon mgr.y on smithi079 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:06:35.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: Reconfiguring daemon mgr.y on smithi079 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:06:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:36.090 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:35.873+0000 7f993f1c2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:06:36.090 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.026+0000 7f993f1c2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:06:36.091 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.089+0000 7f993f1c2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:06:36.358 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.156+0000 7f993f1c2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:06:36.358 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.237+0000 7f993f1c2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:06:36.358 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.298+0000 7f993f1c2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:06:36.358 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:36.357+0000 7f993f1c2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:06:36.713 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:36.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:36.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:36 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:36 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:37.474 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:06:37.120+0000 7f993f1c2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:06:37.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: Standby manager daemon x started 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: Standby manager daemon x started 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:06:37.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:06:37.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:06:37.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:37 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: Standby manager daemon x started 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:06:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:37 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/1068000886' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2187267016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]': finished 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:06:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:38 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2187267016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]': finished 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T08:06:38.979 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:06:38.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2187267016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]: dispatch 2024-09-17T08:06:38.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "793e335a-ea62-499c-af46-e48c3190408a"}]': finished 2024-09-17T08:06:38.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T08:06:38.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:38 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:39 smithi173 ceph-mon[33187]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/728359279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:06:40.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:39 smithi079 ceph-mon[40649]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:40.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/728359279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:06:40.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:39 smithi079 ceph-mon[29565]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:40.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/728359279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:06:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:41 smithi173 ceph-mon[33187]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:42.016 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:41 smithi079 ceph-mon[29565]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:42.017 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:41 smithi079 ceph-mon[40649]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:43.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:43 smithi079 ceph-mon[29565]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:43.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:43 smithi079 ceph-mon[40649]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:43 smithi173 ceph-mon[33187]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:44 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:06:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:44 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:44 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:06:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:44 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:44 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T08:06:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:44 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:45.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:45 smithi079 ceph-mon[29565]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:45.918 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:45 smithi079 ceph-mon[29565]: Deploying daemon osd.0 on smithi079 2024-09-17T08:06:45.919 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:45 smithi079 ceph-mon[40649]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:45.919 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:45 smithi079 ceph-mon[40649]: Deploying daemon osd.0 on smithi079 2024-09-17T08:06:45.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:45 smithi173 ceph-mon[33187]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:45.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:45 smithi173 ceph-mon[33187]: Deploying daemon osd.0 on smithi079 2024-09-17T08:06:47.908 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:47 smithi079 ceph-mon[29565]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:47.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:47 smithi079 ceph-mon[40649]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:47 smithi173 ceph-mon[33187]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:49 smithi173 ceph-mon[33187]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:49.984 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[40649]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[29565]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:49.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:49.986 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:51.958 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:51 smithi079 ceph-mon[29565]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:51.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:51 smithi079 ceph-mon[40649]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:51 smithi173 ceph-mon[33187]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:52.896 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 0 on host 'smithi079' 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:53.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T08:06:53.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:06:53.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:53.528 DEBUG:teuthology.orchestra.run.smithi079:osd.0> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T08:06:53.531 INFO:tasks.cephadm:Deploying osd.1 on smithi079 with /dev/vg_nvme/lv_3... 2024-09-17T08:06:53.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-17T08:06:54.348 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:06:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[40649]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:06:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[40649]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[40649]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[40649]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[29565]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[29565]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[29565]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-mon[29565]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:54.871 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0[44871]: 2024-09-17T08:06:54.579+0000 7f115618f640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T08:06:54.871 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:06:54 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0[44871]: 2024-09-17T08:06:54.582+0000 7f1151fb8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:06:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:54 smithi173 ceph-mon[33187]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T08:06:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:54 smithi173 ceph-mon[33187]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T08:06:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:54 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:54 smithi173 ceph-mon[33187]: from='osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:54 smithi173 ceph-mon[33187]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:06:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[29565]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[29565]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:06:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[29565]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:06:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[40649]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[40649]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:06:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[40649]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:06:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:55 smithi173 ceph-mon[33187]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:55 smithi173 ceph-mon[33187]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:06:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:55 smithi173 ceph-mon[33187]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T08:06:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[29565]: osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997] boot 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[29565]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[40649]: osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997] boot 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[40649]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:06:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:56 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:06:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:56 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:06:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:56 smithi173 ceph-mon[33187]: osd.0 [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997] boot 2024-09-17T08:06:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:56 smithi173 ceph-mon[33187]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T08:06:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:06:57.190 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-17T08:06:57.283 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-17T08:06:57.298 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-09-17T08:06:57.298 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-09-17T08:06:57.298 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136782 s, 767 MB/s 2024-09-17T08:06:57.298 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:57.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:57.820 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:57.821 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:06:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:06:57.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:06:58.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-09-17T08:06:58.317 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:06:58.714 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:58 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi079 2024-09-17T08:06:58.714 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:58 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 16064M 2024-09-17T08:06:58.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:58 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi079 2024-09-17T08:06:58.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:58 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 16064M 2024-09-17T08:06:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:58 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi079 2024-09-17T08:06:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:58 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 16064M 2024-09-17T08:06:59.744 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:06:59 smithi079 ceph-mon[29565]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:06:59.745 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:06:59 smithi079 ceph-mon[40649]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:06:59.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:06:59 smithi173 ceph-mon[33187]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:00.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:00.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:00.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:00.783 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:00.783 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:00.783 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:00 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:00 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:00 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:00 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:01.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:01 smithi079 ceph-mon[29565]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:01.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:01 smithi079 ceph-mon[29565]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:01 smithi079 ceph-mon[40649]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:01 smithi079 ceph-mon[40649]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:01 smithi173 ceph-mon[33187]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:01 smithi173 ceph-mon[33187]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1624153325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]': finished 2024-09-17T08:07:02.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[40649]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:07:02.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1624153325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]': finished 2024-09-17T08:07:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[29565]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:07:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1624153325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]: dispatch 2024-09-17T08:07:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d"}]': finished 2024-09-17T08:07:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:02 smithi173 ceph-mon[33187]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T08:07:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:03 smithi079 ceph-mon[29565]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3305806743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:03 smithi079 ceph-mon[40649]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3305806743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:03 smithi173 ceph-mon[33187]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3305806743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:05 smithi079 ceph-mon[29565]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:05 smithi079 ceph-mon[40649]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:05 smithi173 ceph-mon[33187]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:07 smithi079 ceph-mon[29565]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:07 smithi079 ceph-mon[40649]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:07 smithi173 ceph-mon[33187]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:07:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:07:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T08:07:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:09 smithi079 ceph-mon[40649]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:09 smithi079 ceph-mon[40649]: Deploying daemon osd.1 on smithi079 2024-09-17T08:07:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:09 smithi079 ceph-mon[29565]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:09 smithi079 ceph-mon[29565]: Deploying daemon osd.1 on smithi079 2024-09-17T08:07:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:09 smithi173 ceph-mon[33187]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:09 smithi173 ceph-mon[33187]: Deploying daemon osd.1 on smithi079 2024-09-17T08:07:11.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:11 smithi079 ceph-mon[29565]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:11.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:11 smithi079 ceph-mon[40649]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:11 smithi173 ceph-mon[33187]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:13 smithi079 ceph-mon[29565]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:13.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:13 smithi079 ceph-mon[40649]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:13 smithi173 ceph-mon[33187]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:14.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:14.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:14.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:14.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:14.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:14.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:14.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:14.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:14.963 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:15 smithi173 ceph-mon[33187]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:15.997 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:15 smithi079 ceph-mon[29565]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:15.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:15 smithi079 ceph-mon[40649]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:16.807 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 1 on host 'smithi079' 2024-09-17T08:07:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[29565]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[29565]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:07:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[40649]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[40649]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:07:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.414 DEBUG:teuthology.orchestra.run.smithi079:osd.1> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T08:07:17.417 INFO:tasks.cephadm:Deploying osd.2 on smithi079 with /dev/vg_nvme/lv_2... 2024-09-17T08:07:17.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-17T08:07:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:17 smithi173 ceph-mon[33187]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:17 smithi173 ceph-mon[33187]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T08:07:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:17 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:17 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:17.671 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:07:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:18.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:18 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:18.709 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1[51274]: 2024-09-17T08:07:18.572+0000 7f923664a640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T08:07:18.709 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:18 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1[51274]: 2024-09-17T08:07:18.580+0000 7f922dc54640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:19.273 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[29565]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:19.274 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:19 smithi079 ceph-mon[40649]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T08:07:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:19 smithi173 ceph-mon[33187]: from='osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614]' entity='osd.1' 2024-09-17T08:07:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-17T08:07:19.716 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-17T08:07:19.733 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-09-17T08:07:19.734 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-09-17T08:07:19.734 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0154195 s, 680 MB/s 2024-09-17T08:07:19.734 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614] boot 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:07:20.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:07:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614] boot 2024-09-17T08:07:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:07:20.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:20 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:20.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-09-17T08:07:20.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:07:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:07:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: osd.1 [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614] boot 2024-09-17T08:07:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T08:07:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:20 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:07:21.272 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:07:21.684 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:21 smithi079 ceph-mon[29565]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:21.684 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:21 smithi079 ceph-mon[29565]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:07:21.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:21 smithi079 ceph-mon[40649]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:21.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:21 smithi079 ceph-mon[40649]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:07:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:21 smithi173 ceph-mon[33187]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:21 smithi173 ceph-mon[33187]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi079 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 8032M 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:23 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi079 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 8032M 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.041 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi079 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.042 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 8032M 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:24.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:23 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:25.004 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:25.004 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:25.004 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:25.004 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:25.005 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:25.005 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:25 smithi173 ceph-mon[33187]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:25 smithi173 ceph-mon[33187]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:26.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:25 smithi079 ceph-mon[29565]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:25 smithi079 ceph-mon[29565]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:26.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:25 smithi079 ceph-mon[40649]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:26.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:25 smithi079 ceph-mon[40649]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]: dispatch 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]': finished 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[29565]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/653493655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]: dispatch 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]': finished 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[40649]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:07:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:26.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/653493655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]: dispatch 2024-09-17T08:07:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3323963270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2"}]': finished 2024-09-17T08:07:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:26 smithi173 ceph-mon[33187]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T08:07:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/653493655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:27 smithi173 ceph-mon[33187]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:28.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:27 smithi079 ceph-mon[29565]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:27 smithi079 ceph-mon[40649]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:29 smithi173 ceph-mon[33187]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:29.974 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:29 smithi079 ceph-mon[29565]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:29.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:29 smithi079 ceph-mon[40649]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:31.967 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:31 smithi079 ceph-mon[29565]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:31.967 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:31 smithi079 ceph-mon[40649]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:31 smithi173 ceph-mon[33187]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:32.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:32 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:07:32.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:32 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:32 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:07:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:32 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:32 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T08:07:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:32 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:33.899 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:33 smithi079 ceph-mon[29565]: Deploying daemon osd.2 on smithi079 2024-09-17T08:07:33.899 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:33 smithi079 ceph-mon[29565]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:33.900 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:33 smithi079 ceph-mon[40649]: Deploying daemon osd.2 on smithi079 2024-09-17T08:07:33.900 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:33 smithi079 ceph-mon[40649]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:33 smithi173 ceph-mon[33187]: Deploying daemon osd.2 on smithi079 2024-09-17T08:07:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:33 smithi173 ceph-mon[33187]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:35.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:35 smithi079 ceph-mon[29565]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:35.818 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:35 smithi079 ceph-mon[40649]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:35 smithi173 ceph-mon[33187]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:37.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[40649]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:37.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:37.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:37.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:37.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[29565]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:37.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:37.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:37.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:37 smithi173 ceph-mon[33187]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:37 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:37 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:37 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:39 smithi173 ceph-mon[33187]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:39 smithi079 ceph-mon[29565]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:39 smithi079 ceph-mon[40649]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:40.226 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 2 on host 'smithi079' 2024-09-17T08:07:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:40 smithi173 ceph-mon[33187]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:40 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:40 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:40 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[29565]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[40649]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:07:40.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:40 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:40.893 DEBUG:teuthology.orchestra.run.smithi079:osd.2> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T08:07:40.895 INFO:tasks.cephadm:Deploying osd.3 on smithi079 with /dev/vg_nvme/lv_1... 2024-09-17T08:07:40.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-17T08:07:41.150 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:07:41.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:41.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:41.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:41.602 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[57373]: 2024-09-17T08:07:41.473+0000 7f66e02be640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T08:07:41.602 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:41 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[57373]: 2024-09-17T08:07:41.476+0000 7f66dc0e7640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:07:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:41.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[29565]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[40649]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:42 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:42 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:07:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:42 smithi173 ceph-mon[33187]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T08:07:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:42 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:42 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652] boot 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652] boot 2024-09-17T08:07:43.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:07:43.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: osd.2 [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652] boot 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T08:07:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:07:44.327 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-17T08:07:44.421 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-17T08:07:44.436 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-09-17T08:07:44.436 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-09-17T08:07:44.436 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128753 s, 814 MB/s 2024-09-17T08:07:44.436 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-09-17T08:07:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:44 smithi173 ceph-mon[33187]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:07:45.071 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:44 smithi079 ceph-mon[40649]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:07:45.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:44 smithi079 ceph-mon[29565]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T08:07:45.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-09-17T08:07:45.707 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.955 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:45 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:45 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi079 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 5354M 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:46.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[29565]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi079 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 5354M 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:07:46.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:46.828 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.828 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.828 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.828 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:46 smithi079 ceph-mon[40649]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi079 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 5354M 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:46.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:46 smithi173 ceph-mon[33187]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:47.088 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:46 smithi079 sudo[61276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:47.088 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:46 smithi079 sudo[61276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:47.088 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:46 smithi079 sudo[61276]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:47.088 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:46 smithi079 sudo[61276]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:47.366 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:47.367 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:47.367 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61299]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:47.367 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61299]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:47.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61330]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:47.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61330]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:47.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:47.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61330]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:47.901 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61417]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:47.902 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61417]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:47.902 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61417]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:47.902 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61417]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:48.165 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61473]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:48.165 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:48.165 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61473]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:48.165 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:47 smithi079 sudo[61473]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:48.165 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:07:48.165 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:48.165 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:48.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:48 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:48.426 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61495]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:48.426 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:48.426 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61495]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:48.426 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61495]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:48.427 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61516]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:48.427 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:48.427 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61516]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:48.427 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61516]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:07:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61526]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61526]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:48 smithi079 sudo[61526]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:49.166 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 sudo[35911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:49.166 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 sudo[35911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:49.166 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 sudo[35911]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:49.166 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:48 smithi173 sudo[35911]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:49.330 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:49 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 sudo[35914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 sudo[35914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 sudo[35914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 sudo[35914]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:49.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:49.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:49 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:49.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T08:07:49.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:49.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61571]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:49.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61571]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T08:07:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T08:07:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T08:07:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:49 smithi079 sudo[61597]: pam_unix(sudo:session): session closed for user root 2024-09-17T08:07:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]: dispatch 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]': finished 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:07:50.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:07:50.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]: dispatch 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]': finished 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:07:50.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]: dispatch 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1649523838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69af955a-9dbe-4d86-8278-27df098a3886"}]': finished 2024-09-17T08:07:50.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T08:07:50.571 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:07:51.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:51 smithi079 ceph-mon[29565]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:51.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:51 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2791462075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:51 smithi079 ceph-mon[40649]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:51 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2791462075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:51.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:51 smithi173 ceph-mon[33187]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:51.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:51 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2791462075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:07:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:53 smithi173 ceph-mon[33187]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:53.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:53 smithi079 ceph-mon[29565]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:53.800 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:53 smithi079 ceph-mon[40649]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:55.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:55 smithi079 ceph-mon[40649]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:55.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:55 smithi079 ceph-mon[29565]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:55 smithi173 ceph-mon[33187]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:57.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[29565]: Deploying daemon osd.3 on smithi079 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[29565]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[40649]: Deploying daemon osd.3 on smithi079 2024-09-17T08:07:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:57 smithi079 ceph-mon[40649]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T08:07:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:57 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:07:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:57 smithi173 ceph-mon[33187]: Deploying daemon osd.3 on smithi079 2024-09-17T08:07:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:57 smithi173 ceph-mon[33187]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:07:59 smithi173 ceph-mon[33187]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:59.792 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:07:59 smithi079 ceph-mon[29565]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:07:59.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:07:59 smithi079 ceph-mon[40649]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:00.647 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:00.647 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:00.647 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[40649]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:08:00.647 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:00.648 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:00.648 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:00.648 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[29565]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:08:00.648 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:00.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:00 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:00 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:00 smithi173 ceph-mon[33187]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T08:08:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:00 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:01.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[40649]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:01.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[29565]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:01.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:01 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:01 smithi173 ceph-mon[33187]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:01 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:01 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:01 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:03.750 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:03 smithi079 ceph-mon[29565]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:03.751 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:03 smithi079 ceph-mon[40649]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:03 smithi173 ceph-mon[33187]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:04.728 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 3 on host 'smithi079' 2024-09-17T08:08:05.393 DEBUG:teuthology.orchestra.run.smithi079:osd.3> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.3.service 2024-09-17T08:08:05.396 INFO:tasks.cephadm:Deploying osd.4 on smithi173 with /dev/vg_nvme/lv_4... 2024-09-17T08:08:05.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-17T08:08:05.600 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:05.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:05.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:05 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T08:08:05.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:05.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:05.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:05 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:06.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[29565]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[29565]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[29565]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[29565]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[40649]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[40649]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T08:08:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:06.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[40649]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:06.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-mon[40649]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:06.621 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3[63405]: 2024-09-17T08:08:06.356+0000 7f47c37a6640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T08:08:06.621 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:08:06 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3[63405]: 2024-09-17T08:08:06.360+0000 7f47bf5cf640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:08:06.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:06 smithi173 ceph-mon[33187]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T08:08:06.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:06 smithi173 ceph-mon[33187]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T08:08:06.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:06 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:06.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:06 smithi173 ceph-mon[33187]: from='osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:06.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:06 smithi173 ceph-mon[33187]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:08:07.537 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-17T08:08:07.619 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-17T08:08:07.619 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: osdmap e26: 4 total, 3 up, 4 in 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:07 smithi173 ceph-mon[33187]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:07.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: osdmap e26: 4 total, 3 up, 4 in 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[40649]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: osdmap e26: 4 total, 3 up, 4 in 2024-09-17T08:08:07.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:07.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:07 smithi079 ceph-mon[29565]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-17T08:08:07.632 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-09-17T08:08:07.632 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-09-17T08:08:07.632 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111891 s, 937 MB/s 2024-09-17T08:08:07.633 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-09-17T08:08:08.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-09-17T08:08:08.472 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:08.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570] boot 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:08.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570] boot 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: osd.3 [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570] boot 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:08.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:09.642 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:09 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi079 2024-09-17T08:08:09.642 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:09 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 4016M 2024-09-17T08:08:09.642 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:09 smithi173 ceph-mon[33187]: osdmap e28: 4 total, 4 up, 4 in 2024-09-17T08:08:09.642 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:09 smithi173 ceph-mon[33187]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi079 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 4016M 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[40649]: osdmap e28: 4 total, 4 up, 4 in 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[40649]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi079 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 4016M 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[29565]: osdmap e28: 4 total, 4 up, 4 in 2024-09-17T08:08:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:09 smithi079 ceph-mon[29565]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:10.650 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:10 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:10.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:10 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:10.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:10 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:10.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:10.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:10.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:10 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:11.433 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:11 smithi173 ceph-mon[33187]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:11.433 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:11 smithi173 ceph-mon[33187]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:11.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:11 smithi079 ceph-mon[40649]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:11.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:11 smithi079 ceph-mon[40649]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:11 smithi079 ceph-mon[29565]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:11 smithi079 ceph-mon[29565]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:12.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/1628585128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:12 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:12 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]': finished 2024-09-17T08:08:12.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:12 smithi173 ceph-mon[33187]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:08:12.789 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:12 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/1628585128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]': finished 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[40649]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/1628585128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]: dispatch 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35"}]': finished 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[29565]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T08:08:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:12 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:13 smithi079 ceph-mon[29565]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/96862800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:13 smithi079 ceph-mon[40649]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/96862800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:13 smithi173 ceph-mon[33187]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/96862800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:15.717 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:15 smithi173 ceph-mon[33187]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:15.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:15 smithi079 ceph-mon[40649]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:15 smithi079 ceph-mon[29565]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:17.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:17 smithi173 ceph-mon[33187]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:17 smithi079 ceph-mon[40649]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:17 smithi079 ceph-mon[29565]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:19 smithi173 ceph-mon[33187]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:08:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:19 smithi173 ceph-mon[33187]: Deploying daemon osd.4 on smithi173 2024-09-17T08:08:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[40649]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[40649]: Deploying daemon osd.4 on smithi173 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[29565]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:19 smithi079 ceph-mon[29565]: Deploying daemon osd.4 on smithi173 2024-09-17T08:08:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:21 smithi173 ceph-mon[33187]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:21 smithi079 ceph-mon[29565]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:21 smithi079 ceph-mon[40649]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:23 smithi173 ceph-mon[33187]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:23 smithi079 ceph-mon[29565]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:23 smithi079 ceph-mon[40649]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:24.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:24.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:24.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:25.501 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:25 smithi173 ceph-mon[33187]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:25.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:25 smithi079 ceph-mon[29565]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:25 smithi079 ceph-mon[40649]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:26.928 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:26.928 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:26.929 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:26 smithi173 ceph-mon[33187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:26.963 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 4 on host 'smithi173' 2024-09-17T08:08:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[40649]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:26 smithi079 ceph-mon[29565]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T08:08:27.590 DEBUG:teuthology.orchestra.run.smithi173:osd.4> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.4.service 2024-09-17T08:08:27.593 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_3... 2024-09-17T08:08:27.593 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-17T08:08:28.113 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:28.113 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.113 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.113 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:08:28.114 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T08:08:28.114 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:28.114 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.114 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:27 smithi173 ceph-mon[33187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:28.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[29565]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:27 smithi079 ceph-mon[40649]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:28.412 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:08:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:08:28.179+0000 7f79c777b640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T08:08:28.412 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:08:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:08:28.185+0000 7f79c35a4640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:08:29.158 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:29.425 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:29 smithi173 ceph-mon[33187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:29.425 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:29 smithi173 ceph-mon[33187]: osdmap e31: 5 total, 4 up, 5 in 2024-09-17T08:08:29.425 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.425 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:29 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.425 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:29 smithi173 ceph-mon[33187]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:29.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[40649]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[40649]: osdmap e31: 5 total, 4 up, 5 in 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[40649]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[29565]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[29565]: osdmap e31: 5 total, 4 up, 5 in 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:29 smithi079 ceph-mon[29565]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597] boot 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:08:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:30.603 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:30.603 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:30.603 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597] boot 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: osd.4 [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597] boot 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T08:08:30.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:08:31.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:08:31.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:31.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:31.358 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.358 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:31.358 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:31.358 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:31 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.520 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-17T08:08:31.598 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-17T08:08:31.611 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-09-17T08:08:31.611 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-09-17T08:08:31.611 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112496 s, 932 MB/s 2024-09-17T08:08:31.612 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-09-17T08:08:31.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:08:31.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:31.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:31 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:32.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-09-17T08:08:32.398 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:32.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:32 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi173 2024-09-17T08:08:32.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:32 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-17T08:08:32.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:32 smithi173 ceph-mon[33187]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:08:32.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:32 smithi173 ceph-mon[33187]: osdmap e35: 5 total, 5 up, 5 in 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi173 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[29565]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[29565]: osdmap e35: 5 total, 5 up, 5 in 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi173 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[40649]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T08:08:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:32 smithi079 ceph-mon[40649]: osdmap e35: 5 total, 5 up, 5 in 2024-09-17T08:08:33.433 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:33 smithi173 ceph-mon[33187]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:33.433 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:33 smithi173 ceph-mon[33187]: osdmap e36: 5 total, 5 up, 5 in 2024-09-17T08:08:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:33 smithi079 ceph-mon[29565]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:33 smithi079 ceph-mon[29565]: osdmap e36: 5 total, 5 up, 5 in 2024-09-17T08:08:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:33 smithi079 ceph-mon[40649]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:33 smithi079 ceph-mon[40649]: osdmap e36: 5 total, 5 up, 5 in 2024-09-17T08:08:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:34 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:34.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:34.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:34 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:35.315 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:35 smithi173 ceph-mon[33187]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:35.316 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:35 smithi173 ceph-mon[33187]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:35 smithi079 ceph-mon[40649]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:35 smithi079 ceph-mon[40649]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:35 smithi079 ceph-mon[29565]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:35 smithi079 ceph-mon[29565]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/327489013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]': finished 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-17T08:08:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/1676547778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:37.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/327489013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]': finished 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/1676547778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/327489013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]: dispatch 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad"}]': finished 2024-09-17T08:08:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T08:08:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-17T08:08:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/1676547778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:08:39.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:39 smithi173 ceph-mon[33187]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-17T08:08:39.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:39 smithi079 ceph-mon[29565]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-17T08:08:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:39 smithi079 ceph-mon[40649]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-17T08:08:41.686 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:41 smithi173 ceph-mon[33187]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:08:41.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:41 smithi079 ceph-mon[29565]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:08:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:41 smithi079 ceph-mon[40649]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T08:08:42.645 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:42 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:08:42.645 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:42 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:42.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:42 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:08:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:42 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:42 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T08:08:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:42 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:43.767 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:43 smithi173 ceph-mon[33187]: Deploying daemon osd.5 on smithi173 2024-09-17T08:08:43.767 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:43 smithi173 ceph-mon[33187]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:08:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:43 smithi079 ceph-mon[40649]: Deploying daemon osd.5 on smithi173 2024-09-17T08:08:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:43 smithi079 ceph-mon[40649]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:08:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:43 smithi079 ceph-mon[29565]: Deploying daemon osd.5 on smithi173 2024-09-17T08:08:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:43 smithi079 ceph-mon[29565]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:08:45.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:45 smithi173 ceph-mon[33187]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:08:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:45 smithi079 ceph-mon[29565]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:08:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:45 smithi079 ceph-mon[40649]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:08:47.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:47 smithi173 ceph-mon[33187]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:47.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:47 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:47.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:47 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:47.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:47 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[29565]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[40649]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:47 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: osdmap e38: 6 total, 5 up, 6 in 2024-09-17T08:08:48.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:48 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:48.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: osdmap e38: 6 total, 5 up, 6 in 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: osdmap e38: 6 total, 5 up, 6 in 2024-09-17T08:08:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:48 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:49.717 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:49 smithi173 ceph-mon[33187]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:49.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:49 smithi079 ceph-mon[40649]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:49.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:49 smithi079 ceph-mon[29565]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:50.789 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2024-09-17T08:08:51.100 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:50 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:50 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:51.439 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.5.service 2024-09-17T08:08:51.442 INFO:tasks.cephadm:Deploying osd.6 on smithi173 with /dev/vg_nvme/lv_2... 2024-09-17T08:08:51.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-17T08:08:52.205 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:52.307 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[29565]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:08:52.307 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[29565]: osdmap e39: 6 total, 5 up, 6 in 2024-09-17T08:08:52.307 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:52.307 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[29565]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.307 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[29565]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[40649]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:08:52.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[40649]: osdmap e39: 6 total, 5 up, 6 in 2024-09-17T08:08:52.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:52.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[40649]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:52 smithi079 ceph-mon[40649]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-mon[33187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T08:08:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-mon[33187]: osdmap e39: 6 total, 5 up, 6 in 2024-09-17T08:08:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-mon[33187]: from='osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-mon[33187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:08:52.632 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:08:52.428+0000 7f60a0d0c640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T08:08:52.633 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:08:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:08:52.432+0000 7f6098b2d640 -1 osd.5 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: osdmap e40: 6 total, 5 up, 6 in 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:53.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:53 smithi173 ceph-mon[33187]: from='osd.5 ' entity='osd.5' 2024-09-17T08:08:53.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:53.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: osdmap e40: 6 total, 5 up, 6 in 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[29565]: from='osd.5 ' entity='osd.5' 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: osdmap e40: 6 total, 5 up, 6 in 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-17T08:08:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:53 smithi079 ceph-mon[40649]: from='osd.5 ' entity='osd.5' 2024-09-17T08:08:54.275 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:08:54.275 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:08:54.275 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:54.275 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432] boot 2024-09-17T08:08:54.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:08:54.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:54 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:54.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432] boot 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: osd.5 [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432] boot 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T08:08:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:54 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:08:55.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:55 smithi173 ceph-mon[33187]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:08:55.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:55 smithi173 ceph-mon[33187]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:55.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:55.350 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-17T08:08:55.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[29565]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:08:55.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[29565]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:55.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[40649]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[40649]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:55.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:55.438 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-17T08:08:55.451 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-09-17T08:08:55.451 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-09-17T08:08:55.451 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114211 s, 918 MB/s 2024-09-17T08:08:55.452 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-09-17T08:08:56.088 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-09-17T08:08:56.290 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:08:56.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi173 2024-09-17T08:08:56.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: osdmap e43: 6 total, 6 up, 6 in 2024-09-17T08:08:56.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi173 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: osdmap e43: 6 total, 6 up, 6 in 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:56.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:56.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:56.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:56 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi173 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: osdmap e43: 6 total, 6 up, 6 in 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:08:56.410 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:56 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:08:57.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[29565]: osdmap e44: 6 total, 6 up, 6 in 2024-09-17T08:08:57.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[29565]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:57.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[29565]: osdmap e45: 6 total, 6 up, 6 in 2024-09-17T08:08:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[40649]: osdmap e44: 6 total, 6 up, 6 in 2024-09-17T08:08:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[40649]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:57.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:57 smithi079 ceph-mon[40649]: osdmap e45: 6 total, 6 up, 6 in 2024-09-17T08:08:57.390 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:57 smithi173 ceph-mon[33187]: osdmap e44: 6 total, 6 up, 6 in 2024-09-17T08:08:57.390 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:57 smithi173 ceph-mon[33187]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:57.390 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:57 smithi173 ceph-mon[33187]: osdmap e45: 6 total, 6 up, 6 in 2024-09-17T08:08:58.456 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:58 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:58.456 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:58 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:58.456 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:58 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:58.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:58.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:58.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:08:58.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:08:58.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:58 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:08:59.423 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:59 smithi173 ceph-mon[33187]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:59.424 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:08:59 smithi173 ceph-mon[33187]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:59.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:59 smithi079 ceph-mon[29565]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:59.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:08:59 smithi079 ceph-mon[29565]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:08:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:59 smithi079 ceph-mon[40649]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:08:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:08:59 smithi079 ceph-mon[40649]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/322739428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:00 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:00 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]': finished 2024-09-17T08:09:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:00 smithi173 ceph-mon[33187]: osdmap e46: 7 total, 6 up, 7 in 2024-09-17T08:09:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:00 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:00.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/322739428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]': finished 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[29565]: osdmap e46: 7 total, 6 up, 7 in 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/322739428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]: dispatch 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19197da7-927a-4422-8d18-ca9918136919"}]': finished 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[40649]: osdmap e46: 7 total, 6 up, 7 in 2024-09-17T08:09:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:00 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:01 smithi173 ceph-mon[33187]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:01 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:09:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:01 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:09:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/138071794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[40649]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/138071794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[29565]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:09:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/138071794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:03.683 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:03 smithi173 ceph-mon[33187]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:03 smithi079 ceph-mon[40649]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:03 smithi079 ceph-mon[29565]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:05.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:05 smithi173 ceph-mon[33187]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:05 smithi079 ceph-mon[40649]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:05 smithi079 ceph-mon[29565]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:06.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:06 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:09:06.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:06 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:06 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:09:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:06 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:06 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T08:09:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:06 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:07.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:07 smithi173 ceph-mon[33187]: Deploying daemon osd.6 on smithi173 2024-09-17T08:09:07.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:07 smithi173 ceph-mon[33187]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:07 smithi079 ceph-mon[29565]: Deploying daemon osd.6 on smithi173 2024-09-17T08:09:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:07 smithi079 ceph-mon[29565]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:07 smithi079 ceph-mon[40649]: Deploying daemon osd.6 on smithi173 2024-09-17T08:09:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:07 smithi079 ceph-mon[40649]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:09 smithi173 ceph-mon[33187]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:09 smithi079 ceph-mon[40649]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:09 smithi079 ceph-mon[29565]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:11.812 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:11 smithi173 ceph-mon[33187]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:11.812 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:11.812 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:11.812 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[29565]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[40649]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:13.762 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:13 smithi173 ceph-mon[33187]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:13 smithi079 ceph-mon[29565]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:13.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:13 smithi079 ceph-mon[40649]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:14.584 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 6 on host 'smithi173' 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:15.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.196 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:14 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.215 DEBUG:teuthology.orchestra.run.smithi173:osd.6> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service 2024-09-17T08:09:15.217 INFO:tasks.cephadm:Deploying osd.7 on smithi173 with /dev/vg_nvme/lv_1... 2024-09-17T08:09:15.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-17T08:09:15.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:15.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:14 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:16.016 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:16.109 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:15 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:09:16.109 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:15 smithi173 ceph-mon[33187]: osdmap e47: 7 total, 6 up, 7 in 2024-09-17T08:09:16.109 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:15 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.109 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:15 smithi173 ceph-mon[33187]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.109 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:15 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[40649]: osdmap e47: 7 total, 6 up, 7 in 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[40649]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[29565]: osdmap e47: 7 total, 6 up, 7 in 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[29565]: from='osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:15 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:16.392 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[50870]: 2024-09-17T08:09:16.346+0000 7f26cdc51640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T08:09:16.393 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[50870]: 2024-09-17T08:09:16.354+0000 7f26c5a72640 -1 osd.6 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:09:16.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: osdmap e48: 7 total, 6 up, 7 in 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: osdmap e49: 7 total, 6 up, 7 in 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:16.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:16 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: osdmap e48: 7 total, 6 up, 7 in 2024-09-17T08:09:17.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: osdmap e49: 7 total, 6 up, 7 in 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: osdmap e48: 7 total, 6 up, 7 in 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: osdmap e49: 7 total, 6 up, 7 in 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:17.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:16 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:18.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:18.195 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:18.196 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.196 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206] boot 2024-09-17T08:09:18.196 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:09:18.196 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:17 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206] boot 2024-09-17T08:09:18.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: osd.6 [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206] boot 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T08:09:18.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:09:18.664 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-17T08:09:18.748 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-17T08:09:18.765 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-09-17T08:09:18.765 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-09-17T08:09:18.765 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148104 s, 708 MB/s 2024-09-17T08:09:18.765 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-09-17T08:09:19.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: osdmap e51: 7 total, 7 up, 7 in 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:19 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: osdmap e51: 7 total, 7 up, 7 in 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: osdmap e51: 7 total, 7 up, 7 in 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:19.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:19 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:19.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-09-17T08:09:19.819 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:20 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi173 2024-09-17T08:09:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:20 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-17T08:09:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:20 smithi173 ceph-mon[33187]: osdmap e52: 7 total, 7 up, 7 in 2024-09-17T08:09:20.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi173 2024-09-17T08:09:20.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-17T08:09:20.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[29565]: osdmap e52: 7 total, 7 up, 7 in 2024-09-17T08:09:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi173 2024-09-17T08:09:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-17T08:09:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:20 smithi079 ceph-mon[40649]: osdmap e52: 7 total, 7 up, 7 in 2024-09-17T08:09:21.256 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:21 smithi173 ceph-mon[33187]: osdmap e53: 7 total, 7 up, 7 in 2024-09-17T08:09:21.256 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:21 smithi173 ceph-mon[33187]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:21.256 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:21 smithi173 ceph-mon[33187]: osdmap e54: 7 total, 7 up, 7 in 2024-09-17T08:09:21.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[40649]: osdmap e53: 7 total, 7 up, 7 in 2024-09-17T08:09:21.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[40649]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:21.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[40649]: osdmap e54: 7 total, 7 up, 7 in 2024-09-17T08:09:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[29565]: osdmap e53: 7 total, 7 up, 7 in 2024-09-17T08:09:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[29565]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:21 smithi079 ceph-mon[29565]: osdmap e54: 7 total, 7 up, 7 in 2024-09-17T08:09:22.307 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:22 smithi173 ceph-mon[33187]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:22.308 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:09:22.308 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:09:22.308 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:22 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[40649]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:09:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:09:22.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[29565]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T08:09:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T08:09:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:22 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:23.354 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:23 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:23.355 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:23 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:23.355 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:23 smithi173 ceph-mon[33187]: osdmap e55: 7 total, 7 up, 7 in 2024-09-17T08:09:23.355 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:23 smithi173 ceph-mon[33187]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:23.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:23.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[40649]: osdmap e55: 7 total, 7 up, 7 in 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[40649]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[29565]: osdmap e55: 7 total, 7 up, 7 in 2024-09-17T08:09:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:23 smithi079 ceph-mon[29565]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/2999878904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]': finished 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[40649]: osdmap e56: 8 total, 7 up, 8 in 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/2999878904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]': finished 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[29565]: osdmap e56: 8 total, 7 up, 8 in 2024-09-17T08:09:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:24 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:24.622 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:24 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/2999878904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.623 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]: dispatch 2024-09-17T08:09:24.623 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0"}]': finished 2024-09-17T08:09:24.623 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:24 smithi173 ceph-mon[33187]: osdmap e56: 8 total, 7 up, 8 in 2024-09-17T08:09:24.623 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:24 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:25.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:25 smithi079 ceph-mon[40649]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-17T08:09:25.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/3491652352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:25.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:25 smithi079 ceph-mon[29565]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-17T08:09:25.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/3491652352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:25 smithi173 ceph-mon[33187]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-17T08:09:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/3491652352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T08:09:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:27 smithi173 ceph-mon[33187]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:27 smithi079 ceph-mon[40649]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:27 smithi079 ceph-mon[29565]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:29 smithi173 ceph-mon[33187]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-17T08:09:29.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:29 smithi079 ceph-mon[40649]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-17T08:09:29.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:29 smithi079 ceph-mon[29565]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-17T08:09:30.834 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:09:30.834 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:30 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:30.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:09:30.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:30 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T08:09:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:30 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:31.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:31 smithi173 ceph-mon[33187]: Deploying daemon osd.7 on smithi173 2024-09-17T08:09:31.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:31 smithi173 ceph-mon[33187]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:31 smithi079 ceph-mon[29565]: Deploying daemon osd.7 on smithi173 2024-09-17T08:09:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:31 smithi079 ceph-mon[29565]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:31 smithi079 ceph-mon[40649]: Deploying daemon osd.7 on smithi173 2024-09-17T08:09:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:31 smithi079 ceph-mon[40649]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:33.798 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:33 smithi173 ceph-mon[33187]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:09:33.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:33 smithi079 ceph-mon[29565]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:09:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:33 smithi079 ceph-mon[40649]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T08:09:35.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:35 smithi173 ceph-mon[33187]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:35.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:35.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:35.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:35 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:35.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[29565]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:35.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:35.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:35.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[40649]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:35 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:37.389 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:37 smithi173 ceph-mon[33187]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:37.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:37 smithi079 ceph-mon[29565]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:37 smithi079 ceph-mon[40649]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:39.020 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 7 on host 'smithi173' 2024-09-17T08:09:39.323 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.323 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.323 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.324 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:39 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:39 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:39.813 DEBUG:teuthology.orchestra.run.smithi173:osd.7> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.7.service 2024-09-17T08:09:39.816 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-17T08:09:39.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T08:09:40.024 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:09:40.335 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-mon[33187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:09:40.335 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-mon[33187]: osdmap e57: 8 total, 7 up, 8 in 2024-09-17T08:09:40.335 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:40.335 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-mon[33187]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.336 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-mon[33187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.455 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[40649]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:09:40.455 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[40649]: osdmap e57: 8 total, 7 up, 8 in 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[40649]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[40649]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[29565]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[29565]: osdmap e57: 8 total, 7 up, 8 in 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:40.456 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[29565]: from='osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.457 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:40 smithi079 ceph-mon[29565]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:09:40.974 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:09:40.631+0000 7f9e53ec4640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T08:09:40.974 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:09:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:09:40.640+0000 7f9e4b4ce640 -1 osd.7 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: osdmap e58: 8 total, 7 up, 8 in 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:41.386 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: from='osd.7 ' entity='osd.7' 2024-09-17T08:09:41.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325] boot 2024-09-17T08:09:41.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:09:41.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:41 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: osdmap e58: 8 total, 7 up, 8 in 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: from='osd.7 ' entity='osd.7' 2024-09-17T08:09:41.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325] boot 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: osdmap e58: 8 total, 7 up, 8 in 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:41.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: from='osd.7 ' entity='osd.7' 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: osd.7 [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325] boot 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T08:09:41.481 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:41 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:09:42.013 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:09:42.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:42.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:42.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2427101933' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:09:42.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[29565]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:09:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2427101933' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:09:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:42 smithi079 ceph-mon[40649]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:09:42.412 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:42 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:42.412 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:42 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:42.412 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2427101933' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T08:09:42.412 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:42 smithi173 ceph-mon[33187]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T08:09:42.748 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1726560581,"num_in_osds":8,"osd_in_since":1726560564,"num_remapped_pgs":1} 2024-09-17T08:09:42.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:09:42.953 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:09:43.327 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:43.328 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[40649]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.329 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.330 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:43.330 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:43.330 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.330 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:43 smithi079 ceph-mon[29565]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:43.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:43 smithi173 ceph-mon[33187]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T08:09:44.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:44 smithi173 ceph-mon[33187]: Detected new or changed devices on smithi173 2024-09-17T08:09:44.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:44 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-17T08:09:44.495 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:44 smithi079 ceph-mon[40649]: Detected new or changed devices on smithi173 2024-09-17T08:09:44.495 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:44 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-17T08:09:44.495 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:44 smithi079 ceph-mon[29565]: Detected new or changed devices on smithi173 2024-09-17T08:09:44.496 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:44 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-17T08:09:44.949 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:09:44.949 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":62,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","created":"2024-09-17T08:04:41.066270+0000","modified":"2024-09-17T08:09:44.203300+0000","last_up_change":"2024-09-17T08:09:41.129965+0000","last_in_change":"2024-09-17T08:09:24.025600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:07:44.689342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"793e335a-ea62-499c-af46-e48c3190408a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6803","nonce":2553823997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6805","nonce":2553823997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6809","nonce":2553823997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6807","nonce":2553823997}]},"public_addr":"172.21.15.79:6803/2553823997","cluster_addr":"172.21.15.79:6805/2553823997","heartbeat_back_addr":"172.21.15.79:6809/2553823997","heartbeat_front_addr":"172.21.15.79:6807/2553823997","state":["exists","up"]},{"osd":1,"uuid":"dda3e35f-7364-47fa-af06-610c50ac856d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6811","nonce":221772614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6813","nonce":221772614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6817","nonce":221772614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6815","nonce":221772614}]},"public_addr":"172.21.15.79:6811/221772614","cluster_addr":"172.21.15.79:6813/221772614","heartbeat_back_addr":"172.21.15.79:6817/221772614","heartbeat_front_addr":"172.21.15.79:6815/221772614","state":["exists","up"]},{"osd":2,"uuid":"e121b6c9-f67b-4802-a92b-9c709ff433c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6819","nonce":383883652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6821","nonce":383883652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6825","nonce":383883652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6823","nonce":383883652}]},"public_addr":"172.21.15.79:6819/383883652","cluster_addr":"172.21.15.79:6821/383883652","heartbeat_back_addr":"172.21.15.79:6825/383883652","heartbeat_front_addr":"172.21.15.79:6823/383883652","state":["exists","up"]},{"osd":3,"uuid":"69af955a-9dbe-4d86-8278-27df098a3886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6827","nonce":3442203570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6829","nonce":3442203570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6833","nonce":3442203570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6831","nonce":3442203570}]},"public_addr":"172.21.15.79:6827/3442203570","cluster_addr":"172.21.15.79:6829/3442203570","heartbeat_back_addr":"172.21.15.79:6833/3442203570","heartbeat_front_addr":"172.21.15.79:6831/3442203570","state":["exists","up"]},{"osd":4,"uuid":"2ab07ff0-d7f3-4fca-a97b-843f7e69fc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6801","nonce":2625332597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6803","nonce":2625332597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6807","nonce":2625332597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6805","nonce":2625332597}]},"public_addr":"172.21.15.173:6801/2625332597","cluster_addr":"172.21.15.173:6803/2625332597","heartbeat_back_addr":"172.21.15.173:6807/2625332597","heartbeat_front_addr":"172.21.15.173:6805/2625332597","state":["exists","up"]},{"osd":5,"uuid":"2fc412d9-7b28-43b0-b7d3-80fff14855ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6809","nonce":329100432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6811","nonce":329100432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6815","nonce":329100432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6813","nonce":329100432}]},"public_addr":"172.21.15.173:6809/329100432","cluster_addr":"172.21.15.173:6811/329100432","heartbeat_back_addr":"172.21.15.173:6815/329100432","heartbeat_front_addr":"172.21.15.173:6813/329100432","state":["exists","up"]},{"osd":6,"uuid":"19197da7-927a-4422-8d18-ca9918136919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6817","nonce":1332566206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6819","nonce":1332566206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6823","nonce":1332566206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6821","nonce":1332566206}]},"public_addr":"172.21.15.173:6817/1332566206","cluster_addr":"172.21.15.173:6819/1332566206","heartbeat_back_addr":"172.21.15.173:6823/1332566206","heartbeat_front_addr":"172.21.15.173:6821/1332566206","state":["exists","up"]},{"osd":7,"uuid":"b32fec83-5977-488d-b36c-bba34b9f3dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6825","nonce":777262325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6827","nonce":777262325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6831","nonce":777262325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6829","nonce":777262325}]},"public_addr":"172.21.15.173:6825/777262325","cluster_addr":"172.21.15.173:6827/777262325","heartbeat_back_addr":"172.21.15.173:6831/777262325","heartbeat_front_addr":"172.21.15.173:6829/777262325","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:16.492174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:08:47.083146+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:08:47.193284+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:22.186061+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:40.366092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:08:51.564084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:15.284553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:39.512854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/3951775509":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/107735338":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6801/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/1830656502":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6800/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/3326230156":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/128416645":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6801/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1850619208":"2024-09-18T08:05:11.956861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:09:45.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:45 smithi173 ceph-mon[33187]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:09:45.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:45 smithi173 ceph-mon[33187]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:45.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3028843108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:09:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[29565]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:09:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[29565]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3028843108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:09:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[40649]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T08:09:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[40649]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:09:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3028843108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:09:45.700 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T08:09:45.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T08:09:45.895 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:09:46.271 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:46 smithi079 ceph-mon[29565]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:09:46.271 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:46 smithi079 ceph-mon[40649]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:09:46.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:46 smithi173 ceph-mon[33187]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T08:09:47.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:47 smithi079 ceph-mon[29565]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-17T08:09:47.340 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:47 smithi079 ceph-mon[40649]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-17T08:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-09-17T08:09:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:47 smithi173 ceph-mon[33187]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-17T08:09:48.361 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1297912799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:09:48.362 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1297912799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:09:48.383 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi079 2024-09-17T08:09:48.383 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi079=foo.a' 2024-09-17T08:09:48.592 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:48.713 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1297912799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T08:09:49.521 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:49 smithi079 ceph-mon[29565]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:49.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:49 smithi079 ceph-mon[40649]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:49.710 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:49 smithi173 ceph-mon[33187]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T08:09:50.381 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled rgw.foo.a update... 2024-09-17T08:09:51.013 DEBUG:teuthology.orchestra.run.smithi079:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T08:09:51.016 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi173 2024-09-17T08:09:51.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-17T08:09:51.211 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:51.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi079=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:51.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: Deploying daemon rgw.foo.a on smithi079 2024-09-17T08:09:51.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:51 smithi173 ceph-mon[33187]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi079=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.670 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: Deploying daemon rgw.foo.a on smithi079 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[40649]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi079=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.671 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: Deploying daemon rgw.foo.a on smithi079 2024-09-17T08:09:51.672 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:51 smithi079 ceph-mon[29565]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T08:09:52.619 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:09:52 smithi079 podman[68783]: 2024-09-17 08:09:52.53957435 +0000 UTC m=+0.134582118 container create 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, ceph=True) 2024-09-17T08:09:53.119 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:09:52 smithi079 podman[68783]: 2024-09-17 08:09:52.793176515 +0000 UTC m=+0.388184284 container init 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b) 2024-09-17T08:09:53.119 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:09:52 smithi079 podman[68783]: 2024-09-17 08:09:52.798114732 +0000 UTC m=+0.393122504 container start 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-17T08:09:53.119 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:09:52 smithi079 bash[68783]: 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 2024-09-17T08:09:53.120 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:09:52 smithi079 systemd[1]: Started Ceph rgw.foo.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:09:53.489 INFO:teuthology.orchestra.run.smithi173.stderr:pool 'datapool' created 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/126870471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:53 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/126870471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/126870471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:53.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:53 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:09:54.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-17T08:09:54.367 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:54.804 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:09:54.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:54 smithi173 ceph-mon[33187]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:09:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:09:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[40649]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: Saving service rgw.foo.a spec with placement smithi079=foo.a;count:1 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T08:09:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T08:09:54.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:54.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T08:09:54.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:54 smithi079 ceph-mon[29565]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T08:09:55.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:55.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:55.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: pgmap v184: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:09:55.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.593 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: pgmap v184: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: pgmap v184: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T08:09:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: Checking dashboard <-> RGW credentials 2024-09-17T08:09:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/2050788276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[29565]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: Checking dashboard <-> RGW credentials 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/2050788276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:09:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:56 smithi079 ceph-mon[40649]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:09:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: Checking dashboard <-> RGW credentials 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/2050788276' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T08:09:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:56 smithi173 ceph-mon[33187]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T08:09:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[29565]: pgmap v187: 68 pgs: 5 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[29565]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[40649]: pgmap v187: 68 pgs: 5 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T08:09:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[40649]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:09:57.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:57 smithi173 ceph-mon[33187]: pgmap v187: 68 pgs: 5 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T08:09:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:57 smithi173 ceph-mon[33187]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T08:09:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2704626303' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T08:09:59.167 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.173 --placement '1;smithi173=iscsi.a' 2024-09-17T08:09:59.369 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: pgmap v190: 100 pgs: 3 creating+activating, 17 creating+peering, 36 unknown, 44 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.751 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:09:59 smithi173 ceph-mon[33187]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: pgmap v190: 100 pgs: 3 creating+activating, 17 creating+peering, 36 unknown, 44 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[40649]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: pgmap v190: 100 pgs: 3 creating+activating, 17 creating+peering, 36 unknown, 44 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2563695255' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T08:09:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:09:59 smithi079 ceph-mon[29565]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: application not enabled on pool 'default.rgw.control' 2024-09-17T08:10:00.791 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:00 smithi173 ceph-mon[33187]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T08:10:00.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: application not enabled on pool 'default.rgw.control' 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[29565]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T08:10:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: application not enabled on pool 'default.rgw.control' 2024-09-17T08:10:00.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:00 smithi079 ceph-mon[40649]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T08:10:01.186 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled iscsi.datapool update... 2024-09-17T08:10:01.831 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: pgmap v193: 132 pgs: 3 creating+activating, 20 creating+peering, 31 unknown, 78 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T08:10:01.831 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.831 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:10:01.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:01 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:01.857 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-17T08:10:01.857 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:10:01.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: pgmap v193: 132 pgs: 3 creating+activating, 20 creating+peering, 31 unknown, 78 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: pgmap v193: 132 pgs: 3 creating+activating, 20 creating+peering, 31 unknown, 78 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3352227429' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:10:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:01 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:01.886 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:10:01.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T08:10:01.913 DEBUG:teuthology.orchestra.run.smithi173:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-17T08:10:01.957 INFO:tasks.cephadm:Adding prometheus.a on smithi173 2024-09-17T08:10:01.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi173=a' 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.173", "placement": "1;smithi173=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: Saving service iscsi.datapool spec with placement smithi173=iscsi.a;count:1 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:10:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:10:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:02 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:02.772 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:10:02.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.173", "placement": "1;smithi173=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: Saving service iscsi.datapool spec with placement smithi173=iscsi.a;count:1 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:10:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.173", "placement": "1;smithi173=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: Saving service iscsi.datapool spec with placement smithi173=iscsi.a;count:1 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3744348857' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T08:10:02.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:02 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: Deploying daemon iscsi.iscsi.a on smithi173 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: pgmap v196: 132 pgs: 20 creating+peering, 13 unknown, 99 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:10:03.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:03 smithi173 ceph-mon[33187]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:10:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: Deploying daemon iscsi.iscsi.a on smithi173 2024-09-17T08:10:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:10:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:10:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: pgmap v196: 132 pgs: 20 creating+peering, 13 unknown, 99 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[29565]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: Deploying daemon iscsi.iscsi.a on smithi173 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:10:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: pgmap v196: 132 pgs: 20 creating+peering, 13 unknown, 99 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-09-17T08:10:03.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:10:03.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:10:03.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:03 smithi079 ceph-mon[40649]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T08:10:05.018 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled prometheus update... 2024-09-17T08:10:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:05 smithi173 ceph-mon[33187]: pgmap v198: 132 pgs: 8 creating+peering, 124 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.9 KiB/s wr, 105 op/s 2024-09-17T08:10:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:05 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:05.771 DEBUG:teuthology.orchestra.run.smithi173:prometheus.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T08:10:05.773 INFO:tasks.cephadm:Adding node-exporter.a on smithi079 2024-09-17T08:10:05.774 INFO:tasks.cephadm:Adding node-exporter.b on smithi173 2024-09-17T08:10:05.774 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi079=a;smithi173=b' 2024-09-17T08:10:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:05 smithi079 ceph-mon[40649]: pgmap v198: 132 pgs: 8 creating+peering, 124 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.9 KiB/s wr, 105 op/s 2024-09-17T08:10:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:05 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:05 smithi079 ceph-mon[29565]: pgmap v198: 132 pgs: 8 creating+peering, 124 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.9 KiB/s wr, 105 op/s 2024-09-17T08:10:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:05 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:06.095 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:10:06.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:06 smithi173 ceph-mon[33187]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:06.629 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:06 smithi173 ceph-mon[33187]: Saving service prometheus spec with placement smithi173=a;count:1 2024-09-17T08:10:06.629 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:06 smithi173 ceph-mon[33187]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[40649]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[40649]: Saving service prometheus spec with placement smithi173=a;count:1 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[40649]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[29565]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[29565]: Saving service prometheus spec with placement smithi173=a;count:1 2024-09-17T08:10:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:06 smithi079 ceph-mon[29565]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T08:10:07.025 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:06 smithi173 systemd[1]: Started Ceph iscsi.iscsi.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:10:07.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Started the configuration object watcher 2024-09-17T08:10:07.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Checking for config object changes every 1s 2024-09-17T08:10:07.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Processing osd blocklist entries for this node 2024-09-17T08:10:07.579 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 6.1 KiB/s wr, 170 op/s 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:07.580 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/2389453225' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:10:07.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Reading the configuration object to update local LIO configuration 2024-09-17T08:10:07.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Configuration does not have an entry for this host(smithi173.front.sepia.ceph.com) - nothing to define to LIO 2024-09-17T08:10:07.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-17T08:10:07.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: * Environment: production 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: Use a production WSGI server instead. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: * Debug mode: off 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug * Running on all addresses. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: * Running on all addresses. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug * Running on http://[fe80::ec4:7aff:fe88:7259]:5000/ (Press CTRL+C to quit) 2024-09-17T08:10:07.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:07 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: * Running on http://[fe80::ec4:7aff:fe88:7259]:5000/ (Press CTRL+C to quit) 2024-09-17T08:10:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 6.1 KiB/s wr, 170 op/s 2024-09-17T08:10:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/2389453225' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 6.1 KiB/s wr, 170 op/s 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/2389453225' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T08:10:08.267 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled node-exporter update... 2024-09-17T08:10:08.832 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:08 smithi173 ceph-mon[33187]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:10:08.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:08 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:08.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:08 smithi173 ceph-mon[33187]: 1.0 deep-scrub starts 2024-09-17T08:10:08.833 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:08 smithi173 ceph-mon[33187]: 1.0 deep-scrub ok 2024-09-17T08:10:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[40649]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:10:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[40649]: 1.0 deep-scrub starts 2024-09-17T08:10:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[40649]: 1.0 deep-scrub ok 2024-09-17T08:10:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[29565]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T08:10:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[29565]: 1.0 deep-scrub starts 2024-09-17T08:10:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:08 smithi079 ceph-mon[29565]: 1.0 deep-scrub ok 2024-09-17T08:10:09.732 DEBUG:teuthology.orchestra.run.smithi079:node-exporter.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T08:10:09.735 DEBUG:teuthology.orchestra.run.smithi173:node-exporter.b> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T08:10:09.738 INFO:tasks.cephadm:Adding alertmanager.a on smithi079 2024-09-17T08:10:09.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi079=a' 2024-09-17T08:10:09.802 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:09 smithi173 ceph-mon[33187]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi079=a;smithi173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:09.802 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:09 smithi173 ceph-mon[33187]: Saving service node-exporter spec with placement smithi079=a;smithi173=b;count:2 2024-09-17T08:10:09.802 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:09 smithi173 ceph-mon[33187]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-17T08:10:09.802 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:09 smithi173 ceph-mon[33187]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:10:09.803 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:09 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[40649]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi079=a;smithi173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[40649]: Saving service node-exporter spec with placement smithi079=a;smithi173=b;count:2 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[40649]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[40649]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[29565]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi079=a;smithi173=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[29565]: Saving service node-exporter spec with placement smithi079=a;smithi173=b;count:2 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[29565]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[29565]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T08:10:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:09 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:10.714 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.0 KiB/s wr, 133 op/s 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:11.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:11 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.0 KiB/s wr, 133 op/s 2024-09-17T08:10:12.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.0 KiB/s wr, 133 op/s 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:12.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:11 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:12.462 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled alertmanager update... 2024-09-17T08:10:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:12 smithi173 ceph-mon[33187]: Deploying daemon prometheus.a on smithi173 2024-09-17T08:10:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:12 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:13.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:12 smithi079 ceph-mon[40649]: Deploying daemon prometheus.a on smithi173 2024-09-17T08:10:13.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:12 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:12 smithi079 ceph-mon[29565]: Deploying daemon prometheus.a on smithi173 2024-09-17T08:10:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:12 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:13.234 DEBUG:teuthology.orchestra.run.smithi079:alertmanager.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T08:10:13.237 INFO:tasks.cephadm:Adding grafana.a on smithi173 2024-09-17T08:10:13.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi173=a' 2024-09-17T08:10:13.469 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:10:13.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:13 smithi173 ceph-mon[33187]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 3.2 KiB/s wr, 105 op/s 2024-09-17T08:10:13.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:13 smithi173 ceph-mon[33187]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi079=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:13.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:13 smithi173 ceph-mon[33187]: Saving service alertmanager spec with placement smithi079=a;count:1 2024-09-17T08:10:14.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[40649]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 3.2 KiB/s wr, 105 op/s 2024-09-17T08:10:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[40649]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi079=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[40649]: Saving service alertmanager spec with placement smithi079=a;count:1 2024-09-17T08:10:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[29565]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 3.2 KiB/s wr, 105 op/s 2024-09-17T08:10:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[29565]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi079=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:13 smithi079 ceph-mon[29565]: Saving service alertmanager spec with placement smithi079=a;count:1 2024-09-17T08:10:15.673 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:15 smithi173 ceph-mon[33187]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 61 op/s 2024-09-17T08:10:15.673 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:15 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:15.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:15 smithi079 ceph-mon[40649]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 61 op/s 2024-09-17T08:10:15.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:15 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:15 smithi079 ceph-mon[29565]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 61 op/s 2024-09-17T08:10:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:15 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:16.137 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled grafana update... 2024-09-17T08:10:17.247 DEBUG:teuthology.orchestra.run.smithi173:grafana.a> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T08:10:17.250 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T08:10:17.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:10:17.458 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:17.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:17 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:10:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:17 smithi173 ceph-mon[33187]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:17 smithi173 ceph-mon[33187]: Saving service grafana spec with placement smithi173=a;count:1 2024-09-17T08:10:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:17 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:17 smithi173 ceph-mon[33187]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 298 B/s wr, 12 op/s 2024-09-17T08:10:17.578 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[40649]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:17.578 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[40649]: Saving service grafana spec with placement smithi173=a;count:1 2024-09-17T08:10:17.578 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:17.578 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[40649]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 298 B/s wr, 12 op/s 2024-09-17T08:10:17.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[29565]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi173=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T08:10:17.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[29565]: Saving service grafana spec with placement smithi173=a;count:1 2024-09-17T08:10:17.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:17.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:17 smithi079 ceph-mon[29565]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 298 B/s wr, 12 op/s 2024-09-17T08:10:18.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:18 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:18.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:18 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:18.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:18 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-09-17T08:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBrOelm6u61FRAA9hfOJLRitMk9Vwhd6Sxxaw== 2024-09-17T08:10:19.407 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:19 smithi079 ceph-mon[29565]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-17T08:10:19.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:19 smithi079 ceph-mon[40649]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-17T08:10:19.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:19 smithi173 ceph-mon[33187]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-17T08:10:20.016 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-17T08:10:20.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T08:10:20.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T08:10:20.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T08:10:20.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:20 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2065938075' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:20 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.357 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:20 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:20.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2065938075' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:20.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2065938075' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:20 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:21.414 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:21 smithi173 ceph-mon[33187]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-17T08:10:21.517 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:10:21.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:21 smithi079 ceph-mon[29565]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-17T08:10:21.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:21 smithi079 ceph-mon[40649]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-17T08:10:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:23 smithi079 ceph-mon[29565]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:23 smithi079 ceph-mon[40649]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:23.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:23 smithi173 ceph-mon[33187]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:24.756 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2024-09-17T08:10:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBwOelmhxKmLBAAPzkIMlTd+6JKk0x37sBOEQ== 2024-09-17T08:10:25.335 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:10:25 smithi173 systemd[1]: Started Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:10:25.619 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.173:0/3539596555' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:25 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:10:25.640 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T08:10:25.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T08:10:25.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T08:10:25.677 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T08:10:25.677 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T08:10:25.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T08:10:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.173:0/3539596555' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.173:0/3539596555' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T08:10:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T08:10:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' 2024-09-17T08:10:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:25 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T08:10:25.883 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:26.474 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: ignoring --setuser ceph since I am not root 2024-09-17T08:10:26.474 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: ignoring --setgroup ceph since I am not root 2024-09-17T08:10:26.474 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.395+0000 7ff69a77b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:10:26.474 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.449+0000 7ff69a77b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:10:26.608 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setuser ceph since I am not root 2024-09-17T08:10:26.608 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ignoring --setgroup ceph since I am not root 2024-09-17T08:10:26.608 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.399+0000 7fc1f3e77040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T08:10:26.608 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.456+0000 7fc1f3e77040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T08:10:26.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.607+0000 7fc1f3e77040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:10:26.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.674+0000 7fc1f3e77040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:10:26.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.820+0000 7fc1f3e77040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:10:26.941 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.597+0000 7ff69a77b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T08:10:26.941 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.661+0000 7ff69a77b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T08:10:26.941 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.803+0000 7ff69a77b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:10:26.941 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:26.940+0000 7ff69a77b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:10:27.255 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:26.963+0000 7fc1f3e77040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T08:10:27.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-mon[40649]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:10:27.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-mon[40649]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T08:10:27.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-mon[29565]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:10:27.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-mon[29565]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T08:10:27.532 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:27.279+0000 7fc1f3e77040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:10:27.532 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:27.340+0000 7fc1f3e77040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:10:27.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:27.243+0000 7ff69a77b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T08:10:27.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:27.302+0000 7ff69a77b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T08:10:27.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:27.480+0000 7ff69a77b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:10:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:27 smithi173 ceph-mon[33187]: from='mgr.14150 172.21.15.79:0/2587759989' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T08:10:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:27 smithi173 ceph-mon[33187]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T08:10:27.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:27 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:27.531+0000 7fc1f3e77040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T08:10:27.952 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:10:28.504 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:28 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4152026783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:10:28.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4152026783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:10:28.505 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:28 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:28.437+0000 7fc1f3e77040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:10:28.677 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3253737241},{"type":"v1","addr":"172.21.15.79:6801","nonce":3253737241}]},"active_addr":"172.21.15.79:6801/3253737241","active_change":"2024-09-17T08:05:34.382641+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.79:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2554615564}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2023211636}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3019822778}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2454308049}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3759482241}]}]} 2024-09-17T08:10:28.684 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T08:10:28.684 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T08:10:28.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:10:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:28 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4152026783' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T08:10:28.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:28.337+0000 7ff69a77b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T08:10:28.724 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:28.401+0000 7ff69a77b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:10:28.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:28 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:28.504+0000 7fc1f3e77040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T08:10:28.892 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:29.284 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.264+0000 7fc1f3e77040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:10:29.357 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.104+0000 7ff69a77b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T08:10:29.357 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.217+0000 7ff69a77b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:10:29.595 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.391+0000 7fc1f3e77040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T08:10:29.595 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.533+0000 7fc1f3e77040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:10:29.628 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.357+0000 7ff69a77b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T08:10:29.628 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.417+0000 7ff69a77b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:10:29.628 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.561+0000 7ff69a77b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:10:29.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.594+0000 7fc1f3e77040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T08:10:29.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.747+0000 7fc1f3e77040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T08:10:29.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.814+0000 7fc1f3e77040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:10:29.903 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.627+0000 7ff69a77b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T08:10:29.903 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.686+0000 7ff69a77b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:10:29.903 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.839+0000 7ff69a77b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:10:30.128 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:29 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:29.874+0000 7fc1f3e77040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T08:10:30.128 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.032+0000 7fc1f3e77040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T08:10:30.128 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.098+0000 7fc1f3e77040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:10:30.165 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.903+0000 7ff69a77b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T08:10:30.166 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:29.968+0000 7ff69a77b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:10:30.166 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:30.048+0000 7ff69a77b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:10:30.166 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:30.107+0000 7ff69a77b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:10:30.411 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.169+0000 7fc1f3e77040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T08:10:30.411 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.256+0000 7fc1f3e77040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T08:10:30.412 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.316+0000 7fc1f3e77040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T08:10:30.412 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:30.375+0000 7fc1f3e77040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:10:30.473 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:30.165+0000 7ff69a77b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T08:10:30.818 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:10:30.818 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":74,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","created":"2024-09-17T08:04:41.066270+0000","modified":"2024-09-17T08:10:06.135971+0000","last_up_change":"2024-09-17T08:09:41.129965+0000","last_in_change":"2024-09-17T08:09:24.025600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:07:44.689342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:09:53.075171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-17T08:09:53.309442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:09:54.771473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:09:56.745903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:09:58.808366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"793e335a-ea62-499c-af46-e48c3190408a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6803","nonce":2553823997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6805","nonce":2553823997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6809","nonce":2553823997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6807","nonce":2553823997}]},"public_addr":"172.21.15.79:6803/2553823997","cluster_addr":"172.21.15.79:6805/2553823997","heartbeat_back_addr":"172.21.15.79:6809/2553823997","heartbeat_front_addr":"172.21.15.79:6807/2553823997","state":["exists","up"]},{"osd":1,"uuid":"dda3e35f-7364-47fa-af06-610c50ac856d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6811","nonce":221772614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6813","nonce":221772614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6817","nonce":221772614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6815","nonce":221772614}]},"public_addr":"172.21.15.79:6811/221772614","cluster_addr":"172.21.15.79:6813/221772614","heartbeat_back_addr":"172.21.15.79:6817/221772614","heartbeat_front_addr":"172.21.15.79:6815/221772614","state":["exists","up"]},{"osd":2,"uuid":"e121b6c9-f67b-4802-a92b-9c709ff433c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6819","nonce":383883652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6821","nonce":383883652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6825","nonce":383883652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6823","nonce":383883652}]},"public_addr":"172.21.15.79:6819/383883652","cluster_addr":"172.21.15.79:6821/383883652","heartbeat_back_addr":"172.21.15.79:6825/383883652","heartbeat_front_addr":"172.21.15.79:6823/383883652","state":["exists","up"]},{"osd":3,"uuid":"69af955a-9dbe-4d86-8278-27df098a3886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6827","nonce":3442203570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6829","nonce":3442203570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6833","nonce":3442203570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6831","nonce":3442203570}]},"public_addr":"172.21.15.79:6827/3442203570","cluster_addr":"172.21.15.79:6829/3442203570","heartbeat_back_addr":"172.21.15.79:6833/3442203570","heartbeat_front_addr":"172.21.15.79:6831/3442203570","state":["exists","up"]},{"osd":4,"uuid":"2ab07ff0-d7f3-4fca-a97b-843f7e69fc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6801","nonce":2625332597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6803","nonce":2625332597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6807","nonce":2625332597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6805","nonce":2625332597}]},"public_addr":"172.21.15.173:6801/2625332597","cluster_addr":"172.21.15.173:6803/2625332597","heartbeat_back_addr":"172.21.15.173:6807/2625332597","heartbeat_front_addr":"172.21.15.173:6805/2625332597","state":["exists","up"]},{"osd":5,"uuid":"2fc412d9-7b28-43b0-b7d3-80fff14855ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6809","nonce":329100432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6811","nonce":329100432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6815","nonce":329100432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6813","nonce":329100432}]},"public_addr":"172.21.15.173:6809/329100432","cluster_addr":"172.21.15.173:6811/329100432","heartbeat_back_addr":"172.21.15.173:6815/329100432","heartbeat_front_addr":"172.21.15.173:6813/329100432","state":["exists","up"]},{"osd":6,"uuid":"19197da7-927a-4422-8d18-ca9918136919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6817","nonce":1332566206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6819","nonce":1332566206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6823","nonce":1332566206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6821","nonce":1332566206}]},"public_addr":"172.21.15.173:6817/1332566206","cluster_addr":"172.21.15.173:6819/1332566206","heartbeat_back_addr":"172.21.15.173:6823/1332566206","heartbeat_front_addr":"172.21.15.173:6821/1332566206","state":["exists","up"]},{"osd":7,"uuid":"b32fec83-5977-488d-b36c-bba34b9f3dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6825","nonce":777262325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6827","nonce":777262325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6831","nonce":777262325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6829","nonce":777262325}]},"public_addr":"172.21.15.173:6825/777262325","cluster_addr":"172.21.15.173:6827/777262325","heartbeat_back_addr":"172.21.15.173:6831/777262325","heartbeat_front_addr":"172.21.15.173:6829/777262325","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:16.492174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:54.281193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:58.129013+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:22.186061+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:40.366092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:10:02.356197+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:15.284553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:39.512854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/3951775509":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/107735338":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6801/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/1830656502":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6800/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/3326230156":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/128416645":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6801/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1850619208":"2024-09-18T08:05:11.956861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:10:31.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2202634893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2202634893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:30.935+0000 7ff69a77b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: [17/Sep/2024:08:10:30] ENGINE Bus STARTING 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: CherryPy Checker: 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: The Application mounted at '' has an empty config. 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: [17/Sep/2024:08:10:31] ENGINE Serving on http://:::9283 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x[35238]: [17/Sep/2024:08:10:31] ENGINE Bus STARTED 2024-09-17T08:10:31.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:30 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2202634893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:31.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:31.194+0000 7fc1f3e77040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T08:10:31.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:10:31] ENGINE Bus STARTING 2024-09-17T08:10:31.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: CherryPy Checker: 2024-09-17T08:10:31.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: The Application mounted at '' has an empty config. 2024-09-17T08:10:31.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:31.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:10:31] ENGINE Serving on http://:::9283 2024-09-17T08:10:31.606 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T08:10:31.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:10:31.630 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:10:31] ENGINE Bus STARTED 2024-09-17T08:10:31.813 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:31.938 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: Standby manager daemon x restarted 2024-09-17T08:10:31.938 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: Standby manager daemon x started 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: Active manager daemon y restarted 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: Activating manager daemon y 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: mgrmap e18: y(active, starting, since 0.00757205s), standbys: x 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:10:31.939 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:10:31.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: Manager daemon y is now available 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: Standby manager daemon x restarted 2024-09-17T08:10:31.941 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: Standby manager daemon x started 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: Active manager daemon y restarted 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: Activating manager daemon y 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: mgrmap e18: y(active, starting, since 0.00757205s), standbys: x 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:10:31.942 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:10:31.943 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: Manager daemon y is now available 2024-09-17T08:10:31.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:31.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:31.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:31.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:10:31.944 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:10:32.123 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: Standby manager daemon x restarted 2024-09-17T08:10:32.123 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: Standby manager daemon x started 2024-09-17T08:10:32.123 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T08:10:32.123 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.? 172.21.15.173:0/3619179543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: Active manager daemon y restarted 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: Activating manager daemon y 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: mgrmap e18: y(active, starting, since 0.00757205s), standbys: x 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T08:10:32.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: Manager daemon y is now available 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:10:32.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:32.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:32.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T08:10:32.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T08:10:32.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:10:33.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[40649]: mgrmap e19: y(active, since 1.01237s), standbys: x 2024-09-17T08:10:33.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:33.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[40649]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[29565]: mgrmap e19: y(active, since 1.01237s), standbys: x 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-mon[29565]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:10:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:33.060+0000 7fc151b71640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:08:10:33] ENGINE Error in HTTPServer.serve 2024-09-17T08:10:33.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: Traceback (most recent call last): 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._connections.run(self.expiration_interval) 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._run(expiration_interval) 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: s = self.context.wrap_socket( 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: return self.sslsocket_class._create( 2024-09-17T08:10:33.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:10:33.372 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self.do_handshake() 2024-09-17T08:10:33.372 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:10:33.372 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: self._sslobj.do_handshake() 2024-09-17T08:10:33.372 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:10:33.372 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:10:33.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:33 smithi173 ceph-mon[33187]: mgrmap e19: y(active, since 1.01237s), standbys: x 2024-09-17T08:10:33.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:33.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:33 smithi173 ceph-mon[33187]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:10:33.761 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":75,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","created":"2024-09-17T08:04:41.066270+0000","modified":"2024-09-17T08:10:31.196170+0000","last_up_change":"2024-09-17T08:09:41.129965+0000","last_in_change":"2024-09-17T08:09:24.025600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:07:44.689342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:09:53.075171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-17T08:09:53.309442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:09:54.771473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:09:56.745903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:09:58.808366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"793e335a-ea62-499c-af46-e48c3190408a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6803","nonce":2553823997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6805","nonce":2553823997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6809","nonce":2553823997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6807","nonce":2553823997}]},"public_addr":"172.21.15.79:6803/2553823997","cluster_addr":"172.21.15.79:6805/2553823997","heartbeat_back_addr":"172.21.15.79:6809/2553823997","heartbeat_front_addr":"172.21.15.79:6807/2553823997","state":["exists","up"]},{"osd":1,"uuid":"dda3e35f-7364-47fa-af06-610c50ac856d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6811","nonce":221772614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6813","nonce":221772614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6817","nonce":221772614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6815","nonce":221772614}]},"public_addr":"172.21.15.79:6811/221772614","cluster_addr":"172.21.15.79:6813/221772614","heartbeat_back_addr":"172.21.15.79:6817/221772614","heartbeat_front_addr":"172.21.15.79:6815/221772614","state":["exists","up"]},{"osd":2,"uuid":"e121b6c9-f67b-4802-a92b-9c709ff433c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6819","nonce":383883652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6821","nonce":383883652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6825","nonce":383883652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":383883652},{"type":"v1","addr":"172.21.15.79:6823","nonce":383883652}]},"public_addr":"172.21.15.79:6819/383883652","cluster_addr":"172.21.15.79:6821/383883652","heartbeat_back_addr":"172.21.15.79:6825/383883652","heartbeat_front_addr":"172.21.15.79:6823/383883652","state":["exists","up"]},{"osd":3,"uuid":"69af955a-9dbe-4d86-8278-27df098a3886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6827","nonce":3442203570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6829","nonce":3442203570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6833","nonce":3442203570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6831","nonce":3442203570}]},"public_addr":"172.21.15.79:6827/3442203570","cluster_addr":"172.21.15.79:6829/3442203570","heartbeat_back_addr":"172.21.15.79:6833/3442203570","heartbeat_front_addr":"172.21.15.79:6831/3442203570","state":["exists","up"]},{"osd":4,"uuid":"2ab07ff0-d7f3-4fca-a97b-843f7e69fc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6801","nonce":2625332597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6803","nonce":2625332597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6807","nonce":2625332597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6805","nonce":2625332597}]},"public_addr":"172.21.15.173:6801/2625332597","cluster_addr":"172.21.15.173:6803/2625332597","heartbeat_back_addr":"172.21.15.173:6807/2625332597","heartbeat_front_addr":"172.21.15.173:6805/2625332597","state":["exists","up"]},{"osd":5,"uuid":"2fc412d9-7b28-43b0-b7d3-80fff14855ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6809","nonce":329100432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6811","nonce":329100432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6815","nonce":329100432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6813","nonce":329100432}]},"public_addr":"172.21.15.173:6809/329100432","cluster_addr":"172.21.15.173:6811/329100432","heartbeat_back_addr":"172.21.15.173:6815/329100432","heartbeat_front_addr":"172.21.15.173:6813/329100432","state":["exists","up"]},{"osd":6,"uuid":"19197da7-927a-4422-8d18-ca9918136919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6817","nonce":1332566206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6819","nonce":1332566206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6823","nonce":1332566206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1332566206},{"type":"v1","addr":"172.21.15.173:6821","nonce":1332566206}]},"public_addr":"172.21.15.173:6817/1332566206","cluster_addr":"172.21.15.173:6819/1332566206","heartbeat_back_addr":"172.21.15.173:6823/1332566206","heartbeat_front_addr":"172.21.15.173:6821/1332566206","state":["exists","up"]},{"osd":7,"uuid":"b32fec83-5977-488d-b36c-bba34b9f3dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6825","nonce":777262325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6827","nonce":777262325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6831","nonce":777262325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6829","nonce":777262325}]},"public_addr":"172.21.15.173:6825/777262325","cluster_addr":"172.21.15.173:6827/777262325","heartbeat_back_addr":"172.21.15.173:6831/777262325","heartbeat_front_addr":"172.21.15.173:6829/777262325","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:16.492174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:54.281193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:58.129013+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:22.186061+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:40.366092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:10:02.356197+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:10:31.144171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:09:39.512854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/3019822778":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3759482241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/2023211636":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/128416645":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2554615564":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3326230156":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1830656502":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1850619208":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6801/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2454308049":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/107735338":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/3951775509":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6800/3253737241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/3253737241":"2024-09-18T08:10:31.196134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:10:34.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: [17/Sep/2024:08:10:32] ENGINE Bus STARTING 2024-09-17T08:10:34.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: [17/Sep/2024:08:10:33] ENGINE Serving on https://172.21.15.79:7150 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: [17/Sep/2024:08:10:33] ENGINE Error in HTTPServer.serve 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: Traceback (most recent call last): 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: self._run(expiration_interval) 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:10:34.370 INFO:journalctl@ceph.mon.c.smithi079.stdout: s = self.context.wrap_socket( 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: return self.sslsocket_class._create( 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: self.do_handshake() 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: self._sslobj.do_handshake() 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: [17/Sep/2024:08:10:33] ENGINE Serving on http://172.21.15.79:8765 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: [17/Sep/2024:08:10:33] ENGINE Bus STARTED 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-17T08:10:34.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/14375137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:34.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: [17/Sep/2024:08:10:32] ENGINE Bus STARTING 2024-09-17T08:10:34.372 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: [17/Sep/2024:08:10:33] ENGINE Serving on https://172.21.15.79:7150 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: [17/Sep/2024:08:10:33] ENGINE Error in HTTPServer.serve 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: Traceback (most recent call last): 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._run(expiration_interval) 2024-09-17T08:10:34.373 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:10:34.374 INFO:journalctl@ceph.mon.a.smithi079.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:10:34.374 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:10:34.374 INFO:journalctl@ceph.mon.a.smithi079.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:10:34.374 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:10:34.374 INFO:journalctl@ceph.mon.a.smithi079.stdout: s = self.context.wrap_socket( 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: return self.sslsocket_class._create( 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: self.do_handshake() 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:10:34.375 INFO:journalctl@ceph.mon.a.smithi079.stdout: self._sslobj.do_handshake() 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: [17/Sep/2024:08:10:33] ENGINE Serving on http://172.21.15.79:8765 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: [17/Sep/2024:08:10:33] ENGINE Bus STARTED 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-17T08:10:34.376 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/14375137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:34.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.377 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:34 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: [17/Sep/2024:08:10:32] ENGINE Bus STARTING 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: [17/Sep/2024:08:10:33] ENGINE Serving on https://172.21.15.79:7150 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: [17/Sep/2024:08:10:33] ENGINE Error in HTTPServer.serve 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: Traceback (most recent call last): 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: self._connections.run(self.expiration_interval) 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: self._run(expiration_interval) 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T08:10:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout: s = self.context.wrap_socket( 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: return self.sslsocket_class._create( 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: self.do_handshake() 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: self._sslobj.do_handshake() 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: [17/Sep/2024:08:10:33] ENGINE Serving on http://172.21.15.79:8765 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: [17/Sep/2024:08:10:33] ENGINE Bus STARTED 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/14375137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:10:34.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:34 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:34.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T08:10:34.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T08:10:34.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T08:10:34.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-17T08:10:34.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-17T08:10:34.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-17T08:10:34.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-17T08:10:34.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-17T08:10:35.170 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.174 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.177 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.193 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.194 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.243 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.243 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.249 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:35.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:10:35.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:10:35.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.554 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:35 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:35 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 3760M 2024-09-17T08:10:36.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:36.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T08:10:36.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[40649]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-17T08:10:36.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 3760M 2024-09-17T08:10:36.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:36.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T08:10:36.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:36 smithi079 ceph-mon[29565]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-17T08:10:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 3760M 2024-09-17T08:10:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T08:10:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:36 smithi173 ceph-mon[33187]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-17T08:10:38.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:38 smithi079 ceph-mon[40649]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:38.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:38 smithi079 ceph-mon[29565]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:38 smithi173 ceph-mon[33187]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 222 MiB used, 715 GiB / 715 GiB avail 2024-09-17T08:10:40.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[40649]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[40649]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[29565]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:10:40.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:40 smithi079 ceph-mon[29565]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T08:10:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:40 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:10:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:40 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:10:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:40 smithi173 ceph-mon[33187]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T08:10:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:40 smithi173 ceph-mon[33187]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T08:10:42.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:42.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:42.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:42.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:42.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:10:42.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout:77309411364 2024-09-17T08:10:42.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T08:10:43.150 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:10:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:10:43.151 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 3760M 2024-09-17T08:10:43.151 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:10:43.151 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:10:43.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 3760M 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:43 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 3760M 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi079:/etc/ceph/ceph.conf 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: Updating smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.500 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.500 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:43.500 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:43 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:10:44.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:44 smithi079 ceph-mon[40649]: Deploying daemon alertmanager.a on smithi079 2024-09-17T08:10:44.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:44 smithi079 ceph-mon[40649]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:44.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:44 smithi079 ceph-mon[29565]: Deploying daemon alertmanager.a on smithi079 2024-09-17T08:10:44.373 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:44 smithi079 ceph-mon[29565]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:44.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:44 smithi173 ceph-mon[33187]: Deploying daemon alertmanager.a on smithi079 2024-09-17T08:10:44.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:44 smithi173 ceph-mon[33187]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-17T08:10:45.265 INFO:teuthology.orchestra.run.smithi079.stdout:253403070479 2024-09-17T08:10:45.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-17T08:10:46.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:46 smithi079 ceph-mon[40649]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:10:46.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:46 smithi079 ceph-mon[29565]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:10:46.545 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:46 smithi173 ceph-mon[33187]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-17T08:10:47.528 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:47.528 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:10:47.993 INFO:teuthology.orchestra.run.smithi079.stdout:214748364820 2024-09-17T08:10:47.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-17T08:10:48.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:48 smithi079 ceph-mon[29565]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:10:48.563 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:48 smithi079 ceph-mon[40649]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:10:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:48 smithi173 ceph-mon[33187]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T08:10:49.499 INFO:teuthology.orchestra.run.smithi079.stdout:137438953503 2024-09-17T08:10:49.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-17T08:10:50.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:50 smithi079 ceph-mon[29565]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:10:50.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:50 smithi079 ceph-mon[40649]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:10:50.455 INFO:teuthology.orchestra.run.smithi079.stdout:34359738415 2024-09-17T08:10:50.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T08:10:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:50 smithi173 ceph-mon[33187]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:10:51.595 INFO:teuthology.orchestra.run.smithi079.stdout:115964117026 2024-09-17T08:10:51.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-17T08:10:51.647 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:52.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:52 smithi079 ceph-mon[29565]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:10:52.432 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:52 smithi079 ceph-mon[40649]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:10:52.453 INFO:teuthology.orchestra.run.smithi079.stdout:176093659162 2024-09-17T08:10:52.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-17T08:10:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:52 smithi173 ceph-mon[33187]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:10:52.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:10:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:10:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:52.873 INFO:teuthology.orchestra.run.smithi079.stdout:55834574893 2024-09-17T08:10:52.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T08:10:52.891 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:52.895 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:53.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:10:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:10:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:10:53.465 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:53.466 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:10:54.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:54 smithi079 ceph-mon[29565]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:54.579 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:54 smithi079 ceph-mon[40649]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:54 smithi173 ceph-mon[33187]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:55.301 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:55 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:10:55.321 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:55 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:10:55.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:55 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:10:55.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:55 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:10:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:55 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:10:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:55 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:10:56.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[29565]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T08:10:56.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:10:56.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:10:56.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:56 smithi079 ceph-mon[40649]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T08:10:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:10:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:56 smithi173 ceph-mon[33187]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T08:10:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:57 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:10:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:57 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:10:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:57 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:10:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:57 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:10:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:10:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:57 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:10:58.026 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:10:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:10:58.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[40649]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:10:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:10:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:58 smithi079 ceph-mon[29565]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:58 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:10:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:58 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:10:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:58 smithi173 ceph-mon[33187]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:10:58.914 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:58.914 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:10:59.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:59 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:10:59.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:10:59 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:10:59.618 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:59 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:10:59.618 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:10:59 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:10:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:59 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:10:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:10:59 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:11:00.431 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[40649]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[29565]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:11:00.432 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:00 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:11:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:00 smithi173 ceph-mon[33187]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-17T08:11:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:00 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:11:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:00 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:11:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:00 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:11:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:00 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:11:01.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:11:01.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:11:01.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:11:01.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:11:01.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T08:11:01.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:01 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:11:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:11:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:11:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:11:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T08:11:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:01 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:02.602 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:02.603 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:11:02.604 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:02 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:02.623 INFO:teuthology.orchestra.run.smithi079.stdout:176093659164 2024-09-17T08:11:02.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:11:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:11:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:02 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:03.321 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:11:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:11:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/512346117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 5.8 deep-scrub starts 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: 5.8 deep-scrub ok 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[40649]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:11:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/512346117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 5.8 deep-scrub starts 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: 5.8 deep-scrub ok 2024-09-17T08:11:03.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:03 smithi079 ceph-mon[29565]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/512346117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 5.8 deep-scrub starts 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: 5.8 deep-scrub ok 2024-09-17T08:11:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:03 smithi173 ceph-mon[33187]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:03.805 INFO:teuthology.orchestra.run.smithi079.stdout:214748364824 2024-09-17T08:11:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:11:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:11:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2636520551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:11:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:11:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:11:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:04 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2636520551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:11:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:04 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:11:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:04 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:11:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:04 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2636520551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout:34359738418 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 6.12 deep-scrub starts 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 6.12 deep-scrub ok 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:11:05.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[29565]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 6.12 deep-scrub starts 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 6.12 deep-scrub ok 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:11:05.441 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:05 smithi079 ceph-mon[40649]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 6.12 deep-scrub starts 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 6.12 deep-scrub ok 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:11:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:05 smithi173 ceph-mon[33187]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:05.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659162 got 176093659164 for osd.5 2024-09-17T08:11:05.789 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:06.464 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3477271162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:11:06.465 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3477271162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:11:06.466 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:06 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3477271162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:11:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:06 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:11:07.025 INFO:teuthology.orchestra.run.smithi079.stdout:77309411369 2024-09-17T08:11:07.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738418 for osd.0 2024-09-17T08:11:07.426 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:11:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:11:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3371121722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:11:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[29565]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-17T08:11:07.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:11:07.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:11:07.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3371121722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:11:07.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:07 smithi079 ceph-mon[40649]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-17T08:11:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:07 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:11:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:07 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:11:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3371121722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:11:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:07 smithi173 ceph-mon[33187]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-17T08:11:07.787 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364820 got 214748364824 for osd.6 2024-09-17T08:11:07.787 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:08.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411364 got 77309411369 for osd.2 2024-09-17T08:11:08.371 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:11:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:11:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:11:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:11:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:11:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:08 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:11:08.712 INFO:teuthology.orchestra.run.smithi079.stdout:115964117031 2024-09-17T08:11:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:11:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:11:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:11:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:11:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:11:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:08 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:11:09.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 5.11 deep-scrub starts 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 5.11 deep-scrub ok 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1014423381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:11:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[40649]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 5.11 deep-scrub starts 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 5.11 deep-scrub ok 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:11:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:11:09.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1014423381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:11:09.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:09 smithi079 ceph-mon[29565]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 5.11 deep-scrub starts 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 5.11 deep-scrub ok 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1014423381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:11:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:09 smithi173 ceph-mon[33187]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:09.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117026 got 115964117031 for osd.3 2024-09-17T08:11:09.810 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:10.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:10 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:11:10.604 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:10 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:11:10.605 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:10 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:11:10.605 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:10 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:11:10.637 INFO:teuthology.orchestra.run.smithi079.stdout:55834574897 2024-09-17T08:11:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:10 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:11:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:10 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:11:10.998 INFO:teuthology.orchestra.run.smithi079.stdout:137438953507 2024-09-17T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:11:11.613 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:11:11.613 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1050114285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:11:11.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2004692202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:11:11.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[29565]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 0 op/s 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1050114285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2004692202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T08:11:11.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:11 smithi079 ceph-mon[40649]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 0 op/s 2024-09-17T08:11:11.632 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953503 got 137438953507 for osd.4 2024-09-17T08:11:11.632 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1050114285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2004692202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T08:11:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:11 smithi173 ceph-mon[33187]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 0 op/s 2024-09-17T08:11:12.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:11:12.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:11:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:11:12.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:11:12.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:12 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:12.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:11:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:11:12.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:11:12.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:12 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:13.149 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:13.406 INFO:teuthology.orchestra.run.smithi079.stdout:253403070485 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 4.6 deep-scrub starts 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 4.6 deep-scrub ok 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[40649]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:11:13.407 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 4.6 deep-scrub starts 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 4.6 deep-scrub ok 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:11:13.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:13 smithi079 ceph-mon[29565]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:11:13.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574897 for osd.1 2024-09-17T08:11:13.615 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 4.6 deep-scrub starts 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 4.6 deep-scrub ok 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:11:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:13 smithi173 ceph-mon[33187]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:11:14.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070479 got 253403070485 for osd.7 2024-09-17T08:11:14.327 DEBUG:teuthology.parallel:result is None 2024-09-17T08:11:14.327 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T08:11:14.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:14.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3411835761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:11:14.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3411835761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:11:14.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:14 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:11:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3411835761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:11:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:14 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:11:14.957 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:11:15.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:11:15.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:11:15.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[29565]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:11:15.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:11:15.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:11:15.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:11:15.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:15 smithi079 ceph-mon[40649]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:11:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:15 smithi173 ceph-mon[33187]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:11:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:11:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:11:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:11:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:11:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:16 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:16.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:16 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:11:16.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:16 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:11:16.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:16 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:17.091 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:16 smithi079 systemd[1]: Started Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:17.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE Bus STOPPING 2024-09-17T08:11:17.636 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE Bus STOPPED 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE Bus STARTING 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE Serving on http://:::9283 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:11:17.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: Deploying daemon grafana.a on smithi173 2024-09-17T08:11:17.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[40649]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.640 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.640 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.640 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: Deploying daemon grafana.a on smithi173 2024-09-17T08:11:17.640 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-mon[29565]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:17.677 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:17.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T08:11:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:17.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: Deploying daemon grafana.a on smithi173 2024-09-17T08:11:17.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:17 smithi173 ceph-mon[33187]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:17.989 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:17 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:17] ENGINE Bus STARTED 2024-09-17T08:11:18.372 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-17T08:11:17.220039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":998,"num_read_kb":844,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":605,"ondisk_log_size":605,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":240852,"kb_used_data":8340,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749720364,"statfs":{"total":767960285184,"available":767713652736,"internally_reserved":0,"allocated":8540160,"data_stored":4896610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.876236"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612728+0000","last_change":"2024-09-17T08:10:00.599431+0000","last_active":"2024-09-17T08:11:08.612728+0000","last_peered":"2024-09-17T08:11:08.612728+0000","last_clean":"2024-09-17T08:11:08.612728+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:08.612728+0000","last_undegraded":"2024-09-17T08:11:08.612728+0000","last_fullsized":"2024-09-17T08:11:08.612728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831096+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:08.831096+0000","last_peered":"2024-09-17T08:11:08.831096+0000","last_clean":"2024-09-17T08:11:08.831096+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:08.831096+0000","last_undegraded":"2024-09-17T08:11:08.831096+0000","last_fullsized":"2024-09-17T08:11:08.831096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612270+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:08.612270+0000","last_peered":"2024-09-17T08:11:08.612270+0000","last_clean":"2024-09-17T08:11:08.612270+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:08.612270+0000","last_undegraded":"2024-09-17T08:11:08.612270+0000","last_fullsized":"2024-09-17T08:11:08.612270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.783370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.631088+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:13.631088+0000","last_peered":"2024-09-17T08:11:13.631088+0000","last_clean":"2024-09-17T08:11:13.631088+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:13.631088+0000","last_undegraded":"2024-09-17T08:11:13.631088+0000","last_fullsized":"2024-09-17T08:11:13.631088+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613277+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:08.613277+0000","last_peered":"2024-09-17T08:11:08.613277+0000","last_clean":"2024-09-17T08:11:08.613277+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:08.613277+0000","last_undegraded":"2024-09-17T08:11:08.613277+0000","last_fullsized":"2024-09-17T08:11:08.613277+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.516200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349425+0000","last_change":"2024-09-17T08:09:56.500998+0000","last_active":"2024-09-17T08:11:08.349425+0000","last_peered":"2024-09-17T08:11:08.349425+0000","last_clean":"2024-09-17T08:11:08.349425+0000","last_became_active":"2024-09-17T08:09:56.500692+0000","last_became_peered":"2024-09-17T08:09:56.500692+0000","last_unstale":"2024-09-17T08:11:08.349425+0000","last_undegraded":"2024-09-17T08:11:08.349425+0000","last_fullsized":"2024-09-17T08:11:08.349425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.970578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345159+0000","last_change":"2024-09-17T08:09:58.507646+0000","last_active":"2024-09-17T08:11:08.345159+0000","last_peered":"2024-09-17T08:11:08.345159+0000","last_clean":"2024-09-17T08:11:08.345159+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:08.345159+0000","last_undegraded":"2024-09-17T08:11:08.345159+0000","last_fullsized":"2024-09-17T08:11:08.345159+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349560+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:08.349560+0000","last_peered":"2024-09-17T08:11:08.349560+0000","last_clean":"2024-09-17T08:11:08.349560+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:08.349560+0000","last_undegraded":"2024-09-17T08:11:08.349560+0000","last_fullsized":"2024-09-17T08:11:08.349560+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492990+0000","last_change":"2024-09-17T08:09:54.507191+0000","last_active":"2024-09-17T08:11:08.492990+0000","last_peered":"2024-09-17T08:11:08.492990+0000","last_clean":"2024-09-17T08:11:08.492990+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:08.492990+0000","last_undegraded":"2024-09-17T08:11:08.492990+0000","last_fullsized":"2024-09-17T08:11:08.492990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"72'5","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349539+0000","last_change":"2024-09-17T08:09:56.503460+0000","last_active":"2024-09-17T08:11:08.349539+0000","last_peered":"2024-09-17T08:11:08.349539+0000","last_clean":"2024-09-17T08:11:08.349539+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:08.349539+0000","last_undegraded":"2024-09-17T08:11:08.349539+0000","last_fullsized":"2024-09-17T08:11:08.349539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:19.318133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492095+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:08.492095+0000","last_peered":"2024-09-17T08:11:08.492095+0000","last_clean":"2024-09-17T08:11:08.492095+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:08.492095+0000","last_undegraded":"2024-09-17T08:11:08.492095+0000","last_fullsized":"2024-09-17T08:11:08.492095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831013+0000","last_change":"2024-09-17T08:10:00.517064+0000","last_active":"2024-09-17T08:11:08.831013+0000","last_peered":"2024-09-17T08:11:08.831013+0000","last_clean":"2024-09-17T08:11:08.831013+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:08.831013+0000","last_undegraded":"2024-09-17T08:11:08.831013+0000","last_fullsized":"2024-09-17T08:11:08.831013+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:24.078122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492981+0000","last_change":"2024-09-17T08:09:54.502336+0000","last_active":"2024-09-17T08:11:08.492981+0000","last_peered":"2024-09-17T08:11:08.492981+0000","last_clean":"2024-09-17T08:11:08.492981+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:08.492981+0000","last_undegraded":"2024-09-17T08:11:08.492981+0000","last_fullsized":"2024-09-17T08:11:08.492981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349842+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:08.349842+0000","last_peered":"2024-09-17T08:11:08.349842+0000","last_clean":"2024-09-17T08:11:08.349842+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:08.349842+0000","last_undegraded":"2024-09-17T08:11:08.349842+0000","last_fullsized":"2024-09-17T08:11:08.349842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831129+0000","last_change":"2024-09-17T08:09:58.512901+0000","last_active":"2024-09-17T08:11:08.831129+0000","last_peered":"2024-09-17T08:11:08.831129+0000","last_clean":"2024-09-17T08:11:08.831129+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:08.831129+0000","last_undegraded":"2024-09-17T08:11:08.831129+0000","last_fullsized":"2024-09-17T08:11:08.831129+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830484+0000","last_change":"2024-09-17T08:10:00.520858+0000","last_active":"2024-09-17T08:11:08.830484+0000","last_peered":"2024-09-17T08:11:08.830484+0000","last_clean":"2024-09-17T08:11:08.830484+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:08.830484+0000","last_undegraded":"2024-09-17T08:11:08.830484+0000","last_fullsized":"2024-09-17T08:11:08.830484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350253+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:08.350253+0000","last_peered":"2024-09-17T08:11:08.350253+0000","last_clean":"2024-09-17T08:11:08.350253+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:08.350253+0000","last_undegraded":"2024-09-17T08:11:08.350253+0000","last_fullsized":"2024-09-17T08:11:08.350253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492553+0000","last_change":"2024-09-17T08:09:54.503924+0000","last_active":"2024-09-17T08:11:08.492553+0000","last_peered":"2024-09-17T08:11:08.492553+0000","last_clean":"2024-09-17T08:11:08.492553+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:08.492553+0000","last_undegraded":"2024-09-17T08:11:08.492553+0000","last_fullsized":"2024-09-17T08:11:08.492553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346285+0000","last_change":"2024-09-17T08:09:58.506645+0000","last_active":"2024-09-17T08:11:08.346285+0000","last_peered":"2024-09-17T08:11:08.346285+0000","last_clean":"2024-09-17T08:11:08.346285+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:08.346285+0000","last_undegraded":"2024-09-17T08:11:08.346285+0000","last_fullsized":"2024-09-17T08:11:08.346285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.614662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348162+0000","last_change":"2024-09-17T08:10:00.521440+0000","last_active":"2024-09-17T08:11:08.348162+0000","last_peered":"2024-09-17T08:11:08.348162+0000","last_clean":"2024-09-17T08:11:08.348162+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:08.348162+0000","last_undegraded":"2024-09-17T08:11:08.348162+0000","last_fullsized":"2024-09-17T08:11:08.348162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.963646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349835+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:08.349835+0000","last_peered":"2024-09-17T08:11:08.349835+0000","last_clean":"2024-09-17T08:11:08.349835+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:08.349835+0000","last_undegraded":"2024-09-17T08:11:08.349835+0000","last_fullsized":"2024-09-17T08:11:08.349835+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346248+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:08.346248+0000","last_peered":"2024-09-17T08:11:08.346248+0000","last_clean":"2024-09-17T08:11:08.346248+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:08.346248+0000","last_undegraded":"2024-09-17T08:11:08.346248+0000","last_fullsized":"2024-09-17T08:11:08.346248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352891+0000","last_change":"2024-09-17T08:09:58.512945+0000","last_active":"2024-09-17T08:11:11.352891+0000","last_peered":"2024-09-17T08:11:11.352891+0000","last_clean":"2024-09-17T08:11:11.352891+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:11.352891+0000","last_undegraded":"2024-09-17T08:11:11.352891+0000","last_fullsized":"2024-09-17T08:11:11.352891+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350018+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:08.350018+0000","last_peered":"2024-09-17T08:11:08.350018+0000","last_clean":"2024-09-17T08:11:08.350018+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:08.350018+0000","last_undegraded":"2024-09-17T08:11:08.350018+0000","last_fullsized":"2024-09-17T08:11:08.350018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.558426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.969980+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:16.969980+0000","last_peered":"2024-09-17T08:11:16.969980+0000","last_clean":"2024-09-17T08:11:16.969980+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:16.969980+0000","last_undegraded":"2024-09-17T08:11:16.969980+0000","last_fullsized":"2024-09-17T08:11:16.969980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345811+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:08.345811+0000","last_peered":"2024-09-17T08:11:08.345811+0000","last_clean":"2024-09-17T08:11:08.345811+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:08.345811+0000","last_undegraded":"2024-09-17T08:11:08.345811+0000","last_fullsized":"2024-09-17T08:11:08.345811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.358620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154991+0000","last_change":"2024-09-17T08:09:58.508668+0000","last_active":"2024-09-17T08:11:11.154991+0000","last_peered":"2024-09-17T08:11:11.154991+0000","last_clean":"2024-09-17T08:11:11.154991+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:11.154991+0000","last_undegraded":"2024-09-17T08:11:11.154991+0000","last_fullsized":"2024-09-17T08:11:11.154991+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831269+0000","last_change":"2024-09-17T08:10:00.517958+0000","last_active":"2024-09-17T08:11:08.831269+0000","last_peered":"2024-09-17T08:11:08.831269+0000","last_clean":"2024-09-17T08:11:08.831269+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:08.831269+0000","last_undegraded":"2024-09-17T08:11:08.831269+0000","last_fullsized":"2024-09-17T08:11:08.831269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.257059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350211+0000","last_change":"2024-09-17T08:09:56.500368+0000","last_active":"2024-09-17T08:11:08.350211+0000","last_peered":"2024-09-17T08:11:08.350211+0000","last_clean":"2024-09-17T08:11:08.350211+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:08.350211+0000","last_undegraded":"2024-09-17T08:11:08.350211+0000","last_fullsized":"2024-09-17T08:11:08.350211+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492037+0000","last_change":"2024-09-17T08:09:54.506854+0000","last_active":"2024-09-17T08:11:08.492037+0000","last_peered":"2024-09-17T08:11:08.492037+0000","last_clean":"2024-09-17T08:11:08.492037+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:08.492037+0000","last_undegraded":"2024-09-17T08:11:08.492037+0000","last_fullsized":"2024-09-17T08:11:08.492037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.363057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831578+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:08.831578+0000","last_peered":"2024-09-17T08:11:08.831578+0000","last_clean":"2024-09-17T08:11:08.831578+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:08.831578+0000","last_undegraded":"2024-09-17T08:11:08.831578+0000","last_fullsized":"2024-09-17T08:11:08.831578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612948+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:08.612948+0000","last_peered":"2024-09-17T08:11:08.612948+0000","last_clean":"2024-09-17T08:11:08.612948+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:08.612948+0000","last_undegraded":"2024-09-17T08:11:08.612948+0000","last_fullsized":"2024-09-17T08:11:08.612948+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.578880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349917+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:08.349917+0000","last_peered":"2024-09-17T08:11:08.349917+0000","last_clean":"2024-09-17T08:11:08.349917+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:08.349917+0000","last_undegraded":"2024-09-17T08:11:08.349917+0000","last_fullsized":"2024-09-17T08:11:08.349917+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494544+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:08.494544+0000","last_peered":"2024-09-17T08:11:08.494544+0000","last_clean":"2024-09-17T08:11:08.494544+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:08.494544+0000","last_undegraded":"2024-09-17T08:11:08.494544+0000","last_fullsized":"2024-09-17T08:11:08.494544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352476+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:11.352476+0000","last_peered":"2024-09-17T08:11:11.352476+0000","last_clean":"2024-09-17T08:11:11.352476+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:11.352476+0000","last_undegraded":"2024-09-17T08:11:11.352476+0000","last_fullsized":"2024-09-17T08:11:11.352476+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.469667+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:11.469667+0000","last_peered":"2024-09-17T08:11:11.469667+0000","last_clean":"2024-09-17T08:11:11.469667+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:11.469667+0000","last_undegraded":"2024-09-17T08:11:11.469667+0000","last_fullsized":"2024-09-17T08:11:11.469667+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612611+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:08.612611+0000","last_peered":"2024-09-17T08:11:08.612611+0000","last_clean":"2024-09-17T08:11:08.612611+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:08.612611+0000","last_undegraded":"2024-09-17T08:11:08.612611+0000","last_fullsized":"2024-09-17T08:11:08.612611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346189+0000","last_change":"2024-09-17T08:09:54.498648+0000","last_active":"2024-09-17T08:11:08.346189+0000","last_peered":"2024-09-17T08:11:08.346189+0000","last_clean":"2024-09-17T08:11:08.346189+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:08.346189+0000","last_undegraded":"2024-09-17T08:11:08.346189+0000","last_fullsized":"2024-09-17T08:11:08.346189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:19.423047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349590+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:08.349590+0000","last_peered":"2024-09-17T08:11:08.349590+0000","last_clean":"2024-09-17T08:11:08.349590+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:08.349590+0000","last_undegraded":"2024-09-17T08:11:08.349590+0000","last_fullsized":"2024-09-17T08:11:08.349590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.957574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349711+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:08.349711+0000","last_peered":"2024-09-17T08:11:08.349711+0000","last_clean":"2024-09-17T08:11:08.349711+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:08.349711+0000","last_undegraded":"2024-09-17T08:11:08.349711+0000","last_fullsized":"2024-09-17T08:11:08.349711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.596565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830839+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:08.830839+0000","last_peered":"2024-09-17T08:11:08.830839+0000","last_clean":"2024-09-17T08:11:08.830839+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:08.830839+0000","last_undegraded":"2024-09-17T08:11:08.830839+0000","last_fullsized":"2024-09-17T08:11:08.830839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.493061+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:08.493061+0000","last_peered":"2024-09-17T08:11:08.493061+0000","last_clean":"2024-09-17T08:11:08.493061+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:08.493061+0000","last_undegraded":"2024-09-17T08:11:08.493061+0000","last_fullsized":"2024-09-17T08:11:08.493061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494046+0000","last_change":"2024-09-17T08:09:58.506025+0000","last_active":"2024-09-17T08:11:08.494046+0000","last_peered":"2024-09-17T08:11:08.494046+0000","last_clean":"2024-09-17T08:11:08.494046+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:08.494046+0000","last_undegraded":"2024-09-17T08:11:08.494046+0000","last_fullsized":"2024-09-17T08:11:08.494046+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.291171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.026108+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:11.026108+0000","last_peered":"2024-09-17T08:11:11.026108+0000","last_clean":"2024-09-17T08:11:11.026108+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:11.026108+0000","last_undegraded":"2024-09-17T08:11:11.026108+0000","last_fullsized":"2024-09-17T08:11:11.026108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345562+0000","last_change":"2024-09-17T08:09:56.502524+0000","last_active":"2024-09-17T08:11:08.345562+0000","last_peered":"2024-09-17T08:11:08.345562+0000","last_clean":"2024-09-17T08:11:08.345562+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:08.345562+0000","last_undegraded":"2024-09-17T08:11:08.345562+0000","last_fullsized":"2024-09-17T08:11:08.345562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.354486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346618+0000","last_change":"2024-09-17T08:09:54.498729+0000","last_active":"2024-09-17T08:11:08.346618+0000","last_peered":"2024-09-17T08:11:08.346618+0000","last_clean":"2024-09-17T08:11:08.346618+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:08.346618+0000","last_undegraded":"2024-09-17T08:11:08.346618+0000","last_fullsized":"2024-09-17T08:11:08.346618+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.547950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155283+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:11.155283+0000","last_peered":"2024-09-17T08:11:11.155283+0000","last_clean":"2024-09-17T08:11:11.155283+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:11.155283+0000","last_undegraded":"2024-09-17T08:11:11.155283+0000","last_fullsized":"2024-09-17T08:11:11.155283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830401+0000","last_change":"2024-09-17T08:10:00.516499+0000","last_active":"2024-09-17T08:11:08.830401+0000","last_peered":"2024-09-17T08:11:08.830401+0000","last_clean":"2024-09-17T08:11:08.830401+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:08.830401+0000","last_undegraded":"2024-09-17T08:11:08.830401+0000","last_fullsized":"2024-09-17T08:11:08.830401+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.391480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154959+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:11.154959+0000","last_peered":"2024-09-17T08:11:11.154959+0000","last_clean":"2024-09-17T08:11:11.154959+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:11.154959+0000","last_undegraded":"2024-09-17T08:11:11.154959+0000","last_fullsized":"2024-09-17T08:11:11.154959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831678+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:08.831678+0000","last_peered":"2024-09-17T08:11:08.831678+0000","last_clean":"2024-09-17T08:11:08.831678+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:08.831678+0000","last_undegraded":"2024-09-17T08:11:08.831678+0000","last_fullsized":"2024-09-17T08:11:08.831678+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830963+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:08.830963+0000","last_peered":"2024-09-17T08:11:08.830963+0000","last_clean":"2024-09-17T08:11:08.830963+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:08.830963+0000","last_undegraded":"2024-09-17T08:11:08.830963+0000","last_fullsized":"2024-09-17T08:11:08.830963+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:10.030674+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:10.030674+0000","last_peered":"2024-09-17T08:11:10.030674+0000","last_clean":"2024-09-17T08:11:10.030674+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:10.030674+0000","last_undegraded":"2024-09-17T08:11:10.030674+0000","last_fullsized":"2024-09-17T08:11:10.030674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346624+0000","last_change":"2024-09-17T08:09:56.503791+0000","last_active":"2024-09-17T08:11:08.346624+0000","last_peered":"2024-09-17T08:11:08.346624+0000","last_clean":"2024-09-17T08:11:08.346624+0000","last_became_active":"2024-09-17T08:09:56.503476+0000","last_became_peered":"2024-09-17T08:09:56.503476+0000","last_unstale":"2024-09-17T08:11:08.346624+0000","last_undegraded":"2024-09-17T08:11:08.346624+0000","last_fullsized":"2024-09-17T08:11:08.346624+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.579571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":124,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.158068+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:16.158068+0000","last_peered":"2024-09-17T08:11:16.158068+0000","last_clean":"2024-09-17T08:11:16.158068+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:16.158068+0000","last_undegraded":"2024-09-17T08:11:16.158068+0000","last_fullsized":"2024-09-17T08:11:16.158068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347634+0000","last_change":"2024-09-17T08:09:54.504840+0000","last_active":"2024-09-17T08:11:08.347634+0000","last_peered":"2024-09-17T08:11:08.347634+0000","last_clean":"2024-09-17T08:11:08.347634+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:08.347634+0000","last_undegraded":"2024-09-17T08:11:08.347634+0000","last_fullsized":"2024-09-17T08:11:08.347634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348090+0000","last_change":"2024-09-17T08:09:58.514115+0000","last_active":"2024-09-17T08:11:08.348090+0000","last_peered":"2024-09-17T08:11:08.348090+0000","last_clean":"2024-09-17T08:11:08.348090+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:08.348090+0000","last_undegraded":"2024-09-17T08:11:08.348090+0000","last_fullsized":"2024-09-17T08:11:08.348090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492725+0000","last_change":"2024-09-17T08:10:00.599802+0000","last_active":"2024-09-17T08:11:08.492725+0000","last_peered":"2024-09-17T08:11:08.492725+0000","last_clean":"2024-09-17T08:11:08.492725+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:08.492725+0000","last_undegraded":"2024-09-17T08:11:08.492725+0000","last_fullsized":"2024-09-17T08:11:08.492725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.895697+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:11.895697+0000","last_peered":"2024-09-17T08:11:11.895697+0000","last_clean":"2024-09-17T08:11:11.895697+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:11.895697+0000","last_undegraded":"2024-09-17T08:11:11.895697+0000","last_fullsized":"2024-09-17T08:11:11.895697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830657+0000","last_change":"2024-09-17T08:09:58.498654+0000","last_active":"2024-09-17T08:11:08.830657+0000","last_peered":"2024-09-17T08:11:08.830657+0000","last_clean":"2024-09-17T08:11:08.830657+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:08.830657+0000","last_undegraded":"2024-09-17T08:11:08.830657+0000","last_fullsized":"2024-09-17T08:11:08.830657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.615615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492108+0000","last_change":"2024-09-17T08:09:54.507600+0000","last_active":"2024-09-17T08:11:08.492108+0000","last_peered":"2024-09-17T08:11:08.492108+0000","last_clean":"2024-09-17T08:11:08.492108+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:08.492108+0000","last_undegraded":"2024-09-17T08:11:08.492108+0000","last_fullsized":"2024-09-17T08:11:08.492108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.885315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831705+0000","last_change":"2024-09-17T08:09:58.502361+0000","last_active":"2024-09-17T08:11:08.831705+0000","last_peered":"2024-09-17T08:11:08.831705+0000","last_clean":"2024-09-17T08:11:08.831705+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:08.831705+0000","last_undegraded":"2024-09-17T08:11:08.831705+0000","last_fullsized":"2024-09-17T08:11:08.831705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.530587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346193+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:08.346193+0000","last_peered":"2024-09-17T08:11:08.346193+0000","last_clean":"2024-09-17T08:11:08.346193+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:08.346193+0000","last_undegraded":"2024-09-17T08:11:08.346193+0000","last_fullsized":"2024-09-17T08:11:08.346193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.624827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.289684+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:11.289684+0000","last_peered":"2024-09-17T08:11:11.289684+0000","last_clean":"2024-09-17T08:11:11.289684+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:11.289684+0000","last_undegraded":"2024-09-17T08:11:11.289684+0000","last_fullsized":"2024-09-17T08:11:11.289684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612823+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:08.612823+0000","last_peered":"2024-09-17T08:11:08.612823+0000","last_clean":"2024-09-17T08:11:08.612823+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:08.612823+0000","last_undegraded":"2024-09-17T08:11:08.612823+0000","last_fullsized":"2024-09-17T08:11:08.612823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.254805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831699+0000","last_change":"2024-09-17T08:09:54.502513+0000","last_active":"2024-09-17T08:11:08.831699+0000","last_peered":"2024-09-17T08:11:08.831699+0000","last_clean":"2024-09-17T08:11:08.831699+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:08.831699+0000","last_undegraded":"2024-09-17T08:11:08.831699+0000","last_fullsized":"2024-09-17T08:11:08.831699+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:12.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":290,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492403+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:08.492403+0000","last_peered":"2024-09-17T08:11:08.492403+0000","last_clean":"2024-09-17T08:11:08.492403+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:08.492403+0000","last_undegraded":"2024-09-17T08:11:08.492403+0000","last_fullsized":"2024-09-17T08:11:08.492403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.066267+0000","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492678+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:08.492678+0000","last_peered":"2024-09-17T08:11:08.492678+0000","last_clean":"2024-09-17T08:11:08.492678+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:08.492678+0000","last_undegraded":"2024-09-17T08:11:08.492678+0000","last_fullsized":"2024-09-17T08:11:08.492678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.857213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831709+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:08.831709+0000","last_peered":"2024-09-17T08:11:08.831709+0000","last_clean":"2024-09-17T08:11:08.831709+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:08.831709+0000","last_undegraded":"2024-09-17T08:11:08.831709+0000","last_fullsized":"2024-09-17T08:11:08.831709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494397+0000","last_change":"2024-09-17T08:09:57.041231+0000","last_active":"2024-09-17T08:11:08.494397+0000","last_peered":"2024-09-17T08:11:08.494397+0000","last_clean":"2024-09-17T08:11:08.494397+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:08.494397+0000","last_undegraded":"2024-09-17T08:11:08.494397+0000","last_fullsized":"2024-09-17T08:11:08.494397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.855672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348259+0000","last_change":"2024-09-17T08:09:54.502363+0000","last_active":"2024-09-17T08:11:08.348259+0000","last_peered":"2024-09-17T08:11:08.348259+0000","last_clean":"2024-09-17T08:11:08.348259+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:08.348259+0000","last_undegraded":"2024-09-17T08:11:08.348259+0000","last_fullsized":"2024-09-17T08:11:08.348259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.973864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.148121+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:15.148121+0000","last_peered":"2024-09-17T08:11:15.148121+0000","last_clean":"2024-09-17T08:11:15.148121+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:15.148121+0000","last_undegraded":"2024-09-17T08:11:15.148121+0000","last_fullsized":"2024-09-17T08:11:15.148121+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492248+0000","last_change":"2024-09-17T08:10:00.525443+0000","last_active":"2024-09-17T08:11:08.492248+0000","last_peered":"2024-09-17T08:11:08.492248+0000","last_clean":"2024-09-17T08:11:08.492248+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:08.492248+0000","last_undegraded":"2024-09-17T08:11:08.492248+0000","last_fullsized":"2024-09-17T08:11:08.492248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830294+0000","last_change":"2024-09-17T08:09:56.506519+0000","last_active":"2024-09-17T08:11:08.830294+0000","last_peered":"2024-09-17T08:11:08.830294+0000","last_clean":"2024-09-17T08:11:08.830294+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:08.830294+0000","last_undegraded":"2024-09-17T08:11:08.830294+0000","last_fullsized":"2024-09-17T08:11:08.830294+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.648151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347044+0000","last_change":"2024-09-17T08:09:54.499164+0000","last_active":"2024-09-17T08:11:08.347044+0000","last_peered":"2024-09-17T08:11:08.347044+0000","last_clean":"2024-09-17T08:11:08.347044+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:08.347044+0000","last_undegraded":"2024-09-17T08:11:08.347044+0000","last_fullsized":"2024-09-17T08:11:08.347044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.690089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347696+0000","last_change":"2024-09-17T08:09:58.511946+0000","last_active":"2024-09-17T08:11:08.347696+0000","last_peered":"2024-09-17T08:11:08.347696+0000","last_clean":"2024-09-17T08:11:08.347696+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:08.347696+0000","last_undegraded":"2024-09-17T08:11:08.347696+0000","last_fullsized":"2024-09-17T08:11:08.347696+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830617+0000","last_change":"2024-09-17T08:10:00.521382+0000","last_active":"2024-09-17T08:11:08.830617+0000","last_peered":"2024-09-17T08:11:08.830617+0000","last_clean":"2024-09-17T08:11:08.830617+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:08.830617+0000","last_undegraded":"2024-09-17T08:11:08.830617+0000","last_fullsized":"2024-09-17T08:11:08.830617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:12.458130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830961+0000","last_change":"2024-09-17T08:09:57.041079+0000","last_active":"2024-09-17T08:11:08.830961+0000","last_peered":"2024-09-17T08:11:08.830961+0000","last_clean":"2024-09-17T08:11:08.830961+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:08.830961+0000","last_undegraded":"2024-09-17T08:11:08.830961+0000","last_fullsized":"2024-09-17T08:11:08.830961+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.635691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348343+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:08.348343+0000","last_peered":"2024-09-17T08:11:08.348343+0000","last_clean":"2024-09-17T08:11:08.348343+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:08.348343+0000","last_undegraded":"2024-09-17T08:11:08.348343+0000","last_fullsized":"2024-09-17T08:11:08.348343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290733+0000","last_change":"2024-09-17T08:09:58.513726+0000","last_active":"2024-09-17T08:11:11.290733+0000","last_peered":"2024-09-17T08:11:11.290733+0000","last_clean":"2024-09-17T08:11:11.290733+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:11.290733+0000","last_undegraded":"2024-09-17T08:11:11.290733+0000","last_fullsized":"2024-09-17T08:11:11.290733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.608466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831127+0000","last_change":"2024-09-17T08:10:00.599030+0000","last_active":"2024-09-17T08:11:08.831127+0000","last_peered":"2024-09-17T08:11:08.831127+0000","last_clean":"2024-09-17T08:11:08.831127+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:08.831127+0000","last_undegraded":"2024-09-17T08:11:08.831127+0000","last_fullsized":"2024-09-17T08:11:08.831127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:26.441216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.614679+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:08.614679+0000","last_peered":"2024-09-17T08:11:08.614679+0000","last_clean":"2024-09-17T08:11:08.614679+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:08.614679+0000","last_undegraded":"2024-09-17T08:11:08.614679+0000","last_fullsized":"2024-09-17T08:11:08.614679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:10.663912+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:10.663912+0000","last_peered":"2024-09-17T08:11:10.663912+0000","last_clean":"2024-09-17T08:11:10.663912+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:10.663912+0000","last_undegraded":"2024-09-17T08:11:10.663912+0000","last_fullsized":"2024-09-17T08:11:10.663912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347666+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:08.347666+0000","last_peered":"2024-09-17T08:11:08.347666+0000","last_clean":"2024-09-17T08:11:08.347666+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:08.347666+0000","last_undegraded":"2024-09-17T08:11:08.347666+0000","last_fullsized":"2024-09-17T08:11:08.347666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612904+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:08.612904+0000","last_peered":"2024-09-17T08:11:08.612904+0000","last_clean":"2024-09-17T08:11:08.612904+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:08.612904+0000","last_undegraded":"2024-09-17T08:11:08.612904+0000","last_fullsized":"2024-09-17T08:11:08.612904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831110+0000","last_change":"2024-09-17T08:09:57.041185+0000","last_active":"2024-09-17T08:11:08.831110+0000","last_peered":"2024-09-17T08:11:08.831110+0000","last_clean":"2024-09-17T08:11:08.831110+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:08.831110+0000","last_undegraded":"2024-09-17T08:11:08.831110+0000","last_fullsized":"2024-09-17T08:11:08.831110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347750+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:08.347750+0000","last_peered":"2024-09-17T08:11:08.347750+0000","last_clean":"2024-09-17T08:11:08.347750+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:08.347750+0000","last_undegraded":"2024-09-17T08:11:08.347750+0000","last_fullsized":"2024-09-17T08:11:08.347750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613233+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:08.613233+0000","last_peered":"2024-09-17T08:11:08.613233+0000","last_clean":"2024-09-17T08:11:08.613233+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:08.613233+0000","last_undegraded":"2024-09-17T08:11:08.613233+0000","last_fullsized":"2024-09-17T08:11:08.613233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.110620+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:14.110620+0000","last_peered":"2024-09-17T08:11:14.110620+0000","last_clean":"2024-09-17T08:11:14.110620+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:14.110620+0000","last_undegraded":"2024-09-17T08:11:14.110620+0000","last_fullsized":"2024-09-17T08:11:14.110620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494215+0000","last_change":"2024-09-17T08:09:54.498318+0000","last_active":"2024-09-17T08:11:08.494215+0000","last_peered":"2024-09-17T08:11:08.494215+0000","last_clean":"2024-09-17T08:11:08.494215+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:08.494215+0000","last_undegraded":"2024-09-17T08:11:08.494215+0000","last_fullsized":"2024-09-17T08:11:08.494215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.145703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613253+0000","last_change":"2024-09-17T08:09:57.043364+0000","last_active":"2024-09-17T08:11:08.613253+0000","last_peered":"2024-09-17T08:11:08.613253+0000","last_clean":"2024-09-17T08:11:08.613253+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:08.613253+0000","last_undegraded":"2024-09-17T08:11:08.613253+0000","last_fullsized":"2024-09-17T08:11:08.613253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.183678+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:12.183678+0000","last_peered":"2024-09-17T08:11:12.183678+0000","last_clean":"2024-09-17T08:11:12.183678+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:12.183678+0000","last_undegraded":"2024-09-17T08:11:12.183678+0000","last_fullsized":"2024-09-17T08:11:12.183678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492728+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:08.492728+0000","last_peered":"2024-09-17T08:11:08.492728+0000","last_clean":"2024-09-17T08:11:08.492728+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:08.492728+0000","last_undegraded":"2024-09-17T08:11:08.492728+0000","last_fullsized":"2024-09-17T08:11:08.492728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613065+0000","last_change":"2024-09-17T08:09:57.041188+0000","last_active":"2024-09-17T08:11:08.613065+0000","last_peered":"2024-09-17T08:11:08.613065+0000","last_clean":"2024-09-17T08:11:08.613065+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:08.613065+0000","last_undegraded":"2024-09-17T08:11:08.613065+0000","last_fullsized":"2024-09-17T08:11:08.613065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831504+0000","last_change":"2024-09-17T08:10:54.331930+0000","last_active":"2024-09-17T08:11:08.831504+0000","last_peered":"2024-09-17T08:11:08.831504+0000","last_clean":"2024-09-17T08:11:08.831504+0000","last_became_active":"2024-09-17T08:09:54.501179+0000","last_became_peered":"2024-09-17T08:09:54.501179+0000","last_unstale":"2024-09-17T08:11:08.831504+0000","last_undegraded":"2024-09-17T08:11:08.831504+0000","last_fullsized":"2024-09-17T08:11:08.831504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.084651+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.217500+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:11.217500+0000","last_peered":"2024-09-17T08:11:11.217500+0000","last_clean":"2024-09-17T08:11:11.217500+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:11.217500+0000","last_undegraded":"2024-09-17T08:11:11.217500+0000","last_fullsized":"2024-09-17T08:11:11.217500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830917+0000","last_change":"2024-09-17T08:10:00.521904+0000","last_active":"2024-09-17T08:11:08.830917+0000","last_peered":"2024-09-17T08:11:08.830917+0000","last_clean":"2024-09-17T08:11:08.830917+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:08.830917+0000","last_undegraded":"2024-09-17T08:11:08.830917+0000","last_fullsized":"2024-09-17T08:11:08.830917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.331400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.214203+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:13.214203+0000","last_peered":"2024-09-17T08:11:13.214203+0000","last_clean":"2024-09-17T08:11:13.214203+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:13.214203+0000","last_undegraded":"2024-09-17T08:11:13.214203+0000","last_fullsized":"2024-09-17T08:11:13.214203+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831082+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:08.831082+0000","last_peered":"2024-09-17T08:11:08.831082+0000","last_clean":"2024-09-17T08:11:08.831082+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:08.831082+0000","last_undegraded":"2024-09-17T08:11:08.831082+0000","last_fullsized":"2024-09-17T08:11:08.831082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352587+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:11.352587+0000","last_peered":"2024-09-17T08:11:11.352587+0000","last_clean":"2024-09-17T08:11:11.352587+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:11.352587+0000","last_undegraded":"2024-09-17T08:11:11.352587+0000","last_fullsized":"2024-09-17T08:11:11.352587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349952+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:08.349952+0000","last_peered":"2024-09-17T08:11:08.349952+0000","last_clean":"2024-09-17T08:11:08.349952+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:08.349952+0000","last_undegraded":"2024-09-17T08:11:08.349952+0000","last_fullsized":"2024-09-17T08:11:08.349952+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.032099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345472+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:08.345472+0000","last_peered":"2024-09-17T08:11:08.345472+0000","last_clean":"2024-09-17T08:11:08.345472+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:08.345472+0000","last_undegraded":"2024-09-17T08:11:08.345472+0000","last_fullsized":"2024-09-17T08:11:08.345472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.317178+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:12.317178+0000","last_peered":"2024-09-17T08:11:12.317178+0000","last_clean":"2024-09-17T08:11:12.317178+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:12.317178+0000","last_undegraded":"2024-09-17T08:11:12.317178+0000","last_fullsized":"2024-09-17T08:11:12.317178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612951+0000","last_change":"2024-09-17T08:09:58.508365+0000","last_active":"2024-09-17T08:11:08.612951+0000","last_peered":"2024-09-17T08:11:08.612951+0000","last_clean":"2024-09-17T08:11:08.612951+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:08.612951+0000","last_undegraded":"2024-09-17T08:11:08.612951+0000","last_fullsized":"2024-09-17T08:11:08.612951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830383+0000","last_change":"2024-09-17T08:10:00.518008+0000","last_active":"2024-09-17T08:11:08.830383+0000","last_peered":"2024-09-17T08:11:08.830383+0000","last_clean":"2024-09-17T08:11:08.830383+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:08.830383+0000","last_undegraded":"2024-09-17T08:11:08.830383+0000","last_fullsized":"2024-09-17T08:11:08.830383+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349650+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:08.349650+0000","last_peered":"2024-09-17T08:11:08.349650+0000","last_clean":"2024-09-17T08:11:08.349650+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:08.349650+0000","last_undegraded":"2024-09-17T08:11:08.349650+0000","last_fullsized":"2024-09-17T08:11:08.349650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346972+0000","last_change":"2024-09-17T08:09:54.496656+0000","last_active":"2024-09-17T08:11:08.346972+0000","last_peered":"2024-09-17T08:11:08.346972+0000","last_clean":"2024-09-17T08:11:08.346972+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:08.346972+0000","last_undegraded":"2024-09-17T08:11:08.346972+0000","last_fullsized":"2024-09-17T08:11:08.346972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.344618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.013139+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:12.013139+0000","last_peered":"2024-09-17T08:11:12.013139+0000","last_clean":"2024-09-17T08:11:12.013139+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:12.013139+0000","last_undegraded":"2024-09-17T08:11:12.013139+0000","last_fullsized":"2024-09-17T08:11:12.013139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.086747+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:14.086747+0000","last_peered":"2024-09-17T08:11:14.086747+0000","last_clean":"2024-09-17T08:11:14.086747+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:14.086747+0000","last_undegraded":"2024-09-17T08:11:14.086747+0000","last_fullsized":"2024-09-17T08:11:14.086747+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612255+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:08.612255+0000","last_peered":"2024-09-17T08:11:08.612255+0000","last_clean":"2024-09-17T08:11:08.612255+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:08.612255+0000","last_undegraded":"2024-09-17T08:11:08.612255+0000","last_fullsized":"2024-09-17T08:11:08.612255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831731+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:08.831731+0000","last_peered":"2024-09-17T08:11:08.831731+0000","last_clean":"2024-09-17T08:11:08.831731+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:08.831731+0000","last_undegraded":"2024-09-17T08:11:08.831731+0000","last_fullsized":"2024-09-17T08:11:08.831731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.493947+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:08.493947+0000","last_peered":"2024-09-17T08:11:08.493947+0000","last_clean":"2024-09-17T08:11:08.493947+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:08.493947+0000","last_undegraded":"2024-09-17T08:11:08.493947+0000","last_fullsized":"2024-09-17T08:11:08.493947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492033+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:08.492033+0000","last_peered":"2024-09-17T08:11:08.492033+0000","last_clean":"2024-09-17T08:11:08.492033+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:08.492033+0000","last_undegraded":"2024-09-17T08:11:08.492033+0000","last_fullsized":"2024-09-17T08:11:08.492033+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.985197+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:08.985197+0000","last_peered":"2024-09-17T08:11:08.985197+0000","last_clean":"2024-09-17T08:11:08.985197+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:08.985197+0000","last_undegraded":"2024-09-17T08:11:08.985197+0000","last_fullsized":"2024-09-17T08:11:08.985197+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"65'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494494+0000","last_change":"2024-09-17T08:09:54.506095+0000","last_active":"2024-09-17T08:11:08.494494+0000","last_peered":"2024-09-17T08:11:08.494494+0000","last_clean":"2024-09-17T08:11:08.494494+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:08.494494+0000","last_undegraded":"2024-09-17T08:11:08.494494+0000","last_fullsized":"2024-09-17T08:11:08.494494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.529326+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:13.529326+0000","last_peered":"2024-09-17T08:11:13.529326+0000","last_clean":"2024-09-17T08:11:13.529326+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:13.529326+0000","last_undegraded":"2024-09-17T08:11:13.529326+0000","last_fullsized":"2024-09-17T08:11:13.529326+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612827+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:08.612827+0000","last_peered":"2024-09-17T08:11:08.612827+0000","last_clean":"2024-09-17T08:11:08.612827+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:08.612827+0000","last_undegraded":"2024-09-17T08:11:08.612827+0000","last_fullsized":"2024-09-17T08:11:08.612827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492515+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:08.492515+0000","last_peered":"2024-09-17T08:11:08.492515+0000","last_clean":"2024-09-17T08:11:08.492515+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:08.492515+0000","last_undegraded":"2024-09-17T08:11:08.492515+0000","last_fullsized":"2024-09-17T08:11:08.492515+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.700248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.188046+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:16.188046+0000","last_peered":"2024-09-17T08:11:16.188046+0000","last_clean":"2024-09-17T08:11:16.188046+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:16.188046+0000","last_undegraded":"2024-09-17T08:11:16.188046+0000","last_fullsized":"2024-09-17T08:11:16.188046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831019+0000","last_change":"2024-09-17T08:09:57.042542+0000","last_active":"2024-09-17T08:11:08.831019+0000","last_peered":"2024-09-17T08:11:08.831019+0000","last_clean":"2024-09-17T08:11:08.831019+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:08.831019+0000","last_undegraded":"2024-09-17T08:11:08.831019+0000","last_fullsized":"2024-09-17T08:11:08.831019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.333853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288857+0000","last_change":"2024-09-17T08:09:58.515290+0000","last_active":"2024-09-17T08:11:11.288857+0000","last_peered":"2024-09-17T08:11:11.288857+0000","last_clean":"2024-09-17T08:11:11.288857+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:11.288857+0000","last_undegraded":"2024-09-17T08:11:11.288857+0000","last_fullsized":"2024-09-17T08:11:11.288857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346357+0000","last_change":"2024-09-17T08:10:00.523915+0000","last_active":"2024-09-17T08:11:08.346357+0000","last_peered":"2024-09-17T08:11:08.346357+0000","last_clean":"2024-09-17T08:11:08.346357+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:08.346357+0000","last_undegraded":"2024-09-17T08:11:08.346357+0000","last_fullsized":"2024-09-17T08:11:08.346357+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.652238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612922+0000","last_change":"2024-09-17T08:09:54.504854+0000","last_active":"2024-09-17T08:11:08.612922+0000","last_peered":"2024-09-17T08:11:08.612922+0000","last_clean":"2024-09-17T08:11:08.612922+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:08.612922+0000","last_undegraded":"2024-09-17T08:11:08.612922+0000","last_fullsized":"2024-09-17T08:11:08.612922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.278783+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:14.278783+0000","last_peered":"2024-09-17T08:11:14.278783+0000","last_clean":"2024-09-17T08:11:14.278783+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:14.278783+0000","last_undegraded":"2024-09-17T08:11:14.278783+0000","last_fullsized":"2024-09-17T08:11:14.278783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831000+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:08.831000+0000","last_peered":"2024-09-17T08:11:08.831000+0000","last_clean":"2024-09-17T08:11:08.831000+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:08.831000+0000","last_undegraded":"2024-09-17T08:11:08.831000+0000","last_fullsized":"2024-09-17T08:11:08.831000+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350037+0000","last_change":"2024-09-17T08:10:00.598000+0000","last_active":"2024-09-17T08:11:08.350037+0000","last_peered":"2024-09-17T08:11:08.350037+0000","last_clean":"2024-09-17T08:11:08.350037+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:08.350037+0000","last_undegraded":"2024-09-17T08:11:08.350037+0000","last_fullsized":"2024-09-17T08:11:08.350037+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.707509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.309985+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:13.309985+0000","last_peered":"2024-09-17T08:11:13.309985+0000","last_clean":"2024-09-17T08:11:13.309985+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:13.309985+0000","last_undegraded":"2024-09-17T08:11:13.309985+0000","last_fullsized":"2024-09-17T08:11:13.309985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348067+0000","last_change":"2024-09-17T08:09:56.504473+0000","last_active":"2024-09-17T08:11:08.348067+0000","last_peered":"2024-09-17T08:11:08.348067+0000","last_clean":"2024-09-17T08:11:08.348067+0000","last_became_active":"2024-09-17T08:09:56.504114+0000","last_became_peered":"2024-09-17T08:09:56.504114+0000","last_unstale":"2024-09-17T08:11:08.348067+0000","last_undegraded":"2024-09-17T08:11:08.348067+0000","last_fullsized":"2024-09-17T08:11:08.348067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.408053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346807+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:08.346807+0000","last_peered":"2024-09-17T08:11:08.346807+0000","last_clean":"2024-09-17T08:11:08.346807+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:08.346807+0000","last_undegraded":"2024-09-17T08:11:08.346807+0000","last_fullsized":"2024-09-17T08:11:08.346807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.494666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613179+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:08.613179+0000","last_peered":"2024-09-17T08:11:08.613179+0000","last_clean":"2024-09-17T08:11:08.613179+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:08.613179+0000","last_undegraded":"2024-09-17T08:11:08.613179+0000","last_fullsized":"2024-09-17T08:11:08.613179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.080158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613025+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:08.613025+0000","last_peered":"2024-09-17T08:11:08.613025+0000","last_clean":"2024-09-17T08:11:08.613025+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:08.613025+0000","last_undegraded":"2024-09-17T08:11:08.613025+0000","last_fullsized":"2024-09-17T08:11:08.613025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612444+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:08.612444+0000","last_peered":"2024-09-17T08:11:08.612444+0000","last_clean":"2024-09-17T08:11:08.612444+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:08.612444+0000","last_undegraded":"2024-09-17T08:11:08.612444+0000","last_fullsized":"2024-09-17T08:11:08.612444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.637634+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:15.637634+0000","last_peered":"2024-09-17T08:11:15.637634+0000","last_clean":"2024-09-17T08:11:15.637634+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:15.637634+0000","last_undegraded":"2024-09-17T08:11:15.637634+0000","last_fullsized":"2024-09-17T08:11:15.637634+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":397,"ondisk_log_size":397,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070486,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32516,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712636,"statfs":{"total":95995035648,"available":95961739264,"internally_reserved":0,"allocated":1437696,"data_stored":981274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.439}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53700000000000003}]}]},{"osd":6,"up_from":50,"seq":214748364827,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32504,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712648,"statfs":{"total":95995035648,"available":95961751552,"internally_reserved":0,"allocated":1425408,"data_stored":980865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.376}]},{"osd":3,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.623}]},{"osd":5,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68400000000000005}]}]},{"osd":5,"up_from":41,"seq":176093659168,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31920,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713232,"statfs":{"total":95995035648,"available":95962349568,"internally_reserved":0,"allocated":827392,"data_stored":389842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.41499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.44},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.42299999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:10:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46200000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":391075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.504}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.82299999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.81299999999999994}]}]},{"osd":3,"up_from":27,"seq":115964117033,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":390552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.308},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34300000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.439}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":390624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.218},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.23200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.26900000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.441},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46000000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.503},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":391362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.30099999999999999},"min":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.24199999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.311}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85099999999999998,"5min":1.329,"15min":1.329},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.622}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1433600,"data_stored":981016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33700000000000002,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.21199999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:10:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.753,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.504}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39900000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.437}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:18.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:18.579 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:18.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 5.d deep-scrub starts 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[40649]: 5.d deep-scrub ok 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:11:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:11:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 5.d deep-scrub starts 2024-09-17T08:11:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-mon[29565]: 5.d deep-scrub ok 2024-09-17T08:11:18.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:11:18.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:11:18.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:11:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:11:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 5.d deep-scrub starts 2024-09-17T08:11:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:18 smithi173 ceph-mon[33187]: 5.d deep-scrub ok 2024-09-17T08:11:18.951 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:18 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[76373]: ts=2024-09-17T08:11:18.818Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001046535s 2024-09-17T08:11:19.672 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:11:19.672 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:11:19.672 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:11:19.672 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[40649]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:19 smithi079 ceph-mon[29565]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:11:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:11:19.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:19 smithi173 ceph-mon[33187]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:20.441 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:20.443 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:20.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:11:20.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:11:20.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:11:20.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.17 deep-scrub starts 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[40649]: 2.17 deep-scrub ok 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:11:20.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.17 deep-scrub starts 2024-09-17T08:11:20.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:20 smithi079 ceph-mon[29565]: 2.17 deep-scrub ok 2024-09-17T08:11:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:11:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.17 deep-scrub starts 2024-09-17T08:11:20.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:20 smithi173 ceph-mon[33187]: 2.17 deep-scrub ok 2024-09-17T08:11:21.197 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-17T08:11:19.220704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":998,"num_read_kb":844,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":605,"ondisk_log_size":605,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":240852,"kb_used_data":8340,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749720364,"statfs":{"total":767960285184,"available":767713652736,"internally_reserved":0,"allocated":8540160,"data_stored":4896610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002360"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612728+0000","last_change":"2024-09-17T08:10:00.599431+0000","last_active":"2024-09-17T08:11:08.612728+0000","last_peered":"2024-09-17T08:11:08.612728+0000","last_clean":"2024-09-17T08:11:08.612728+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:08.612728+0000","last_undegraded":"2024-09-17T08:11:08.612728+0000","last_fullsized":"2024-09-17T08:11:08.612728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831096+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:08.831096+0000","last_peered":"2024-09-17T08:11:08.831096+0000","last_clean":"2024-09-17T08:11:08.831096+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:08.831096+0000","last_undegraded":"2024-09-17T08:11:08.831096+0000","last_fullsized":"2024-09-17T08:11:08.831096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612270+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:08.612270+0000","last_peered":"2024-09-17T08:11:08.612270+0000","last_clean":"2024-09-17T08:11:08.612270+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:08.612270+0000","last_undegraded":"2024-09-17T08:11:08.612270+0000","last_fullsized":"2024-09-17T08:11:08.612270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.783370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.631088+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:13.631088+0000","last_peered":"2024-09-17T08:11:13.631088+0000","last_clean":"2024-09-17T08:11:13.631088+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:13.631088+0000","last_undegraded":"2024-09-17T08:11:13.631088+0000","last_fullsized":"2024-09-17T08:11:13.631088+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613277+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:08.613277+0000","last_peered":"2024-09-17T08:11:08.613277+0000","last_clean":"2024-09-17T08:11:08.613277+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:08.613277+0000","last_undegraded":"2024-09-17T08:11:08.613277+0000","last_fullsized":"2024-09-17T08:11:08.613277+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.516200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349425+0000","last_change":"2024-09-17T08:09:56.500998+0000","last_active":"2024-09-17T08:11:08.349425+0000","last_peered":"2024-09-17T08:11:08.349425+0000","last_clean":"2024-09-17T08:11:08.349425+0000","last_became_active":"2024-09-17T08:09:56.500692+0000","last_became_peered":"2024-09-17T08:09:56.500692+0000","last_unstale":"2024-09-17T08:11:08.349425+0000","last_undegraded":"2024-09-17T08:11:08.349425+0000","last_fullsized":"2024-09-17T08:11:08.349425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.970578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345159+0000","last_change":"2024-09-17T08:09:58.507646+0000","last_active":"2024-09-17T08:11:08.345159+0000","last_peered":"2024-09-17T08:11:08.345159+0000","last_clean":"2024-09-17T08:11:08.345159+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:08.345159+0000","last_undegraded":"2024-09-17T08:11:08.345159+0000","last_fullsized":"2024-09-17T08:11:08.345159+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349560+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:08.349560+0000","last_peered":"2024-09-17T08:11:08.349560+0000","last_clean":"2024-09-17T08:11:08.349560+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:08.349560+0000","last_undegraded":"2024-09-17T08:11:08.349560+0000","last_fullsized":"2024-09-17T08:11:08.349560+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492990+0000","last_change":"2024-09-17T08:09:54.507191+0000","last_active":"2024-09-17T08:11:08.492990+0000","last_peered":"2024-09-17T08:11:08.492990+0000","last_clean":"2024-09-17T08:11:08.492990+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:08.492990+0000","last_undegraded":"2024-09-17T08:11:08.492990+0000","last_fullsized":"2024-09-17T08:11:08.492990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"72'5","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349539+0000","last_change":"2024-09-17T08:09:56.503460+0000","last_active":"2024-09-17T08:11:08.349539+0000","last_peered":"2024-09-17T08:11:08.349539+0000","last_clean":"2024-09-17T08:11:08.349539+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:08.349539+0000","last_undegraded":"2024-09-17T08:11:08.349539+0000","last_fullsized":"2024-09-17T08:11:08.349539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:19.318133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492095+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:08.492095+0000","last_peered":"2024-09-17T08:11:08.492095+0000","last_clean":"2024-09-17T08:11:08.492095+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:08.492095+0000","last_undegraded":"2024-09-17T08:11:08.492095+0000","last_fullsized":"2024-09-17T08:11:08.492095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831013+0000","last_change":"2024-09-17T08:10:00.517064+0000","last_active":"2024-09-17T08:11:08.831013+0000","last_peered":"2024-09-17T08:11:08.831013+0000","last_clean":"2024-09-17T08:11:08.831013+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:08.831013+0000","last_undegraded":"2024-09-17T08:11:08.831013+0000","last_fullsized":"2024-09-17T08:11:08.831013+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:24.078122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492981+0000","last_change":"2024-09-17T08:09:54.502336+0000","last_active":"2024-09-17T08:11:08.492981+0000","last_peered":"2024-09-17T08:11:08.492981+0000","last_clean":"2024-09-17T08:11:08.492981+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:08.492981+0000","last_undegraded":"2024-09-17T08:11:08.492981+0000","last_fullsized":"2024-09-17T08:11:08.492981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349842+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:08.349842+0000","last_peered":"2024-09-17T08:11:08.349842+0000","last_clean":"2024-09-17T08:11:08.349842+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:08.349842+0000","last_undegraded":"2024-09-17T08:11:08.349842+0000","last_fullsized":"2024-09-17T08:11:08.349842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831129+0000","last_change":"2024-09-17T08:09:58.512901+0000","last_active":"2024-09-17T08:11:08.831129+0000","last_peered":"2024-09-17T08:11:08.831129+0000","last_clean":"2024-09-17T08:11:08.831129+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:08.831129+0000","last_undegraded":"2024-09-17T08:11:08.831129+0000","last_fullsized":"2024-09-17T08:11:08.831129+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830484+0000","last_change":"2024-09-17T08:10:00.520858+0000","last_active":"2024-09-17T08:11:08.830484+0000","last_peered":"2024-09-17T08:11:08.830484+0000","last_clean":"2024-09-17T08:11:08.830484+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:08.830484+0000","last_undegraded":"2024-09-17T08:11:08.830484+0000","last_fullsized":"2024-09-17T08:11:08.830484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350253+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:08.350253+0000","last_peered":"2024-09-17T08:11:08.350253+0000","last_clean":"2024-09-17T08:11:08.350253+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:08.350253+0000","last_undegraded":"2024-09-17T08:11:08.350253+0000","last_fullsized":"2024-09-17T08:11:08.350253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492553+0000","last_change":"2024-09-17T08:09:54.503924+0000","last_active":"2024-09-17T08:11:08.492553+0000","last_peered":"2024-09-17T08:11:08.492553+0000","last_clean":"2024-09-17T08:11:08.492553+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:08.492553+0000","last_undegraded":"2024-09-17T08:11:08.492553+0000","last_fullsized":"2024-09-17T08:11:08.492553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346285+0000","last_change":"2024-09-17T08:09:58.506645+0000","last_active":"2024-09-17T08:11:08.346285+0000","last_peered":"2024-09-17T08:11:08.346285+0000","last_clean":"2024-09-17T08:11:08.346285+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:08.346285+0000","last_undegraded":"2024-09-17T08:11:08.346285+0000","last_fullsized":"2024-09-17T08:11:08.346285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.614662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348162+0000","last_change":"2024-09-17T08:10:00.521440+0000","last_active":"2024-09-17T08:11:08.348162+0000","last_peered":"2024-09-17T08:11:08.348162+0000","last_clean":"2024-09-17T08:11:08.348162+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:08.348162+0000","last_undegraded":"2024-09-17T08:11:08.348162+0000","last_fullsized":"2024-09-17T08:11:08.348162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.963646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349835+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:08.349835+0000","last_peered":"2024-09-17T08:11:08.349835+0000","last_clean":"2024-09-17T08:11:08.349835+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:08.349835+0000","last_undegraded":"2024-09-17T08:11:08.349835+0000","last_fullsized":"2024-09-17T08:11:08.349835+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346248+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:08.346248+0000","last_peered":"2024-09-17T08:11:08.346248+0000","last_clean":"2024-09-17T08:11:08.346248+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:08.346248+0000","last_undegraded":"2024-09-17T08:11:08.346248+0000","last_fullsized":"2024-09-17T08:11:08.346248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352891+0000","last_change":"2024-09-17T08:09:58.512945+0000","last_active":"2024-09-17T08:11:11.352891+0000","last_peered":"2024-09-17T08:11:11.352891+0000","last_clean":"2024-09-17T08:11:11.352891+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:11.352891+0000","last_undegraded":"2024-09-17T08:11:11.352891+0000","last_fullsized":"2024-09-17T08:11:11.352891+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350018+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:08.350018+0000","last_peered":"2024-09-17T08:11:08.350018+0000","last_clean":"2024-09-17T08:11:08.350018+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:08.350018+0000","last_undegraded":"2024-09-17T08:11:08.350018+0000","last_fullsized":"2024-09-17T08:11:08.350018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.558426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.969980+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:16.969980+0000","last_peered":"2024-09-17T08:11:16.969980+0000","last_clean":"2024-09-17T08:11:16.969980+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:16.969980+0000","last_undegraded":"2024-09-17T08:11:16.969980+0000","last_fullsized":"2024-09-17T08:11:16.969980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345811+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:08.345811+0000","last_peered":"2024-09-17T08:11:08.345811+0000","last_clean":"2024-09-17T08:11:08.345811+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:08.345811+0000","last_undegraded":"2024-09-17T08:11:08.345811+0000","last_fullsized":"2024-09-17T08:11:08.345811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.358620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154991+0000","last_change":"2024-09-17T08:09:58.508668+0000","last_active":"2024-09-17T08:11:11.154991+0000","last_peered":"2024-09-17T08:11:11.154991+0000","last_clean":"2024-09-17T08:11:11.154991+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:11.154991+0000","last_undegraded":"2024-09-17T08:11:11.154991+0000","last_fullsized":"2024-09-17T08:11:11.154991+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831269+0000","last_change":"2024-09-17T08:10:00.517958+0000","last_active":"2024-09-17T08:11:08.831269+0000","last_peered":"2024-09-17T08:11:08.831269+0000","last_clean":"2024-09-17T08:11:08.831269+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:08.831269+0000","last_undegraded":"2024-09-17T08:11:08.831269+0000","last_fullsized":"2024-09-17T08:11:08.831269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.257059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350211+0000","last_change":"2024-09-17T08:09:56.500368+0000","last_active":"2024-09-17T08:11:08.350211+0000","last_peered":"2024-09-17T08:11:08.350211+0000","last_clean":"2024-09-17T08:11:08.350211+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:08.350211+0000","last_undegraded":"2024-09-17T08:11:08.350211+0000","last_fullsized":"2024-09-17T08:11:08.350211+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492037+0000","last_change":"2024-09-17T08:09:54.506854+0000","last_active":"2024-09-17T08:11:08.492037+0000","last_peered":"2024-09-17T08:11:08.492037+0000","last_clean":"2024-09-17T08:11:08.492037+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:08.492037+0000","last_undegraded":"2024-09-17T08:11:08.492037+0000","last_fullsized":"2024-09-17T08:11:08.492037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.363057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831578+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:08.831578+0000","last_peered":"2024-09-17T08:11:08.831578+0000","last_clean":"2024-09-17T08:11:08.831578+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:08.831578+0000","last_undegraded":"2024-09-17T08:11:08.831578+0000","last_fullsized":"2024-09-17T08:11:08.831578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612948+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:08.612948+0000","last_peered":"2024-09-17T08:11:08.612948+0000","last_clean":"2024-09-17T08:11:08.612948+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:08.612948+0000","last_undegraded":"2024-09-17T08:11:08.612948+0000","last_fullsized":"2024-09-17T08:11:08.612948+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.578880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349917+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:08.349917+0000","last_peered":"2024-09-17T08:11:08.349917+0000","last_clean":"2024-09-17T08:11:08.349917+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:08.349917+0000","last_undegraded":"2024-09-17T08:11:08.349917+0000","last_fullsized":"2024-09-17T08:11:08.349917+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494544+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:08.494544+0000","last_peered":"2024-09-17T08:11:08.494544+0000","last_clean":"2024-09-17T08:11:08.494544+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:08.494544+0000","last_undegraded":"2024-09-17T08:11:08.494544+0000","last_fullsized":"2024-09-17T08:11:08.494544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352476+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:11.352476+0000","last_peered":"2024-09-17T08:11:11.352476+0000","last_clean":"2024-09-17T08:11:11.352476+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:11.352476+0000","last_undegraded":"2024-09-17T08:11:11.352476+0000","last_fullsized":"2024-09-17T08:11:11.352476+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.469667+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:11.469667+0000","last_peered":"2024-09-17T08:11:11.469667+0000","last_clean":"2024-09-17T08:11:11.469667+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:11.469667+0000","last_undegraded":"2024-09-17T08:11:11.469667+0000","last_fullsized":"2024-09-17T08:11:11.469667+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612611+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:08.612611+0000","last_peered":"2024-09-17T08:11:08.612611+0000","last_clean":"2024-09-17T08:11:08.612611+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:08.612611+0000","last_undegraded":"2024-09-17T08:11:08.612611+0000","last_fullsized":"2024-09-17T08:11:08.612611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346189+0000","last_change":"2024-09-17T08:09:54.498648+0000","last_active":"2024-09-17T08:11:08.346189+0000","last_peered":"2024-09-17T08:11:08.346189+0000","last_clean":"2024-09-17T08:11:08.346189+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:08.346189+0000","last_undegraded":"2024-09-17T08:11:08.346189+0000","last_fullsized":"2024-09-17T08:11:08.346189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:19.423047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349590+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:08.349590+0000","last_peered":"2024-09-17T08:11:08.349590+0000","last_clean":"2024-09-17T08:11:08.349590+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:08.349590+0000","last_undegraded":"2024-09-17T08:11:08.349590+0000","last_fullsized":"2024-09-17T08:11:08.349590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.957574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349711+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:08.349711+0000","last_peered":"2024-09-17T08:11:08.349711+0000","last_clean":"2024-09-17T08:11:08.349711+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:08.349711+0000","last_undegraded":"2024-09-17T08:11:08.349711+0000","last_fullsized":"2024-09-17T08:11:08.349711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.596565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830839+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:08.830839+0000","last_peered":"2024-09-17T08:11:08.830839+0000","last_clean":"2024-09-17T08:11:08.830839+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:08.830839+0000","last_undegraded":"2024-09-17T08:11:08.830839+0000","last_fullsized":"2024-09-17T08:11:08.830839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.493061+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:08.493061+0000","last_peered":"2024-09-17T08:11:08.493061+0000","last_clean":"2024-09-17T08:11:08.493061+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:08.493061+0000","last_undegraded":"2024-09-17T08:11:08.493061+0000","last_fullsized":"2024-09-17T08:11:08.493061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494046+0000","last_change":"2024-09-17T08:09:58.506025+0000","last_active":"2024-09-17T08:11:08.494046+0000","last_peered":"2024-09-17T08:11:08.494046+0000","last_clean":"2024-09-17T08:11:08.494046+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:08.494046+0000","last_undegraded":"2024-09-17T08:11:08.494046+0000","last_fullsized":"2024-09-17T08:11:08.494046+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.291171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.026108+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:11.026108+0000","last_peered":"2024-09-17T08:11:11.026108+0000","last_clean":"2024-09-17T08:11:11.026108+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:11.026108+0000","last_undegraded":"2024-09-17T08:11:11.026108+0000","last_fullsized":"2024-09-17T08:11:11.026108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345562+0000","last_change":"2024-09-17T08:09:56.502524+0000","last_active":"2024-09-17T08:11:08.345562+0000","last_peered":"2024-09-17T08:11:08.345562+0000","last_clean":"2024-09-17T08:11:08.345562+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:08.345562+0000","last_undegraded":"2024-09-17T08:11:08.345562+0000","last_fullsized":"2024-09-17T08:11:08.345562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.354486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346618+0000","last_change":"2024-09-17T08:09:54.498729+0000","last_active":"2024-09-17T08:11:08.346618+0000","last_peered":"2024-09-17T08:11:08.346618+0000","last_clean":"2024-09-17T08:11:08.346618+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:08.346618+0000","last_undegraded":"2024-09-17T08:11:08.346618+0000","last_fullsized":"2024-09-17T08:11:08.346618+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.547950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155283+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:11.155283+0000","last_peered":"2024-09-17T08:11:11.155283+0000","last_clean":"2024-09-17T08:11:11.155283+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:11.155283+0000","last_undegraded":"2024-09-17T08:11:11.155283+0000","last_fullsized":"2024-09-17T08:11:11.155283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830401+0000","last_change":"2024-09-17T08:10:00.516499+0000","last_active":"2024-09-17T08:11:08.830401+0000","last_peered":"2024-09-17T08:11:08.830401+0000","last_clean":"2024-09-17T08:11:08.830401+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:08.830401+0000","last_undegraded":"2024-09-17T08:11:08.830401+0000","last_fullsized":"2024-09-17T08:11:08.830401+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.391480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154959+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:11.154959+0000","last_peered":"2024-09-17T08:11:11.154959+0000","last_clean":"2024-09-17T08:11:11.154959+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:11.154959+0000","last_undegraded":"2024-09-17T08:11:11.154959+0000","last_fullsized":"2024-09-17T08:11:11.154959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831678+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:08.831678+0000","last_peered":"2024-09-17T08:11:08.831678+0000","last_clean":"2024-09-17T08:11:08.831678+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:08.831678+0000","last_undegraded":"2024-09-17T08:11:08.831678+0000","last_fullsized":"2024-09-17T08:11:08.831678+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830963+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:08.830963+0000","last_peered":"2024-09-17T08:11:08.830963+0000","last_clean":"2024-09-17T08:11:08.830963+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:08.830963+0000","last_undegraded":"2024-09-17T08:11:08.830963+0000","last_fullsized":"2024-09-17T08:11:08.830963+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:10.030674+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:10.030674+0000","last_peered":"2024-09-17T08:11:10.030674+0000","last_clean":"2024-09-17T08:11:10.030674+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:10.030674+0000","last_undegraded":"2024-09-17T08:11:10.030674+0000","last_fullsized":"2024-09-17T08:11:10.030674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346624+0000","last_change":"2024-09-17T08:09:56.503791+0000","last_active":"2024-09-17T08:11:08.346624+0000","last_peered":"2024-09-17T08:11:08.346624+0000","last_clean":"2024-09-17T08:11:08.346624+0000","last_became_active":"2024-09-17T08:09:56.503476+0000","last_became_peered":"2024-09-17T08:09:56.503476+0000","last_unstale":"2024-09-17T08:11:08.346624+0000","last_undegraded":"2024-09-17T08:11:08.346624+0000","last_fullsized":"2024-09-17T08:11:08.346624+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.579571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":124,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.158068+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:16.158068+0000","last_peered":"2024-09-17T08:11:16.158068+0000","last_clean":"2024-09-17T08:11:16.158068+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:16.158068+0000","last_undegraded":"2024-09-17T08:11:16.158068+0000","last_fullsized":"2024-09-17T08:11:16.158068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347634+0000","last_change":"2024-09-17T08:09:54.504840+0000","last_active":"2024-09-17T08:11:08.347634+0000","last_peered":"2024-09-17T08:11:08.347634+0000","last_clean":"2024-09-17T08:11:08.347634+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:08.347634+0000","last_undegraded":"2024-09-17T08:11:08.347634+0000","last_fullsized":"2024-09-17T08:11:08.347634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348090+0000","last_change":"2024-09-17T08:09:58.514115+0000","last_active":"2024-09-17T08:11:08.348090+0000","last_peered":"2024-09-17T08:11:08.348090+0000","last_clean":"2024-09-17T08:11:08.348090+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:08.348090+0000","last_undegraded":"2024-09-17T08:11:08.348090+0000","last_fullsized":"2024-09-17T08:11:08.348090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492725+0000","last_change":"2024-09-17T08:10:00.599802+0000","last_active":"2024-09-17T08:11:08.492725+0000","last_peered":"2024-09-17T08:11:08.492725+0000","last_clean":"2024-09-17T08:11:08.492725+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:08.492725+0000","last_undegraded":"2024-09-17T08:11:08.492725+0000","last_fullsized":"2024-09-17T08:11:08.492725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.895697+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:11.895697+0000","last_peered":"2024-09-17T08:11:11.895697+0000","last_clean":"2024-09-17T08:11:11.895697+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:11.895697+0000","last_undegraded":"2024-09-17T08:11:11.895697+0000","last_fullsized":"2024-09-17T08:11:11.895697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830657+0000","last_change":"2024-09-17T08:09:58.498654+0000","last_active":"2024-09-17T08:11:08.830657+0000","last_peered":"2024-09-17T08:11:08.830657+0000","last_clean":"2024-09-17T08:11:08.830657+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:08.830657+0000","last_undegraded":"2024-09-17T08:11:08.830657+0000","last_fullsized":"2024-09-17T08:11:08.830657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.615615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492108+0000","last_change":"2024-09-17T08:09:54.507600+0000","last_active":"2024-09-17T08:11:08.492108+0000","last_peered":"2024-09-17T08:11:08.492108+0000","last_clean":"2024-09-17T08:11:08.492108+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:08.492108+0000","last_undegraded":"2024-09-17T08:11:08.492108+0000","last_fullsized":"2024-09-17T08:11:08.492108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.885315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831705+0000","last_change":"2024-09-17T08:09:58.502361+0000","last_active":"2024-09-17T08:11:08.831705+0000","last_peered":"2024-09-17T08:11:08.831705+0000","last_clean":"2024-09-17T08:11:08.831705+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:08.831705+0000","last_undegraded":"2024-09-17T08:11:08.831705+0000","last_fullsized":"2024-09-17T08:11:08.831705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.530587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346193+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:08.346193+0000","last_peered":"2024-09-17T08:11:08.346193+0000","last_clean":"2024-09-17T08:11:08.346193+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:08.346193+0000","last_undegraded":"2024-09-17T08:11:08.346193+0000","last_fullsized":"2024-09-17T08:11:08.346193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.624827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.289684+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:11.289684+0000","last_peered":"2024-09-17T08:11:11.289684+0000","last_clean":"2024-09-17T08:11:11.289684+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:11.289684+0000","last_undegraded":"2024-09-17T08:11:11.289684+0000","last_fullsized":"2024-09-17T08:11:11.289684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612823+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:08.612823+0000","last_peered":"2024-09-17T08:11:08.612823+0000","last_clean":"2024-09-17T08:11:08.612823+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:08.612823+0000","last_undegraded":"2024-09-17T08:11:08.612823+0000","last_fullsized":"2024-09-17T08:11:08.612823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.254805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831699+0000","last_change":"2024-09-17T08:09:54.502513+0000","last_active":"2024-09-17T08:11:08.831699+0000","last_peered":"2024-09-17T08:11:08.831699+0000","last_clean":"2024-09-17T08:11:08.831699+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:08.831699+0000","last_undegraded":"2024-09-17T08:11:08.831699+0000","last_fullsized":"2024-09-17T08:11:08.831699+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:12.414362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":290,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492403+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:08.492403+0000","last_peered":"2024-09-17T08:11:08.492403+0000","last_clean":"2024-09-17T08:11:08.492403+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:08.492403+0000","last_undegraded":"2024-09-17T08:11:08.492403+0000","last_fullsized":"2024-09-17T08:11:08.492403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.066267+0000","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492678+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:08.492678+0000","last_peered":"2024-09-17T08:11:08.492678+0000","last_clean":"2024-09-17T08:11:08.492678+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:08.492678+0000","last_undegraded":"2024-09-17T08:11:08.492678+0000","last_fullsized":"2024-09-17T08:11:08.492678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.857213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831709+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:08.831709+0000","last_peered":"2024-09-17T08:11:08.831709+0000","last_clean":"2024-09-17T08:11:08.831709+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:08.831709+0000","last_undegraded":"2024-09-17T08:11:08.831709+0000","last_fullsized":"2024-09-17T08:11:08.831709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494397+0000","last_change":"2024-09-17T08:09:57.041231+0000","last_active":"2024-09-17T08:11:08.494397+0000","last_peered":"2024-09-17T08:11:08.494397+0000","last_clean":"2024-09-17T08:11:08.494397+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:08.494397+0000","last_undegraded":"2024-09-17T08:11:08.494397+0000","last_fullsized":"2024-09-17T08:11:08.494397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.855672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348259+0000","last_change":"2024-09-17T08:09:54.502363+0000","last_active":"2024-09-17T08:11:08.348259+0000","last_peered":"2024-09-17T08:11:08.348259+0000","last_clean":"2024-09-17T08:11:08.348259+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:08.348259+0000","last_undegraded":"2024-09-17T08:11:08.348259+0000","last_fullsized":"2024-09-17T08:11:08.348259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.973864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.148121+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:15.148121+0000","last_peered":"2024-09-17T08:11:15.148121+0000","last_clean":"2024-09-17T08:11:15.148121+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:15.148121+0000","last_undegraded":"2024-09-17T08:11:15.148121+0000","last_fullsized":"2024-09-17T08:11:15.148121+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492248+0000","last_change":"2024-09-17T08:10:00.525443+0000","last_active":"2024-09-17T08:11:08.492248+0000","last_peered":"2024-09-17T08:11:08.492248+0000","last_clean":"2024-09-17T08:11:08.492248+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:08.492248+0000","last_undegraded":"2024-09-17T08:11:08.492248+0000","last_fullsized":"2024-09-17T08:11:08.492248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830294+0000","last_change":"2024-09-17T08:09:56.506519+0000","last_active":"2024-09-17T08:11:08.830294+0000","last_peered":"2024-09-17T08:11:08.830294+0000","last_clean":"2024-09-17T08:11:08.830294+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:08.830294+0000","last_undegraded":"2024-09-17T08:11:08.830294+0000","last_fullsized":"2024-09-17T08:11:08.830294+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.648151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347044+0000","last_change":"2024-09-17T08:09:54.499164+0000","last_active":"2024-09-17T08:11:08.347044+0000","last_peered":"2024-09-17T08:11:08.347044+0000","last_clean":"2024-09-17T08:11:08.347044+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:08.347044+0000","last_undegraded":"2024-09-17T08:11:08.347044+0000","last_fullsized":"2024-09-17T08:11:08.347044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:13.690089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347696+0000","last_change":"2024-09-17T08:09:58.511946+0000","last_active":"2024-09-17T08:11:08.347696+0000","last_peered":"2024-09-17T08:11:08.347696+0000","last_clean":"2024-09-17T08:11:08.347696+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:08.347696+0000","last_undegraded":"2024-09-17T08:11:08.347696+0000","last_fullsized":"2024-09-17T08:11:08.347696+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830617+0000","last_change":"2024-09-17T08:10:00.521382+0000","last_active":"2024-09-17T08:11:08.830617+0000","last_peered":"2024-09-17T08:11:08.830617+0000","last_clean":"2024-09-17T08:11:08.830617+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:08.830617+0000","last_undegraded":"2024-09-17T08:11:08.830617+0000","last_fullsized":"2024-09-17T08:11:08.830617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:12.458130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830961+0000","last_change":"2024-09-17T08:09:57.041079+0000","last_active":"2024-09-17T08:11:08.830961+0000","last_peered":"2024-09-17T08:11:08.830961+0000","last_clean":"2024-09-17T08:11:08.830961+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:08.830961+0000","last_undegraded":"2024-09-17T08:11:08.830961+0000","last_fullsized":"2024-09-17T08:11:08.830961+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.635691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348343+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:08.348343+0000","last_peered":"2024-09-17T08:11:08.348343+0000","last_clean":"2024-09-17T08:11:08.348343+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:08.348343+0000","last_undegraded":"2024-09-17T08:11:08.348343+0000","last_fullsized":"2024-09-17T08:11:08.348343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290733+0000","last_change":"2024-09-17T08:09:58.513726+0000","last_active":"2024-09-17T08:11:11.290733+0000","last_peered":"2024-09-17T08:11:11.290733+0000","last_clean":"2024-09-17T08:11:11.290733+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:11.290733+0000","last_undegraded":"2024-09-17T08:11:11.290733+0000","last_fullsized":"2024-09-17T08:11:11.290733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.608466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831127+0000","last_change":"2024-09-17T08:10:00.599030+0000","last_active":"2024-09-17T08:11:08.831127+0000","last_peered":"2024-09-17T08:11:08.831127+0000","last_clean":"2024-09-17T08:11:08.831127+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:08.831127+0000","last_undegraded":"2024-09-17T08:11:08.831127+0000","last_fullsized":"2024-09-17T08:11:08.831127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:26.441216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.614679+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:08.614679+0000","last_peered":"2024-09-17T08:11:08.614679+0000","last_clean":"2024-09-17T08:11:08.614679+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:08.614679+0000","last_undegraded":"2024-09-17T08:11:08.614679+0000","last_fullsized":"2024-09-17T08:11:08.614679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:10.663912+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:10.663912+0000","last_peered":"2024-09-17T08:11:10.663912+0000","last_clean":"2024-09-17T08:11:10.663912+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:10.663912+0000","last_undegraded":"2024-09-17T08:11:10.663912+0000","last_fullsized":"2024-09-17T08:11:10.663912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347666+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:08.347666+0000","last_peered":"2024-09-17T08:11:08.347666+0000","last_clean":"2024-09-17T08:11:08.347666+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:08.347666+0000","last_undegraded":"2024-09-17T08:11:08.347666+0000","last_fullsized":"2024-09-17T08:11:08.347666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612904+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:08.612904+0000","last_peered":"2024-09-17T08:11:08.612904+0000","last_clean":"2024-09-17T08:11:08.612904+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:08.612904+0000","last_undegraded":"2024-09-17T08:11:08.612904+0000","last_fullsized":"2024-09-17T08:11:08.612904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"72'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831110+0000","last_change":"2024-09-17T08:09:57.041185+0000","last_active":"2024-09-17T08:11:08.831110+0000","last_peered":"2024-09-17T08:11:08.831110+0000","last_clean":"2024-09-17T08:11:08.831110+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:08.831110+0000","last_undegraded":"2024-09-17T08:11:08.831110+0000","last_fullsized":"2024-09-17T08:11:08.831110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.347750+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:08.347750+0000","last_peered":"2024-09-17T08:11:08.347750+0000","last_clean":"2024-09-17T08:11:08.347750+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:08.347750+0000","last_undegraded":"2024-09-17T08:11:08.347750+0000","last_fullsized":"2024-09-17T08:11:08.347750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613233+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:08.613233+0000","last_peered":"2024-09-17T08:11:08.613233+0000","last_clean":"2024-09-17T08:11:08.613233+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:08.613233+0000","last_undegraded":"2024-09-17T08:11:08.613233+0000","last_fullsized":"2024-09-17T08:11:08.613233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.110620+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:14.110620+0000","last_peered":"2024-09-17T08:11:14.110620+0000","last_clean":"2024-09-17T08:11:14.110620+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:14.110620+0000","last_undegraded":"2024-09-17T08:11:14.110620+0000","last_fullsized":"2024-09-17T08:11:14.110620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494215+0000","last_change":"2024-09-17T08:09:54.498318+0000","last_active":"2024-09-17T08:11:08.494215+0000","last_peered":"2024-09-17T08:11:08.494215+0000","last_clean":"2024-09-17T08:11:08.494215+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:08.494215+0000","last_undegraded":"2024-09-17T08:11:08.494215+0000","last_fullsized":"2024-09-17T08:11:08.494215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.145703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613253+0000","last_change":"2024-09-17T08:09:57.043364+0000","last_active":"2024-09-17T08:11:08.613253+0000","last_peered":"2024-09-17T08:11:08.613253+0000","last_clean":"2024-09-17T08:11:08.613253+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:08.613253+0000","last_undegraded":"2024-09-17T08:11:08.613253+0000","last_fullsized":"2024-09-17T08:11:08.613253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.183678+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:12.183678+0000","last_peered":"2024-09-17T08:11:12.183678+0000","last_clean":"2024-09-17T08:11:12.183678+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:12.183678+0000","last_undegraded":"2024-09-17T08:11:12.183678+0000","last_fullsized":"2024-09-17T08:11:12.183678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492728+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:08.492728+0000","last_peered":"2024-09-17T08:11:08.492728+0000","last_clean":"2024-09-17T08:11:08.492728+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:08.492728+0000","last_undegraded":"2024-09-17T08:11:08.492728+0000","last_fullsized":"2024-09-17T08:11:08.492728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613065+0000","last_change":"2024-09-17T08:09:57.041188+0000","last_active":"2024-09-17T08:11:08.613065+0000","last_peered":"2024-09-17T08:11:08.613065+0000","last_clean":"2024-09-17T08:11:08.613065+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:08.613065+0000","last_undegraded":"2024-09-17T08:11:08.613065+0000","last_fullsized":"2024-09-17T08:11:08.613065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831504+0000","last_change":"2024-09-17T08:10:54.331930+0000","last_active":"2024-09-17T08:11:08.831504+0000","last_peered":"2024-09-17T08:11:08.831504+0000","last_clean":"2024-09-17T08:11:08.831504+0000","last_became_active":"2024-09-17T08:09:54.501179+0000","last_became_peered":"2024-09-17T08:09:54.501179+0000","last_unstale":"2024-09-17T08:11:08.831504+0000","last_undegraded":"2024-09-17T08:11:08.831504+0000","last_fullsized":"2024-09-17T08:11:08.831504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.084651+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.217500+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:11.217500+0000","last_peered":"2024-09-17T08:11:11.217500+0000","last_clean":"2024-09-17T08:11:11.217500+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:11.217500+0000","last_undegraded":"2024-09-17T08:11:11.217500+0000","last_fullsized":"2024-09-17T08:11:11.217500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830917+0000","last_change":"2024-09-17T08:10:00.521904+0000","last_active":"2024-09-17T08:11:08.830917+0000","last_peered":"2024-09-17T08:11:08.830917+0000","last_clean":"2024-09-17T08:11:08.830917+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:08.830917+0000","last_undegraded":"2024-09-17T08:11:08.830917+0000","last_fullsized":"2024-09-17T08:11:08.830917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.331400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.214203+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:13.214203+0000","last_peered":"2024-09-17T08:11:13.214203+0000","last_clean":"2024-09-17T08:11:13.214203+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:13.214203+0000","last_undegraded":"2024-09-17T08:11:13.214203+0000","last_fullsized":"2024-09-17T08:11:13.214203+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831082+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:08.831082+0000","last_peered":"2024-09-17T08:11:08.831082+0000","last_clean":"2024-09-17T08:11:08.831082+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:08.831082+0000","last_undegraded":"2024-09-17T08:11:08.831082+0000","last_fullsized":"2024-09-17T08:11:08.831082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":49,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352587+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:11.352587+0000","last_peered":"2024-09-17T08:11:11.352587+0000","last_clean":"2024-09-17T08:11:11.352587+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:11.352587+0000","last_undegraded":"2024-09-17T08:11:11.352587+0000","last_fullsized":"2024-09-17T08:11:11.352587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349952+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:08.349952+0000","last_peered":"2024-09-17T08:11:08.349952+0000","last_clean":"2024-09-17T08:11:08.349952+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:08.349952+0000","last_undegraded":"2024-09-17T08:11:08.349952+0000","last_fullsized":"2024-09-17T08:11:08.349952+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.032099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.345472+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:08.345472+0000","last_peered":"2024-09-17T08:11:08.345472+0000","last_clean":"2024-09-17T08:11:08.345472+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:08.345472+0000","last_undegraded":"2024-09-17T08:11:08.345472+0000","last_fullsized":"2024-09-17T08:11:08.345472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.317178+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:12.317178+0000","last_peered":"2024-09-17T08:11:12.317178+0000","last_clean":"2024-09-17T08:11:12.317178+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:12.317178+0000","last_undegraded":"2024-09-17T08:11:12.317178+0000","last_fullsized":"2024-09-17T08:11:12.317178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612951+0000","last_change":"2024-09-17T08:09:58.508365+0000","last_active":"2024-09-17T08:11:08.612951+0000","last_peered":"2024-09-17T08:11:08.612951+0000","last_clean":"2024-09-17T08:11:08.612951+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:08.612951+0000","last_undegraded":"2024-09-17T08:11:08.612951+0000","last_fullsized":"2024-09-17T08:11:08.612951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.830383+0000","last_change":"2024-09-17T08:10:00.518008+0000","last_active":"2024-09-17T08:11:08.830383+0000","last_peered":"2024-09-17T08:11:08.830383+0000","last_clean":"2024-09-17T08:11:08.830383+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:08.830383+0000","last_undegraded":"2024-09-17T08:11:08.830383+0000","last_fullsized":"2024-09-17T08:11:08.830383+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.349650+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:08.349650+0000","last_peered":"2024-09-17T08:11:08.349650+0000","last_clean":"2024-09-17T08:11:08.349650+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:08.349650+0000","last_undegraded":"2024-09-17T08:11:08.349650+0000","last_fullsized":"2024-09-17T08:11:08.349650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346972+0000","last_change":"2024-09-17T08:09:54.496656+0000","last_active":"2024-09-17T08:11:08.346972+0000","last_peered":"2024-09-17T08:11:08.346972+0000","last_clean":"2024-09-17T08:11:08.346972+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:08.346972+0000","last_undegraded":"2024-09-17T08:11:08.346972+0000","last_fullsized":"2024-09-17T08:11:08.346972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.344618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.013139+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:12.013139+0000","last_peered":"2024-09-17T08:11:12.013139+0000","last_clean":"2024-09-17T08:11:12.013139+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:12.013139+0000","last_undegraded":"2024-09-17T08:11:12.013139+0000","last_fullsized":"2024-09-17T08:11:12.013139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.086747+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:14.086747+0000","last_peered":"2024-09-17T08:11:14.086747+0000","last_clean":"2024-09-17T08:11:14.086747+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:14.086747+0000","last_undegraded":"2024-09-17T08:11:14.086747+0000","last_fullsized":"2024-09-17T08:11:14.086747+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612255+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:08.612255+0000","last_peered":"2024-09-17T08:11:08.612255+0000","last_clean":"2024-09-17T08:11:08.612255+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:08.612255+0000","last_undegraded":"2024-09-17T08:11:08.612255+0000","last_fullsized":"2024-09-17T08:11:08.612255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831731+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:08.831731+0000","last_peered":"2024-09-17T08:11:08.831731+0000","last_clean":"2024-09-17T08:11:08.831731+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:08.831731+0000","last_undegraded":"2024-09-17T08:11:08.831731+0000","last_fullsized":"2024-09-17T08:11:08.831731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.493947+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:08.493947+0000","last_peered":"2024-09-17T08:11:08.493947+0000","last_clean":"2024-09-17T08:11:08.493947+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:08.493947+0000","last_undegraded":"2024-09-17T08:11:08.493947+0000","last_fullsized":"2024-09-17T08:11:08.493947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492033+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:08.492033+0000","last_peered":"2024-09-17T08:11:08.492033+0000","last_clean":"2024-09-17T08:11:08.492033+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:08.492033+0000","last_undegraded":"2024-09-17T08:11:08.492033+0000","last_fullsized":"2024-09-17T08:11:08.492033+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.985197+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:08.985197+0000","last_peered":"2024-09-17T08:11:08.985197+0000","last_clean":"2024-09-17T08:11:08.985197+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:08.985197+0000","last_undegraded":"2024-09-17T08:11:08.985197+0000","last_fullsized":"2024-09-17T08:11:08.985197+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"65'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.494494+0000","last_change":"2024-09-17T08:09:54.506095+0000","last_active":"2024-09-17T08:11:08.494494+0000","last_peered":"2024-09-17T08:11:08.494494+0000","last_clean":"2024-09-17T08:11:08.494494+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:08.494494+0000","last_undegraded":"2024-09-17T08:11:08.494494+0000","last_fullsized":"2024-09-17T08:11:08.494494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.529326+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:13.529326+0000","last_peered":"2024-09-17T08:11:13.529326+0000","last_clean":"2024-09-17T08:11:13.529326+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:13.529326+0000","last_undegraded":"2024-09-17T08:11:13.529326+0000","last_fullsized":"2024-09-17T08:11:13.529326+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612827+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:08.612827+0000","last_peered":"2024-09-17T08:11:08.612827+0000","last_clean":"2024-09-17T08:11:08.612827+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:08.612827+0000","last_undegraded":"2024-09-17T08:11:08.612827+0000","last_fullsized":"2024-09-17T08:11:08.612827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.492515+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:08.492515+0000","last_peered":"2024-09-17T08:11:08.492515+0000","last_clean":"2024-09-17T08:11:08.492515+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:08.492515+0000","last_undegraded":"2024-09-17T08:11:08.492515+0000","last_fullsized":"2024-09-17T08:11:08.492515+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.700248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.188046+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:16.188046+0000","last_peered":"2024-09-17T08:11:16.188046+0000","last_clean":"2024-09-17T08:11:16.188046+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:16.188046+0000","last_undegraded":"2024-09-17T08:11:16.188046+0000","last_fullsized":"2024-09-17T08:11:16.188046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831019+0000","last_change":"2024-09-17T08:09:57.042542+0000","last_active":"2024-09-17T08:11:08.831019+0000","last_peered":"2024-09-17T08:11:08.831019+0000","last_clean":"2024-09-17T08:11:08.831019+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:08.831019+0000","last_undegraded":"2024-09-17T08:11:08.831019+0000","last_fullsized":"2024-09-17T08:11:08.831019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.333853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288857+0000","last_change":"2024-09-17T08:09:58.515290+0000","last_active":"2024-09-17T08:11:11.288857+0000","last_peered":"2024-09-17T08:11:11.288857+0000","last_clean":"2024-09-17T08:11:11.288857+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:11.288857+0000","last_undegraded":"2024-09-17T08:11:11.288857+0000","last_fullsized":"2024-09-17T08:11:11.288857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346357+0000","last_change":"2024-09-17T08:10:00.523915+0000","last_active":"2024-09-17T08:11:08.346357+0000","last_peered":"2024-09-17T08:11:08.346357+0000","last_clean":"2024-09-17T08:11:08.346357+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:08.346357+0000","last_undegraded":"2024-09-17T08:11:08.346357+0000","last_fullsized":"2024-09-17T08:11:08.346357+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:11.652238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612922+0000","last_change":"2024-09-17T08:09:54.504854+0000","last_active":"2024-09-17T08:11:08.612922+0000","last_peered":"2024-09-17T08:11:08.612922+0000","last_clean":"2024-09-17T08:11:08.612922+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:08.612922+0000","last_undegraded":"2024-09-17T08:11:08.612922+0000","last_fullsized":"2024-09-17T08:11:08.612922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.278783+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:14.278783+0000","last_peered":"2024-09-17T08:11:14.278783+0000","last_clean":"2024-09-17T08:11:14.278783+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:14.278783+0000","last_undegraded":"2024-09-17T08:11:14.278783+0000","last_fullsized":"2024-09-17T08:11:14.278783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.831000+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:08.831000+0000","last_peered":"2024-09-17T08:11:08.831000+0000","last_clean":"2024-09-17T08:11:08.831000+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:08.831000+0000","last_undegraded":"2024-09-17T08:11:08.831000+0000","last_fullsized":"2024-09-17T08:11:08.831000+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.350037+0000","last_change":"2024-09-17T08:10:00.598000+0000","last_active":"2024-09-17T08:11:08.350037+0000","last_peered":"2024-09-17T08:11:08.350037+0000","last_clean":"2024-09-17T08:11:08.350037+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:08.350037+0000","last_undegraded":"2024-09-17T08:11:08.350037+0000","last_fullsized":"2024-09-17T08:11:08.350037+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.707509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.309985+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:13.309985+0000","last_peered":"2024-09-17T08:11:13.309985+0000","last_clean":"2024-09-17T08:11:13.309985+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:13.309985+0000","last_undegraded":"2024-09-17T08:11:13.309985+0000","last_fullsized":"2024-09-17T08:11:13.309985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"72'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.348067+0000","last_change":"2024-09-17T08:09:56.504473+0000","last_active":"2024-09-17T08:11:08.348067+0000","last_peered":"2024-09-17T08:11:08.348067+0000","last_clean":"2024-09-17T08:11:08.348067+0000","last_became_active":"2024-09-17T08:09:56.504114+0000","last_became_peered":"2024-09-17T08:09:56.504114+0000","last_unstale":"2024-09-17T08:11:08.348067+0000","last_undegraded":"2024-09-17T08:11:08.348067+0000","last_fullsized":"2024-09-17T08:11:08.348067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:20.408053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.346807+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:08.346807+0000","last_peered":"2024-09-17T08:11:08.346807+0000","last_clean":"2024-09-17T08:11:08.346807+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:08.346807+0000","last_undegraded":"2024-09-17T08:11:08.346807+0000","last_fullsized":"2024-09-17T08:11:08.346807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.494666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613179+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:08.613179+0000","last_peered":"2024-09-17T08:11:08.613179+0000","last_clean":"2024-09-17T08:11:08.613179+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:08.613179+0000","last_undegraded":"2024-09-17T08:11:08.613179+0000","last_fullsized":"2024-09-17T08:11:08.613179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:16.080158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.613025+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:08.613025+0000","last_peered":"2024-09-17T08:11:08.613025+0000","last_clean":"2024-09-17T08:11:08.613025+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:08.613025+0000","last_undegraded":"2024-09-17T08:11:08.613025+0000","last_fullsized":"2024-09-17T08:11:08.613025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-17T08:11:08.612444+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:08.612444+0000","last_peered":"2024-09-17T08:11:08.612444+0000","last_clean":"2024-09-17T08:11:08.612444+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:08.612444+0000","last_undegraded":"2024-09-17T08:11:08.612444+0000","last_fullsized":"2024-09-17T08:11:08.612444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.637634+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:15.637634+0000","last_peered":"2024-09-17T08:11:15.637634+0000","last_clean":"2024-09-17T08:11:15.637634+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:15.637634+0000","last_undegraded":"2024-09-17T08:11:15.637634+0000","last_fullsized":"2024-09-17T08:11:15.637634+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":397,"ondisk_log_size":397,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070486,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32516,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712636,"statfs":{"total":95995035648,"available":95961739264,"internally_reserved":0,"allocated":1437696,"data_stored":981274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.439}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53700000000000003}]}]},{"osd":6,"up_from":50,"seq":214748364827,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32504,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712648,"statfs":{"total":95995035648,"available":95961751552,"internally_reserved":0,"allocated":1425408,"data_stored":980865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.376}]},{"osd":3,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.623}]},{"osd":5,"last update":"Tue Sep 17 08:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68400000000000005}]}]},{"osd":5,"up_from":41,"seq":176093659168,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31920,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713232,"statfs":{"total":95995035648,"available":95962349568,"internally_reserved":0,"allocated":827392,"data_stored":389842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.41499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.44},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.42299999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:10:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46200000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":391075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.504}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.82299999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.81299999999999994}]}]},{"osd":3,"up_from":27,"seq":115964117033,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":390552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.308},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34300000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.439}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64000000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":390624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.218},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.23200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.26900000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.441},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46000000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.503},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":876544,"data_stored":391362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.30099999999999999},"min":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.24199999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.311}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85099999999999998,"5min":1.329,"15min":1.329},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.622}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1433600,"data_stored":981016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33700000000000002,"15min":0.31900000000000001},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.21199999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:10:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.753,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.504}]},{"osd":6,"last update":"Tue Sep 17 08:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39900000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.437}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:21.204 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T08:11:21.204 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T08:11:21.205 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T08:11:21.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T08:11:21.408 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.5 deep-scrub starts 2024-09-17T08:11:21.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: 6.5 deep-scrub ok 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1017 B/s rd, 0 op/s 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.5 deep-scrub starts 2024-09-17T08:11:21.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: 6.5 deep-scrub ok 2024-09-17T08:11:21.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:21.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:21.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1017 B/s rd, 0 op/s 2024-09-17T08:11:21.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:21 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:21.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.5 deep-scrub starts 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: 6.5 deep-scrub ok 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1017 B/s rd, 0 op/s 2024-09-17T08:11:21.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:21 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:22.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T08:11:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:22 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:22.739 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:11:22.740 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T08:11:22.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:22 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:23.178 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T08:11:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:23.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 4.16 deep-scrub starts 2024-09-17T08:11:23.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 4.16 deep-scrub ok 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 2.1 deep-scrub starts 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: 2.1 deep-scrub ok 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/790491053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:11:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:23 smithi173 ceph-mon[33187]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 4.16 deep-scrub starts 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 4.16 deep-scrub ok 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 2.1 deep-scrub starts 2024-09-17T08:11:23.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: 2.1 deep-scrub ok 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/790491053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[40649]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 4.16 deep-scrub starts 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 4.16 deep-scrub ok 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 2.1 deep-scrub starts 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: 2.1 deep-scrub ok 2024-09-17T08:11:23.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/790491053' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T08:11:23.772 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:23 smithi079 ceph-mon[29565]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:23.791 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T08:11:23.791 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T08:11:23.791 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T08:11:23.804 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T08:11:23.804 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'reef', 'sha1': '1cbdc9211df088cccbd94423c7cf3a2b85fe432b'}} 2024-09-17T08:11:23.804 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T08:11:23.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T08:11:24.001 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:24.677 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:24 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:11:24.677 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:24 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:11:24.678 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:24 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:11:24.678 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:24 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:11:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:11:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:11:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:24 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:11:25.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[40649]: 5.a deep-scrub starts 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[40649]: 5.a deep-scrub ok 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[40649]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[29565]: 5.a deep-scrub starts 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[29565]: 5.a deep-scrub ok 2024-09-17T08:11:25.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-mon[29565]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:25 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:25 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:25 smithi173 ceph-mon[33187]: 5.a deep-scrub starts 2024-09-17T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:25 smithi173 ceph-mon[33187]: 5.a deep-scrub ok 2024-09-17T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:25 smithi173 ceph-mon[33187]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:26.259 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:25 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:25] ENGINE Bus STOPPING 2024-09-17T08:11:26.259 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:11:26.260 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:26] ENGINE Bus STOPPED 2024-09-17T08:11:26.260 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:26] ENGINE Bus STARTING 2024-09-17T08:11:26.260 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:26] ENGINE Serving on http://:::9283 2024-09-17T08:11:26.260 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:11:26] ENGINE Bus STARTED 2024-09-17T08:11:26.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1744266804' entity='client.admin' 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:11:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1744266804' entity='client.admin' 2024-09-17T08:11:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:11:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:11:26.700 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1744266804' entity='client.admin' 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:11:26.701 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:26 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:11:26.706 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T08:11:26.706 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T08:11:26.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:26.901 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:27.021 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:26 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[76373]: ts=2024-09-17T08:11:26.821Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003713472s 2024-09-17T08:11:27.349 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 systemd[1]: Starting Ceph grafana.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:27.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:11:27.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:11:27.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[29565]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:27.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:11:27.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:11:27.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:27 smithi079 ceph-mon[40649]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:27.663 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:11:27.663 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:11:27.663 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-mon[33187]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:27.664 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 podman[66390]: 2024-09-17 08:11:27.277213199 +0000 UTC m=+0.015218891 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-17T08:11:27.664 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 podman[66390]: 2024-09-17 08:11:27.403344529 +0000 UTC m=+0.141350211 container create 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, description=Ceph Grafana Container, distribution-scope=public, maintainer=Paul Cuzner , vcs-type=git, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664) 2024-09-17T08:11:27.664 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 podman[66390]: 2024-09-17 08:11:27.664014545 +0000 UTC m=+0.402020239 container init 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, name=ubi8, distribution-scope=public, maintainer=Paul Cuzner , release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, io.openshift.tags=base rhel8, io.buildah.version=1.32.0) 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 podman[66390]: 2024-09-17 08:11:27.668792522 +0000 UTC m=+0.406798212 container start 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, distribution-scope=public, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, architecture=x86_64, description=Ceph Grafana Container, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., maintainer=Paul Cuzner ) 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 bash[66390]: 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 systemd[1]: Started Ceph grafana.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.819992691Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.82018098Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.8202004Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-17T08:11:27.914 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.82020736Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820214061Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820221577Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820227655Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820233667Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820240589Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-17T08:11:27.915 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820246515Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820254008Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820259554Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820265634Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=settings t=2024-09-17T08:11:27.820271618Z level=info msg="App mode production" 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore t=2024-09-17T08:11:27.820318067Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore t=2024-09-17T08:11:27.820334388Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.821224916Z level=info msg="Starting DB migrations" 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.821297489Z level=info msg="Executing migration" id="create migration_log table" 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.821836566Z level=info msg="Executing migration" id="create user table" 2024-09-17T08:11:27.916 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.822295635Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.823112716Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.824012245Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.824693436Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.825665516Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.826490163Z level=info msg="Executing migration" id="create user table v2" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.827355927Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.828125892Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.828954779Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.829330187Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.830167131Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.830610264Z level=info msg="Executing migration" id="Update user table charset" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.830947862Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-17T08:11:27.917 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.831373549Z level=info msg="Executing migration" id="Add missing user data" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.831785692Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.832659346Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.833501055Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.833926921Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.835057193Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.838624015Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.839106844Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.841920099Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.844586498Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.847477372Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.850093295Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.850485144Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-17T08:11:27.918 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.853205414Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.853605285Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.85591884Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.856593163Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.858541552Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.858958683Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.859339067Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.862029046Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.862425509Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.864680716Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.865230224Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.867330667Z level=info msg="Executing migration" id="create star table" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.869973963Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-17T08:11:27.919 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.872528725Z level=info msg="Executing migration" id="create org table v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.875171629Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.875566794Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.87823904Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.880650006Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.883326334Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.886016851Z level=info msg="Executing migration" id="Update org table charset" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.888615094Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.891215269Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.891675915Z level=info msg="Executing migration" id="create dashboard table" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.894443702Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.897125911Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.899757939Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.900265794Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-17T08:11:27.920 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.90303988Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.905741288Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.906708856Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.907126237Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.907941824Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.908733513Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.909509296Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.909914579Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.910670815Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.911447868Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.91226268Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.91276408Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.913539585Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-17T08:11:27.921 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.914344459Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-17T08:11:27.922 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.915157058Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-17T08:11:27.922 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.915692825Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-17T08:11:27.922 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.916448729Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.917294757Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.918093737Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.918619243Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.919419205Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.920266004Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.921110013Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.921573341Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-17T08:11:28.165 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.922343096Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.92267192Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.923156366Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.923958251Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.924888319Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.925713262Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.92649254Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.927387217Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.927732633Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.928514862Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.92932142Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.930144311Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.930534151Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-17T08:11:28.166 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.933138994Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.93376389Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.935854313Z level=info msg="Executing migration" id="create data_source table" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.938400227Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.938851628Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.941669419Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.944156555Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.946994789Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.94811677Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.950860137Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.953502041Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.956280178Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.956715677Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.959380213Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-17T08:11:28.167 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.962035654Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-17T08:11:28.168 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.964850253Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-17T08:11:28.168 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.965262733Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-17T08:11:28.168 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.968172407Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-17T08:11:28.169 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.968541575Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-17T08:11:28.169 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.970795856Z level=info msg="Executing migration" id="Add uid column" 2024-09-17T08:11:28.169 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.971579997Z level=info msg="Executing migration" id="Update uid value" 2024-09-17T08:11:28.169 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.973377091Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-17T08:11:28.169 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.973955546Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.97441896Z level=info msg="Executing migration" id="create api_key table" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.977223973Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.97974098Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.982097189Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.982618594Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.985177531Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.988010067Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.990757295Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.993368296Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-17T08:11:28.170 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.995910692Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:27.998625541Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.001289957Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.001979815Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.004518351Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.004974315Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.005779232Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.006634087Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.007447368Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.008255876Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-17T08:11:28.171 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.009114102Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.009984439Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.010792972Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.011765762Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.01247726Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.013567186Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.014180186Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.015174512Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.015869156Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.016753877Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.017609622Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.018425258Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.019287292Z level=info msg="Executing migration" id="create quota table v1" 2024-09-17T08:11:28.172 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.020173385Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.021103373Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.021859571Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.022267693Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.023251397Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.024024942Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.024910702Z level=info msg="Executing migration" id="create session table" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.025357759Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.02566548Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.025972756Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.026439956Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.02732709Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-17T08:11:28.173 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.028174976Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.028560885Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.028854921Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.029187815Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.029628893Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.029929019Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.030616956Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.030944835Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.031619544Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.032532889Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.033297141Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.033785115Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.034643026Z level=info msg="Executing migration" id="create alert table v1" 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.035482867Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-17T08:11:28.174 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.036463639Z level=info msg="Executing migration" id="add index alert state" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.037188938Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.037639184Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.038041897Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.038511919Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.03892814Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.040394942Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.040823639Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.041309247Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.041631149Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.04200182Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.042463235Z level=info msg="Executing migration" id="Add column is_default" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.043190894Z level=info msg="Executing migration" id="Add column frequency" 2024-09-17T08:11:28.175 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.043884932Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.044589643Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.045264388Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.045739054Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.046063677Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.046412778Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.046904819Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.047400705Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.047778177Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.048201337Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.048672418Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.049376804Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.05003147Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.050389126Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T08:11:28.176 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.050862344Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.051341311Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.052037104Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.052386758Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.052803521Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.05327613Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.053636121Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.054118453Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.054592333Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.055051567Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.055498381Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.056007561Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.058977764Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T08:11:28.177 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.06172208Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.062486218Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.065473872Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.068074837Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.070692894Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.073709563Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.076379803Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.07902908Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.08199705Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.084801123Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.087580983Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.091148896Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.093763311Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.096601531Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T08:11:28.178 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.099339285Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.102141392Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.105004185Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.107872922Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.110509341Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.113139628Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.115987728Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.118594812Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.121444678Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.12400333Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.126801365Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T08:11:28.179 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.129478254Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.132239489Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.134786699Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.137533961Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.140248578Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.140686905Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.143407953Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.145908813Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.148575602Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.151318649Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.154026729Z level=info msg="Executing migration" id="create team table" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.154752395Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.155710626Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T08:11:28.180 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.156459584Z level=info msg="Executing migration" id="create team member table" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.157410167Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.15803157Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.159063651Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.159670328Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.160524651Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.161362301Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.162234036Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.163066959Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.164075775Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.164728381Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.165194988Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.166063641Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.166900983Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T08:11:28.181 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.169267823Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:11:28.416 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: Deploying daemon node-exporter.a on smithi079 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:11:28.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:11:28.418 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.171960976Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T08:11:28.418 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.172741981Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T08:11:28.418 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.173245977Z level=info msg="Executing migration" id="create tag table" 2024-09-17T08:11:28.418 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.174060806Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T08:11:28.418 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.174905865Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.175691501Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.176188023Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.17696315Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.17884025Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.179643082Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.180123486Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.18091682Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.181709933Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.182481924Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.182946104Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.183713748Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T08:11:28.419 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.18454975Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.185539521Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.18628317Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.187106648Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.187912058Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.18869963Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.189130013Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.190186286Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.190769495Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.191751679Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.192384932Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.193288534Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.194310345Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.195166727Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T08:11:28.420 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.195847371Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.196570224Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.197249751Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.198079863Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.199031805Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.199905423Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.202508878Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.205203523Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.205605539Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.208449647Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.211201439Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.213980953Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.214432431Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T08:11:28.421 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.217159593Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.219917444Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.222650782Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.22307453Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.226005901Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.228870653Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.231518053Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.232096369Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.23499093Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.23794299Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.240756911Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.241714645Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.2445684Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T08:11:28.422 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.247118778Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.250632873Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.25400519Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.256481742Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.259192595Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.261900245Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.262340634Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.264999466Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.267684962Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.270318388Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.270815651Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.273550843Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.276001626Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.278731776Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T08:11:28.423 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.281414839Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.282058068Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.28305216Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.284098133Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.28458849Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.285382513Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.286186198Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.286976064Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.287448379Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.288244383Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.289048789Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.28994609Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.290968548Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T08:11:28.424 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.291938796Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.29274564Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.293565293Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.294426604Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.295224031Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.296034609Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.296860163Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.297161412Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.298053213Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.299024197Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.299961772Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.300497686Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.301278778Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T08:11:28.425 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.302275754Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.30304748Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.303503346Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.304432046Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.305196343Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.306129384Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.306721391Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.307499711Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.30760872Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.308315808Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.309166267Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.30961191Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.309738413Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-17T08:11:28.426 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.309855765Z level=info msg="alerts found to migrate" alerts=0 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.312258193Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.315019803Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.317737506Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.31816117Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.320850317Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.323527636Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.326277962Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.326761251Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.329346457Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.331862252Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.336224591Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.337520719Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.340140446Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T08:11:28.427 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.344390163Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T08:11:28.428 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.348582698Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T08:11:28.428 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.349106295Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T08:11:28.428 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.351952357Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T08:11:28.428 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.354654674Z level=info msg="Executing migration" id="create permission table" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.357255643Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.357854494Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.358733956Z level=info msg="Executing migration" id="create role table" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.359692865Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.360667323Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.36160342Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.362470005Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.363482386Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T08:11:28.429 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.364219247Z level=info msg="Executing migration" id="create team role table" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.364661198Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.36552706Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.366382091Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.367256294Z level=info msg="Executing migration" id="create user role table" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.368090227Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.369097334Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.369785332Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.370746226Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.371287355Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.37215037Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.372937471Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.373882962Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.375700817Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T08:11:28.430 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.378423059Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.378854985Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.379297028Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.379670175Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.380100121Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.381037603Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.381431095Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.381856743Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.382191626Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.382564414Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.383024816Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.38338662Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.383674791Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T08:11:28.431 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.384023994Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.38440538Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.384824972Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.38577062Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.386074119Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.386477087Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.386911782Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.387337578Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.388308007Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.388733622Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.391567444Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.391672742Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.39197545Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:11:28.432 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.392058073Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.39232706Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.392678775Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.393108615Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.393537005Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.393973747Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.394379327Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.39477676Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.395183697Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.395618999Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.396043938Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.39648857Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.399409849Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.402583511Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T08:11:28.433 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.403586312Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.404585661Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.404941949Z level=info msg="Executing migration" id="add share column" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.40602831Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.406456597Z level=info msg="Executing migration" id="create file table" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.406889348Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.409808678Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.410361006Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.410758991Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.413576907Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.416370135Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T08:11:28.434 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.419262149Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.607 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: Deploying daemon node-exporter.a on smithi079 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:11:28.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: Deploying daemon node-exporter.a on smithi079 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:11:28.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:28 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:11:28.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.422036431Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T08:11:28.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.424952474Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T08:11:28.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.427613312Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T08:11:28.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.430526598Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.433193189Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.435870333Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.438719418Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.441485014Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.444121336Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.446962934Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.449515082Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.459428134Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.462337794Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.465138101Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.468592518Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.471122501Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T08:11:28.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.473960662Z level=info msg="Executing migration" id="create folder table" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.476606117Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.479209125Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.482049558Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.484774227Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=migrator t=2024-09-17T08:11:28.487389526Z level=info msg="migrations completed" performed=485 skipped=0 duration=666.093306ms 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore t=2024-09-17T08:11:28.487806542Z level=info msg="Created default organization" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=plugin.loader t=2024-09-17T08:11:28.548476361Z level=info msg="Plugin registered" pluginID=input 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=plugin.loader t=2024-09-17T08:11:28.576169798Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=plugin.loader t=2024-09-17T08:11:28.576190542Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=secrets t=2024-09-17T08:11:28.576225353Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=query_data t=2024-09-17T08:11:28.578017187Z level=info msg="Query Service initialization" 2024-09-17T08:11:28.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=live.push_http t=2024-09-17T08:11:28.580327317Z level=info msg="Live Push Gateway initialization" 2024-09-17T08:11:28.869 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:28 smithi079 systemd[1]: Starting Ceph node-exporter.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:28.891 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:29.219 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:28 smithi079 bash[77546]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:11:29.474 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:11:29.474 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:11:29.474 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[40649]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:11:29.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:11:29.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:29.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:29 smithi079 ceph-mon[29565]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-mon[33187]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=infra.usagestats.collector t=2024-09-17T08:11:29.304709243Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=provisioning.datasources t=2024-09-17T08:11:29.360190159Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=provisioning.datasources t=2024-09-17T08:11:29.391955708Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=provisioning.alerting t=2024-09-17T08:11:29.414511678Z level=info msg="starting to provision alerting" 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=provisioning.alerting t=2024-09-17T08:11:29.414534529Z level=info msg="finished to provision alerting" 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=ngalert.state.manager t=2024-09-17T08:11:29.414767203Z level=info msg="Warming state cache for startup" 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=grafanaStorageLogger t=2024-09-17T08:11:29.414801399Z level=info msg="storage starting" 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=http.server t=2024-09-17T08:11:29.416650533Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T08:11:29.725 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=ngalert.state.manager t=2024-09-17T08:11:29.424973875Z level=info msg="State cache has been initialized" states=0 duration=10.205835ms 2024-09-17T08:11:29.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=ticker t=2024-09-17T08:11:29.425037791Z level=info msg=starting first_tick=2024-09-17T08:11:30Z 2024-09-17T08:11:29.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=ngalert.multiorg.alertmanager t=2024-09-17T08:11:29.425039494Z level=info msg="starting MultiOrg Alertmanager" 2024-09-17T08:11:29.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore.transactions t=2024-09-17T08:11:29.425041169Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:11:29.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore.transactions t=2024-09-17T08:11:29.479164019Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:11:29.726 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:11:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore.transactions t=2024-09-17T08:11:29.489505922Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-17T08:11:29.729 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:29 smithi079 bash[77546]: Getting image source signatures 2024-09-17T08:11:29.730 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:29 smithi079 bash[77546]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:11:29.730 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:29 smithi079 bash[77546]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:11:29.730 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:29 smithi079 bash[77546]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:11:29.751 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-17T08:11:27.222253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1008,"num_read_kb":854,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":611,"ondisk_log_size":611,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239776,"kb_used_data":7092,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749721440,"statfs":{"total":767960285184,"available":767714754560,"internally_reserved":0,"allocated":7262208,"data_stored":4353930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002213"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351985+0000","last_change":"2024-09-17T08:10:00.599431+0000","last_active":"2024-09-17T08:11:11.351985+0000","last_peered":"2024-09-17T08:11:11.351985+0000","last_clean":"2024-09-17T08:11:11.351985+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:11.351985+0000","last_undegraded":"2024-09-17T08:11:11.351985+0000","last_fullsized":"2024-09-17T08:11:11.351985+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155306+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:11.155306+0000","last_peered":"2024-09-17T08:11:11.155306+0000","last_clean":"2024-09-17T08:11:11.155306+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:11.155306+0000","last_undegraded":"2024-09-17T08:11:11.155306+0000","last_fullsized":"2024-09-17T08:11:11.155306+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":53,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351921+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:11.351921+0000","last_peered":"2024-09-17T08:11:11.351921+0000","last_clean":"2024-09-17T08:11:11.351921+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:11.351921+0000","last_undegraded":"2024-09-17T08:11:11.351921+0000","last_fullsized":"2024-09-17T08:11:11.351921+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.631177+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:13.631177+0000","last_peered":"2024-09-17T08:11:13.631177+0000","last_clean":"2024-09-17T08:11:13.631177+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:13.631177+0000","last_undegraded":"2024-09-17T08:11:13.631177+0000","last_fullsized":"2024-09-17T08:11:13.631177+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352963+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:11.352963+0000","last_peered":"2024-09-17T08:11:11.352963+0000","last_clean":"2024-09-17T08:11:11.352963+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:11.352963+0000","last_undegraded":"2024-09-17T08:11:11.352963+0000","last_fullsized":"2024-09-17T08:11:11.352963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"72'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154125+0000","last_change":"2024-09-17T08:09:56.500998+0000","last_active":"2024-09-17T08:11:11.154125+0000","last_peered":"2024-09-17T08:11:11.154125+0000","last_clean":"2024-09-17T08:11:11.154125+0000","last_became_active":"2024-09-17T08:09:56.500692+0000","last_became_peered":"2024-09-17T08:09:56.500692+0000","last_unstale":"2024-09-17T08:11:11.154125+0000","last_undegraded":"2024-09-17T08:11:11.154125+0000","last_fullsized":"2024-09-17T08:11:11.154125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.970578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154461+0000","last_change":"2024-09-17T08:09:58.507646+0000","last_active":"2024-09-17T08:11:11.154461+0000","last_peered":"2024-09-17T08:11:11.154461+0000","last_clean":"2024-09-17T08:11:11.154461+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:11.154461+0000","last_undegraded":"2024-09-17T08:11:11.154461+0000","last_fullsized":"2024-09-17T08:11:11.154461+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154288+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:11.154288+0000","last_peered":"2024-09-17T08:11:11.154288+0000","last_clean":"2024-09-17T08:11:11.154288+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:11.154288+0000","last_undegraded":"2024-09-17T08:11:11.154288+0000","last_fullsized":"2024-09-17T08:11:11.154288+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.381486+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:11:22.381486+0000","last_peered":"2024-09-17T08:11:22.381486+0000","last_clean":"2024-09-17T08:11:22.381486+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:22.381486+0000","last_undegraded":"2024-09-17T08:11:22.381486+0000","last_fullsized":"2024-09-17T08:11:22.381486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"72'5","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154443+0000","last_change":"2024-09-17T08:09:56.503460+0000","last_active":"2024-09-17T08:11:11.154443+0000","last_peered":"2024-09-17T08:11:11.154443+0000","last_clean":"2024-09-17T08:11:11.154443+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:11.154443+0000","last_undegraded":"2024-09-17T08:11:11.154443+0000","last_fullsized":"2024-09-17T08:11:11.154443+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290211+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:11.290211+0000","last_peered":"2024-09-17T08:11:11.290211+0000","last_clean":"2024-09-17T08:11:11.290211+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:11.290211+0000","last_undegraded":"2024-09-17T08:11:11.290211+0000","last_fullsized":"2024-09-17T08:11:11.290211+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.216438+0000","last_change":"2024-09-17T08:10:00.517064+0000","last_active":"2024-09-17T08:11:11.216438+0000","last_peered":"2024-09-17T08:11:11.216438+0000","last_clean":"2024-09-17T08:11:11.216438+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:11.216438+0000","last_undegraded":"2024-09-17T08:11:11.216438+0000","last_fullsized":"2024-09-17T08:11:11.216438+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:24.078122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290981+0000","last_change":"2024-09-17T08:09:54.502336+0000","last_active":"2024-09-17T08:11:11.290981+0000","last_peered":"2024-09-17T08:11:11.290981+0000","last_clean":"2024-09-17T08:11:11.290981+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:11.290981+0000","last_undegraded":"2024-09-17T08:11:11.290981+0000","last_fullsized":"2024-09-17T08:11:11.290981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154477+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:11.154477+0000","last_peered":"2024-09-17T08:11:11.154477+0000","last_clean":"2024-09-17T08:11:11.154477+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:11.154477+0000","last_undegraded":"2024-09-17T08:11:11.154477+0000","last_fullsized":"2024-09-17T08:11:11.154477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:18.175153+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:11:18.175153+0000","last_peered":"2024-09-17T08:11:18.175153+0000","last_clean":"2024-09-17T08:11:18.175153+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:18.175153+0000","last_undegraded":"2024-09-17T08:11:18.175153+0000","last_fullsized":"2024-09-17T08:11:18.175153+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.041377+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:11:21.041377+0000","last_peered":"2024-09-17T08:11:21.041377+0000","last_clean":"2024-09-17T08:11:21.041377+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:21.041377+0000","last_undegraded":"2024-09-17T08:11:21.041377+0000","last_fullsized":"2024-09-17T08:11:21.041377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154821+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:11.154821+0000","last_peered":"2024-09-17T08:11:11.154821+0000","last_clean":"2024-09-17T08:11:11.154821+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:11.154821+0000","last_undegraded":"2024-09-17T08:11:11.154821+0000","last_fullsized":"2024-09-17T08:11:11.154821+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:18.510464+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:11:18.510464+0000","last_peered":"2024-09-17T08:11:18.510464+0000","last_clean":"2024-09-17T08:11:18.510464+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:18.510464+0000","last_undegraded":"2024-09-17T08:11:18.510464+0000","last_fullsized":"2024-09-17T08:11:18.510464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154752+0000","last_change":"2024-09-17T08:09:58.506645+0000","last_active":"2024-09-17T08:11:11.154752+0000","last_peered":"2024-09-17T08:11:11.154752+0000","last_clean":"2024-09-17T08:11:11.154752+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:11.154752+0000","last_undegraded":"2024-09-17T08:11:11.154752+0000","last_fullsized":"2024-09-17T08:11:11.154752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352721+0000","last_change":"2024-09-17T08:10:00.521440+0000","last_active":"2024-09-17T08:11:11.352721+0000","last_peered":"2024-09-17T08:11:11.352721+0000","last_clean":"2024-09-17T08:11:11.352721+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:11.352721+0000","last_undegraded":"2024-09-17T08:11:11.352721+0000","last_fullsized":"2024-09-17T08:11:11.352721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154684+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:11.154684+0000","last_peered":"2024-09-17T08:11:11.154684+0000","last_clean":"2024-09-17T08:11:11.154684+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:11.154684+0000","last_undegraded":"2024-09-17T08:11:11.154684+0000","last_fullsized":"2024-09-17T08:11:11.154684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155071+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:11.155071+0000","last_peered":"2024-09-17T08:11:11.155071+0000","last_clean":"2024-09-17T08:11:11.155071+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:11.155071+0000","last_undegraded":"2024-09-17T08:11:11.155071+0000","last_fullsized":"2024-09-17T08:11:11.155071+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777499+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:11:21.777499+0000","last_peered":"2024-09-17T08:11:21.777499+0000","last_clean":"2024-09-17T08:11:21.777499+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:21.777499+0000","last_undegraded":"2024-09-17T08:11:21.777499+0000","last_fullsized":"2024-09-17T08:11:21.777499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:20.492012+0000","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154850+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:11.154850+0000","last_peered":"2024-09-17T08:11:11.154850+0000","last_clean":"2024-09-17T08:11:11.154850+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:11.154850+0000","last_undegraded":"2024-09-17T08:11:11.154850+0000","last_fullsized":"2024-09-17T08:11:11.154850+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.558426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.970059+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:16.970059+0000","last_peered":"2024-09-17T08:11:16.970059+0000","last_clean":"2024-09-17T08:11:16.970059+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:16.970059+0000","last_undegraded":"2024-09-17T08:11:16.970059+0000","last_fullsized":"2024-09-17T08:11:16.970059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154672+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:11.154672+0000","last_peered":"2024-09-17T08:11:11.154672+0000","last_clean":"2024-09-17T08:11:11.154672+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:11.154672+0000","last_undegraded":"2024-09-17T08:11:11.154672+0000","last_fullsized":"2024-09-17T08:11:11.154672+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.358620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779387+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:11:21.779387+0000","last_peered":"2024-09-17T08:11:21.779387+0000","last_clean":"2024-09-17T08:11:21.779387+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:21.779387+0000","last_undegraded":"2024-09-17T08:11:21.779387+0000","last_fullsized":"2024-09-17T08:11:21.779387+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.125976+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:11:27.125976+0000","last_peered":"2024-09-17T08:11:27.125976+0000","last_clean":"2024-09-17T08:11:27.125976+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:27.125976+0000","last_undegraded":"2024-09-17T08:11:27.125976+0000","last_fullsized":"2024-09-17T08:11:27.125976+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.618306+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:11:25.618306+0000","last_peered":"2024-09-17T08:11:25.618306+0000","last_clean":"2024-09-17T08:11:25.618306+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:25.618306+0000","last_undegraded":"2024-09-17T08:11:25.618306+0000","last_fullsized":"2024-09-17T08:11:25.618306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290415+0000","last_change":"2024-09-17T08:09:54.506854+0000","last_active":"2024-09-17T08:11:11.290415+0000","last_peered":"2024-09-17T08:11:11.290415+0000","last_clean":"2024-09-17T08:11:11.290415+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:11.290415+0000","last_undegraded":"2024-09-17T08:11:11.290415+0000","last_fullsized":"2024-09-17T08:11:11.290415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.217162+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:11.217162+0000","last_peered":"2024-09-17T08:11:11.217162+0000","last_clean":"2024-09-17T08:11:11.217162+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:11.217162+0000","last_undegraded":"2024-09-17T08:11:11.217162+0000","last_fullsized":"2024-09-17T08:11:11.217162+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352475+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:11.352475+0000","last_peered":"2024-09-17T08:11:11.352475+0000","last_clean":"2024-09-17T08:11:11.352475+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:11.352475+0000","last_undegraded":"2024-09-17T08:11:11.352475+0000","last_fullsized":"2024-09-17T08:11:11.352475+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154245+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:11.154245+0000","last_peered":"2024-09-17T08:11:11.154245+0000","last_clean":"2024-09-17T08:11:11.154245+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:11.154245+0000","last_undegraded":"2024-09-17T08:11:11.154245+0000","last_fullsized":"2024-09-17T08:11:11.154245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288978+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:11.288978+0000","last_peered":"2024-09-17T08:11:11.288978+0000","last_clean":"2024-09-17T08:11:11.288978+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:11.288978+0000","last_undegraded":"2024-09-17T08:11:11.288978+0000","last_fullsized":"2024-09-17T08:11:11.288978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384682+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:21.384682+0000","last_peered":"2024-09-17T08:11:21.384682+0000","last_clean":"2024-09-17T08:11:21.384682+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:21.384682+0000","last_undegraded":"2024-09-17T08:11:21.384682+0000","last_fullsized":"2024-09-17T08:11:21.384682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.469761+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:11.469761+0000","last_peered":"2024-09-17T08:11:11.469761+0000","last_clean":"2024-09-17T08:11:11.469761+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:11.469761+0000","last_undegraded":"2024-09-17T08:11:11.469761+0000","last_fullsized":"2024-09-17T08:11:11.469761+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351776+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:11.351776+0000","last_peered":"2024-09-17T08:11:11.351776+0000","last_clean":"2024-09-17T08:11:11.351776+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:11.351776+0000","last_undegraded":"2024-09-17T08:11:11.351776+0000","last_fullsized":"2024-09-17T08:11:11.351776+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154163+0000","last_change":"2024-09-17T08:09:54.498648+0000","last_active":"2024-09-17T08:11:11.154163+0000","last_peered":"2024-09-17T08:11:11.154163+0000","last_clean":"2024-09-17T08:11:11.154163+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:11.154163+0000","last_undegraded":"2024-09-17T08:11:11.154163+0000","last_fullsized":"2024-09-17T08:11:11.154163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154161+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:11.154161+0000","last_peered":"2024-09-17T08:11:11.154161+0000","last_clean":"2024-09-17T08:11:11.154161+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:11.154161+0000","last_undegraded":"2024-09-17T08:11:11.154161+0000","last_fullsized":"2024-09-17T08:11:11.154161+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.957574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154396+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:11.154396+0000","last_peered":"2024-09-17T08:11:11.154396+0000","last_clean":"2024-09-17T08:11:11.154396+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:11.154396+0000","last_undegraded":"2024-09-17T08:11:11.154396+0000","last_fullsized":"2024-09-17T08:11:11.154396+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.596565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":75,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155463+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:11.155463+0000","last_peered":"2024-09-17T08:11:11.155463+0000","last_clean":"2024-09-17T08:11:11.155463+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:11.155463+0000","last_undegraded":"2024-09-17T08:11:11.155463+0000","last_fullsized":"2024-09-17T08:11:11.155463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.291199+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:11.291199+0000","last_peered":"2024-09-17T08:11:11.291199+0000","last_clean":"2024-09-17T08:11:11.291199+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:11.291199+0000","last_undegraded":"2024-09-17T08:11:11.291199+0000","last_fullsized":"2024-09-17T08:11:11.291199+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288308+0000","last_change":"2024-09-17T08:09:58.506025+0000","last_active":"2024-09-17T08:11:11.288308+0000","last_peered":"2024-09-17T08:11:11.288308+0000","last_clean":"2024-09-17T08:11:11.288308+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:11.288308+0000","last_undegraded":"2024-09-17T08:11:11.288308+0000","last_fullsized":"2024-09-17T08:11:11.288308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:25.291171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.153714+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:11.153714+0000","last_peered":"2024-09-17T08:11:11.153714+0000","last_clean":"2024-09-17T08:11:11.153714+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:11.153714+0000","last_undegraded":"2024-09-17T08:11:11.153714+0000","last_fullsized":"2024-09-17T08:11:11.153714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:17.921084+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:11:17.921084+0000","last_peered":"2024-09-17T08:11:17.921084+0000","last_clean":"2024-09-17T08:11:17.921084+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:17.921084+0000","last_undegraded":"2024-09-17T08:11:17.921084+0000","last_fullsized":"2024-09-17T08:11:17.921084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:18.904435+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:11:18.904435+0000","last_peered":"2024-09-17T08:11:18.904435+0000","last_clean":"2024-09-17T08:11:18.904435+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:18.904435+0000","last_undegraded":"2024-09-17T08:11:18.904435+0000","last_fullsized":"2024-09-17T08:11:18.904435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.429946+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:21.429946+0000","last_peered":"2024-09-17T08:11:21.429946+0000","last_clean":"2024-09-17T08:11:21.429946+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:21.429946+0000","last_undegraded":"2024-09-17T08:11:21.429946+0000","last_fullsized":"2024-09-17T08:11:21.429946+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154214+0000","last_change":"2024-09-17T08:10:00.516499+0000","last_active":"2024-09-17T08:11:11.154214+0000","last_peered":"2024-09-17T08:11:11.154214+0000","last_clean":"2024-09-17T08:11:11.154214+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:11.154214+0000","last_undegraded":"2024-09-17T08:11:11.154214+0000","last_fullsized":"2024-09-17T08:11:11.154214+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.391480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385221+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:21.385221+0000","last_peered":"2024-09-17T08:11:21.385221+0000","last_clean":"2024-09-17T08:11:21.385221+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:21.385221+0000","last_undegraded":"2024-09-17T08:11:21.385221+0000","last_fullsized":"2024-09-17T08:11:21.385221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.216726+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:11.216726+0000","last_peered":"2024-09-17T08:11:11.216726+0000","last_clean":"2024-09-17T08:11:11.216726+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:11.216726+0000","last_undegraded":"2024-09-17T08:11:11.216726+0000","last_fullsized":"2024-09-17T08:11:11.216726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155269+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:11.155269+0000","last_peered":"2024-09-17T08:11:11.155269+0000","last_clean":"2024-09-17T08:11:11.155269+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:11.155269+0000","last_undegraded":"2024-09-17T08:11:11.155269+0000","last_fullsized":"2024-09-17T08:11:11.155269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352301+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:11.352301+0000","last_peered":"2024-09-17T08:11:11.352301+0000","last_clean":"2024-09-17T08:11:11.352301+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:11.352301+0000","last_undegraded":"2024-09-17T08:11:11.352301+0000","last_fullsized":"2024-09-17T08:11:11.352301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154507+0000","last_change":"2024-09-17T08:09:56.503791+0000","last_active":"2024-09-17T08:11:11.154507+0000","last_peered":"2024-09-17T08:11:11.154507+0000","last_clean":"2024-09-17T08:11:11.154507+0000","last_became_active":"2024-09-17T08:09:56.503476+0000","last_became_peered":"2024-09-17T08:09:56.503476+0000","last_unstale":"2024-09-17T08:11:11.154507+0000","last_undegraded":"2024-09-17T08:11:11.154507+0000","last_fullsized":"2024-09-17T08:11:11.154507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.579571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":135,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.170736+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:26.170736+0000","last_peered":"2024-09-17T08:11:26.170736+0000","last_clean":"2024-09-17T08:11:26.170736+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:26.170736+0000","last_undegraded":"2024-09-17T08:11:26.170736+0000","last_fullsized":"2024-09-17T08:11:26.170736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":86,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.081903+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:11:23.081903+0000","last_peered":"2024-09-17T08:11:23.081903+0000","last_clean":"2024-09-17T08:11:23.081903+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:23.081903+0000","last_undegraded":"2024-09-17T08:11:23.081903+0000","last_fullsized":"2024-09-17T08:11:23.081903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.110925+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:11:27.110925+0000","last_peered":"2024-09-17T08:11:27.110925+0000","last_clean":"2024-09-17T08:11:27.110925+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:27.110925+0000","last_undegraded":"2024-09-17T08:11:27.110925+0000","last_fullsized":"2024-09-17T08:11:27.110925+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:20.454612+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:11:20.454612+0000","last_peered":"2024-09-17T08:11:20.454612+0000","last_clean":"2024-09-17T08:11:20.454612+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:20.454612+0000","last_undegraded":"2024-09-17T08:11:20.454612+0000","last_fullsized":"2024-09-17T08:11:20.454612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.895771+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:11.895771+0000","last_peered":"2024-09-17T08:11:11.895771+0000","last_clean":"2024-09-17T08:11:11.895771+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:11.895771+0000","last_undegraded":"2024-09-17T08:11:11.895771+0000","last_fullsized":"2024-09-17T08:11:11.895771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154961+0000","last_change":"2024-09-17T08:09:58.498654+0000","last_active":"2024-09-17T08:11:11.154961+0000","last_peered":"2024-09-17T08:11:11.154961+0000","last_clean":"2024-09-17T08:11:11.154961+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:11.154961+0000","last_undegraded":"2024-09-17T08:11:11.154961+0000","last_fullsized":"2024-09-17T08:11:11.154961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.615615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290509+0000","last_change":"2024-09-17T08:09:54.507600+0000","last_active":"2024-09-17T08:11:11.290509+0000","last_peered":"2024-09-17T08:11:11.290509+0000","last_clean":"2024-09-17T08:11:11.290509+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:11.290509+0000","last_undegraded":"2024-09-17T08:11:11.290509+0000","last_fullsized":"2024-09-17T08:11:11.290509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.148550+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:11:21.148550+0000","last_peered":"2024-09-17T08:11:21.148550+0000","last_clean":"2024-09-17T08:11:21.148550+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:21.148550+0000","last_undegraded":"2024-09-17T08:11:21.148550+0000","last_fullsized":"2024-09-17T08:11:21.148550+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154675+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:11.154675+0000","last_peered":"2024-09-17T08:11:11.154675+0000","last_clean":"2024-09-17T08:11:11.154675+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:11.154675+0000","last_undegraded":"2024-09-17T08:11:11.154675+0000","last_fullsized":"2024-09-17T08:11:11.154675+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.624827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.289842+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:11.289842+0000","last_peered":"2024-09-17T08:11:11.289842+0000","last_clean":"2024-09-17T08:11:11.289842+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:11.289842+0000","last_undegraded":"2024-09-17T08:11:11.289842+0000","last_fullsized":"2024-09-17T08:11:11.289842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351963+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:11.351963+0000","last_peered":"2024-09-17T08:11:11.351963+0000","last_clean":"2024-09-17T08:11:11.351963+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:11.351963+0000","last_undegraded":"2024-09-17T08:11:11.351963+0000","last_fullsized":"2024-09-17T08:11:11.351963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:19.154226+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:11:19.154226+0000","last_peered":"2024-09-17T08:11:19.154226+0000","last_clean":"2024-09-17T08:11:19.154226+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:19.154226+0000","last_undegraded":"2024-09-17T08:11:19.154226+0000","last_fullsized":"2024-09-17T08:11:19.154226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":291,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290628+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:11.290628+0000","last_peered":"2024-09-17T08:11:11.290628+0000","last_clean":"2024-09-17T08:11:11.290628+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:11.290628+0000","last_undegraded":"2024-09-17T08:11:11.290628+0000","last_fullsized":"2024-09-17T08:11:11.290628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290809+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:11.290809+0000","last_peered":"2024-09-17T08:11:11.290809+0000","last_clean":"2024-09-17T08:11:11.290809+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:11.290809+0000","last_undegraded":"2024-09-17T08:11:11.290809+0000","last_fullsized":"2024-09-17T08:11:11.290809+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.217272+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:11.217272+0000","last_peered":"2024-09-17T08:11:11.217272+0000","last_clean":"2024-09-17T08:11:11.217272+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:11.217272+0000","last_undegraded":"2024-09-17T08:11:11.217272+0000","last_fullsized":"2024-09-17T08:11:11.217272+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288560+0000","last_change":"2024-09-17T08:09:57.041231+0000","last_active":"2024-09-17T08:11:11.288560+0000","last_peered":"2024-09-17T08:11:11.288560+0000","last_clean":"2024-09-17T08:11:11.288560+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:11.288560+0000","last_undegraded":"2024-09-17T08:11:11.288560+0000","last_fullsized":"2024-09-17T08:11:11.288560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:23.855672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352270+0000","last_change":"2024-09-17T08:09:54.502363+0000","last_active":"2024-09-17T08:11:11.352270+0000","last_peered":"2024-09-17T08:11:11.352270+0000","last_clean":"2024-09-17T08:11:11.352270+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:11.352270+0000","last_undegraded":"2024-09-17T08:11:11.352270+0000","last_fullsized":"2024-09-17T08:11:11.352270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.148208+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:15.148208+0000","last_peered":"2024-09-17T08:11:15.148208+0000","last_clean":"2024-09-17T08:11:15.148208+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:15.148208+0000","last_undegraded":"2024-09-17T08:11:15.148208+0000","last_fullsized":"2024-09-17T08:11:15.148208+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290562+0000","last_change":"2024-09-17T08:10:00.525443+0000","last_active":"2024-09-17T08:11:11.290562+0000","last_peered":"2024-09-17T08:11:11.290562+0000","last_clean":"2024-09-17T08:11:11.290562+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:11.290562+0000","last_undegraded":"2024-09-17T08:11:11.290562+0000","last_fullsized":"2024-09-17T08:11:11.290562+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155379+0000","last_change":"2024-09-17T08:09:56.506519+0000","last_active":"2024-09-17T08:11:11.155379+0000","last_peered":"2024-09-17T08:11:11.155379+0000","last_clean":"2024-09-17T08:11:11.155379+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:11.155379+0000","last_undegraded":"2024-09-17T08:11:11.155379+0000","last_fullsized":"2024-09-17T08:11:11.155379+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:22.648151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:20.897086+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:11:20.897086+0000","last_peered":"2024-09-17T08:11:20.897086+0000","last_clean":"2024-09-17T08:11:20.897086+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:20.897086+0000","last_undegraded":"2024-09-17T08:11:20.897086+0000","last_fullsized":"2024-09-17T08:11:20.897086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.124566+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:11:25.124566+0000","last_peered":"2024-09-17T08:11:25.124566+0000","last_clean":"2024-09-17T08:11:25.124566+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:25.124566+0000","last_undegraded":"2024-09-17T08:11:25.124566+0000","last_fullsized":"2024-09-17T08:11:25.124566+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.073461+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:11:23.073461+0000","last_peered":"2024-09-17T08:11:23.073461+0000","last_clean":"2024-09-17T08:11:23.073461+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:23.073461+0000","last_undegraded":"2024-09-17T08:11:23.073461+0000","last_fullsized":"2024-09-17T08:11:23.073461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.079065+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:11:23.079065+0000","last_peered":"2024-09-17T08:11:23.079065+0000","last_clean":"2024-09-17T08:11:23.079065+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:23.079065+0000","last_undegraded":"2024-09-17T08:11:23.079065+0000","last_fullsized":"2024-09-17T08:11:23.079065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352934+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:11.352934+0000","last_peered":"2024-09-17T08:11:11.352934+0000","last_clean":"2024-09-17T08:11:11.352934+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:11.352934+0000","last_undegraded":"2024-09-17T08:11:11.352934+0000","last_fullsized":"2024-09-17T08:11:11.352934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778273+0000","last_change":"2024-09-17T08:09:58.513726+0000","last_active":"2024-09-17T08:11:21.778273+0000","last_peered":"2024-09-17T08:11:21.778273+0000","last_clean":"2024-09-17T08:11:21.778273+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:21.778273+0000","last_undegraded":"2024-09-17T08:11:21.778273+0000","last_fullsized":"2024-09-17T08:11:21.778273+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.216450+0000","last_change":"2024-09-17T08:10:00.599030+0000","last_active":"2024-09-17T08:11:11.216450+0000","last_peered":"2024-09-17T08:11:11.216450+0000","last_clean":"2024-09-17T08:11:11.216450+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:11.216450+0000","last_undegraded":"2024-09-17T08:11:11.216450+0000","last_fullsized":"2024-09-17T08:11:11.216450+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:26.441216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154200+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:11.154200+0000","last_peered":"2024-09-17T08:11:11.154200+0000","last_clean":"2024-09-17T08:11:11.154200+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:11.154200+0000","last_undegraded":"2024-09-17T08:11:11.154200+0000","last_fullsized":"2024-09-17T08:11:11.154200+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155170+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:11.155170+0000","last_peered":"2024-09-17T08:11:11.155170+0000","last_clean":"2024-09-17T08:11:11.155170+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:11.155170+0000","last_undegraded":"2024-09-17T08:11:11.155170+0000","last_fullsized":"2024-09-17T08:11:11.155170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351449+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:11.351449+0000","last_peered":"2024-09-17T08:11:11.351449+0000","last_clean":"2024-09-17T08:11:11.351449+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:11.351449+0000","last_undegraded":"2024-09-17T08:11:11.351449+0000","last_fullsized":"2024-09-17T08:11:11.351449+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352273+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:11.352273+0000","last_peered":"2024-09-17T08:11:11.352273+0000","last_clean":"2024-09-17T08:11:11.352273+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:11.352273+0000","last_undegraded":"2024-09-17T08:11:11.352273+0000","last_fullsized":"2024-09-17T08:11:11.352273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.027135+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:11:22.027135+0000","last_peered":"2024-09-17T08:11:22.027135+0000","last_clean":"2024-09-17T08:11:22.027135+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:22.027135+0000","last_undegraded":"2024-09-17T08:11:22.027135+0000","last_fullsized":"2024-09-17T08:11:22.027135+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351467+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:11.351467+0000","last_peered":"2024-09-17T08:11:11.351467+0000","last_clean":"2024-09-17T08:11:11.351467+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:11.351467+0000","last_undegraded":"2024-09-17T08:11:11.351467+0000","last_fullsized":"2024-09-17T08:11:11.351467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352697+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:11.352697+0000","last_peered":"2024-09-17T08:11:11.352697+0000","last_clean":"2024-09-17T08:11:11.352697+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:11.352697+0000","last_undegraded":"2024-09-17T08:11:11.352697+0000","last_fullsized":"2024-09-17T08:11:11.352697+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:27.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.110687+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:14.110687+0000","last_peered":"2024-09-17T08:11:14.110687+0000","last_clean":"2024-09-17T08:11:14.110687+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:14.110687+0000","last_undegraded":"2024-09-17T08:11:14.110687+0000","last_fullsized":"2024-09-17T08:11:14.110687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.318902+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:11:21.318902+0000","last_peered":"2024-09-17T08:11:21.318902+0000","last_clean":"2024-09-17T08:11:21.318902+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:21.318902+0000","last_undegraded":"2024-09-17T08:11:21.318902+0000","last_fullsized":"2024-09-17T08:11:21.318902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.028634+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:11:23.028634+0000","last_peered":"2024-09-17T08:11:23.028634+0000","last_clean":"2024-09-17T08:11:23.028634+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:23.028634+0000","last_undegraded":"2024-09-17T08:11:23.028634+0000","last_fullsized":"2024-09-17T08:11:23.028634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.183764+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:12.183764+0000","last_peered":"2024-09-17T08:11:12.183764+0000","last_clean":"2024-09-17T08:11:12.183764+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:12.183764+0000","last_undegraded":"2024-09-17T08:11:12.183764+0000","last_fullsized":"2024-09-17T08:11:12.183764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290931+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:11.290931+0000","last_peered":"2024-09-17T08:11:11.290931+0000","last_clean":"2024-09-17T08:11:11.290931+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:11.290931+0000","last_undegraded":"2024-09-17T08:11:11.290931+0000","last_fullsized":"2024-09-17T08:11:11.290931+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352774+0000","last_change":"2024-09-17T08:09:57.041188+0000","last_active":"2024-09-17T08:11:11.352774+0000","last_peered":"2024-09-17T08:11:11.352774+0000","last_clean":"2024-09-17T08:11:11.352774+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:11.352774+0000","last_undegraded":"2024-09-17T08:11:11.352774+0000","last_fullsized":"2024-09-17T08:11:11.352774+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.216688+0000","last_change":"2024-09-17T08:10:54.331930+0000","last_active":"2024-09-17T08:11:11.216688+0000","last_peered":"2024-09-17T08:11:11.216688+0000","last_clean":"2024-09-17T08:11:11.216688+0000","last_became_active":"2024-09-17T08:09:54.501179+0000","last_became_peered":"2024-09-17T08:09:54.501179+0000","last_unstale":"2024-09-17T08:11:11.216688+0000","last_undegraded":"2024-09-17T08:11:11.216688+0000","last_fullsized":"2024-09-17T08:11:11.216688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.084651+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679248+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:21.679248+0000","last_peered":"2024-09-17T08:11:21.679248+0000","last_clean":"2024-09-17T08:11:21.679248+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:21.679248+0000","last_undegraded":"2024-09-17T08:11:21.679248+0000","last_fullsized":"2024-09-17T08:11:21.679248+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:24.057019+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:11:24.057019+0000","last_peered":"2024-09-17T08:11:24.057019+0000","last_clean":"2024-09-17T08:11:24.057019+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:24.057019+0000","last_undegraded":"2024-09-17T08:11:24.057019+0000","last_fullsized":"2024-09-17T08:11:24.057019+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.214309+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:13.214309+0000","last_peered":"2024-09-17T08:11:13.214309+0000","last_clean":"2024-09-17T08:11:13.214309+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:13.214309+0000","last_undegraded":"2024-09-17T08:11:13.214309+0000","last_fullsized":"2024-09-17T08:11:13.214309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155565+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:11.155565+0000","last_peered":"2024-09-17T08:11:11.155565+0000","last_clean":"2024-09-17T08:11:11.155565+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:11.155565+0000","last_undegraded":"2024-09-17T08:11:11.155565+0000","last_fullsized":"2024-09-17T08:11:11.155565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385453+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:21.385453+0000","last_peered":"2024-09-17T08:11:21.385453+0000","last_clean":"2024-09-17T08:11:21.385453+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:21.385453+0000","last_undegraded":"2024-09-17T08:11:21.385453+0000","last_fullsized":"2024-09-17T08:11:21.385453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155009+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:11.155009+0000","last_peered":"2024-09-17T08:11:11.155009+0000","last_clean":"2024-09-17T08:11:11.155009+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:11.155009+0000","last_undegraded":"2024-09-17T08:11:11.155009+0000","last_fullsized":"2024-09-17T08:11:11.155009+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.032099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154871+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:11.154871+0000","last_peered":"2024-09-17T08:11:11.154871+0000","last_clean":"2024-09-17T08:11:11.154871+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:11.154871+0000","last_undegraded":"2024-09-17T08:11:11.154871+0000","last_fullsized":"2024-09-17T08:11:11.154871+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.317266+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:12.317266+0000","last_peered":"2024-09-17T08:11:12.317266+0000","last_clean":"2024-09-17T08:11:12.317266+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:12.317266+0000","last_undegraded":"2024-09-17T08:11:12.317266+0000","last_fullsized":"2024-09-17T08:11:12.317266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352342+0000","last_change":"2024-09-17T08:09:58.508365+0000","last_active":"2024-09-17T08:11:11.352342+0000","last_peered":"2024-09-17T08:11:11.352342+0000","last_clean":"2024-09-17T08:11:11.352342+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:11.352342+0000","last_undegraded":"2024-09-17T08:11:11.352342+0000","last_fullsized":"2024-09-17T08:11:11.352342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:19.995619+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:11:19.995619+0000","last_peered":"2024-09-17T08:11:19.995619+0000","last_clean":"2024-09-17T08:11:19.995619+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:19.995619+0000","last_undegraded":"2024-09-17T08:11:19.995619+0000","last_fullsized":"2024-09-17T08:11:19.995619+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":58,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154446+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:11.154446+0000","last_peered":"2024-09-17T08:11:11.154446+0000","last_clean":"2024-09-17T08:11:11.154446+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:11.154446+0000","last_undegraded":"2024-09-17T08:11:11.154446+0000","last_fullsized":"2024-09-17T08:11:11.154446+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.933499+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:11:26.933499+0000","last_peered":"2024-09-17T08:11:26.933499+0000","last_clean":"2024-09-17T08:11:26.933499+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:26.933499+0000","last_undegraded":"2024-09-17T08:11:26.933499+0000","last_fullsized":"2024-09-17T08:11:26.933499+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:12.013206+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:12.013206+0000","last_peered":"2024-09-17T08:11:12.013206+0000","last_clean":"2024-09-17T08:11:12.013206+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:12.013206+0000","last_undegraded":"2024-09-17T08:11:12.013206+0000","last_fullsized":"2024-09-17T08:11:12.013206+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.086825+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:14.086825+0000","last_peered":"2024-09-17T08:11:14.086825+0000","last_clean":"2024-09-17T08:11:14.086825+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:14.086825+0000","last_undegraded":"2024-09-17T08:11:14.086825+0000","last_fullsized":"2024-09-17T08:11:14.086825+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351698+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:11.351698+0000","last_peered":"2024-09-17T08:11:11.351698+0000","last_clean":"2024-09-17T08:11:11.351698+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:11.351698+0000","last_undegraded":"2024-09-17T08:11:11.351698+0000","last_fullsized":"2024-09-17T08:11:11.351698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.217182+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:11.217182+0000","last_peered":"2024-09-17T08:11:11.217182+0000","last_clean":"2024-09-17T08:11:11.217182+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:11.217182+0000","last_undegraded":"2024-09-17T08:11:11.217182+0000","last_fullsized":"2024-09-17T08:11:11.217182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.288293+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:11.288293+0000","last_peered":"2024-09-17T08:11:11.288293+0000","last_clean":"2024-09-17T08:11:11.288293+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:11.288293+0000","last_undegraded":"2024-09-17T08:11:11.288293+0000","last_fullsized":"2024-09-17T08:11:11.288293+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290147+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:11.290147+0000","last_peered":"2024-09-17T08:11:11.290147+0000","last_clean":"2024-09-17T08:11:11.290147+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:11.290147+0000","last_undegraded":"2024-09-17T08:11:11.290147+0000","last_fullsized":"2024-09-17T08:11:11.290147+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":58,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.351834+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:11.351834+0000","last_peered":"2024-09-17T08:11:11.351834+0000","last_clean":"2024-09-17T08:11:11.351834+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:11.351834+0000","last_undegraded":"2024-09-17T08:11:11.351834+0000","last_fullsized":"2024-09-17T08:11:11.351834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:19.313452+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:11:19.313452+0000","last_peered":"2024-09-17T08:11:19.313452+0000","last_clean":"2024-09-17T08:11:19.313452+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:19.313452+0000","last_undegraded":"2024-09-17T08:11:19.313452+0000","last_fullsized":"2024-09-17T08:11:19.313452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.529422+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:13.529422+0000","last_peered":"2024-09-17T08:11:13.529422+0000","last_clean":"2024-09-17T08:11:13.529422+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:13.529422+0000","last_undegraded":"2024-09-17T08:11:13.529422+0000","last_fullsized":"2024-09-17T08:11:13.529422+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352056+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:11.352056+0000","last_peered":"2024-09-17T08:11:11.352056+0000","last_clean":"2024-09-17T08:11:11.352056+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:11.352056+0000","last_undegraded":"2024-09-17T08:11:11.352056+0000","last_fullsized":"2024-09-17T08:11:11.352056+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.095477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.290547+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:11.290547+0000","last_peered":"2024-09-17T08:11:11.290547+0000","last_clean":"2024-09-17T08:11:11.290547+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:11.290547+0000","last_undegraded":"2024-09-17T08:11:11.290547+0000","last_fullsized":"2024-09-17T08:11:11.290547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:28.700248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:16.188158+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:16.188158+0000","last_peered":"2024-09-17T08:11:16.188158+0000","last_clean":"2024-09-17T08:11:16.188158+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:16.188158+0000","last_undegraded":"2024-09-17T08:11:16.188158+0000","last_fullsized":"2024-09-17T08:11:16.188158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.077654+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:11:25.077654+0000","last_peered":"2024-09-17T08:11:25.077654+0000","last_clean":"2024-09-17T08:11:25.077654+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:25.077654+0000","last_undegraded":"2024-09-17T08:11:25.077654+0000","last_fullsized":"2024-09-17T08:11:25.077654+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383025+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:11:21.383025+0000","last_peered":"2024-09-17T08:11:21.383025+0000","last_clean":"2024-09-17T08:11:21.383025+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:21.383025+0000","last_undegraded":"2024-09-17T08:11:21.383025+0000","last_fullsized":"2024-09-17T08:11:21.383025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:19.931978+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:11:19.931978+0000","last_peered":"2024-09-17T08:11:19.931978+0000","last_clean":"2024-09-17T08:11:19.931978+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:19.931978+0000","last_undegraded":"2024-09-17T08:11:19.931978+0000","last_fullsized":"2024-09-17T08:11:19.931978+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.016874+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:11:26.016874+0000","last_peered":"2024-09-17T08:11:26.016874+0000","last_clean":"2024-09-17T08:11:26.016874+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:26.016874+0000","last_undegraded":"2024-09-17T08:11:26.016874+0000","last_fullsized":"2024-09-17T08:11:26.016874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":58,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:14.278901+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:14.278901+0000","last_peered":"2024-09-17T08:11:14.278901+0000","last_clean":"2024-09-17T08:11:14.278901+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:14.278901+0000","last_undegraded":"2024-09-17T08:11:14.278901+0000","last_fullsized":"2024-09-17T08:11:14.278901+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.155479+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:11.155479+0000","last_peered":"2024-09-17T08:11:11.155479+0000","last_clean":"2024-09-17T08:11:11.155479+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:11.155479+0000","last_undegraded":"2024-09-17T08:11:11.155479+0000","last_fullsized":"2024-09-17T08:11:11.155479+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154881+0000","last_change":"2024-09-17T08:10:00.598000+0000","last_active":"2024-09-17T08:11:11.154881+0000","last_peered":"2024-09-17T08:11:11.154881+0000","last_clean":"2024-09-17T08:11:11.154881+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:11.154881+0000","last_undegraded":"2024-09-17T08:11:11.154881+0000","last_fullsized":"2024-09-17T08:11:11.154881+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:13.310069+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:13.310069+0000","last_peered":"2024-09-17T08:11:13.310069+0000","last_clean":"2024-09-17T08:11:13.310069+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:13.310069+0000","last_undegraded":"2024-09-17T08:11:13.310069+0000","last_fullsized":"2024-09-17T08:11:13.310069+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"72'15","reported_seq":53,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352045+0000","last_change":"2024-09-17T08:09:56.504473+0000","last_active":"2024-09-17T08:11:11.352045+0000","last_peered":"2024-09-17T08:11:11.352045+0000","last_clean":"2024-09-17T08:11:11.352045+0000","last_became_active":"2024-09-17T08:09:56.504114+0000","last_became_peered":"2024-09-17T08:09:56.504114+0000","last_unstale":"2024-09-17T08:11:11.352045+0000","last_undegraded":"2024-09-17T08:11:11.352045+0000","last_fullsized":"2024-09-17T08:11:11.352045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.154358+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:11.154358+0000","last_peered":"2024-09-17T08:11:11.154358+0000","last_clean":"2024-09-17T08:11:11.154358+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:11.154358+0000","last_undegraded":"2024-09-17T08:11:11.154358+0000","last_fullsized":"2024-09-17T08:11:11.154358+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:21.494666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352301+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:11.352301+0000","last_peered":"2024-09-17T08:11:11.352301+0000","last_clean":"2024-09-17T08:11:11.352301+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:11.352301+0000","last_undegraded":"2024-09-17T08:11:11.352301+0000","last_fullsized":"2024-09-17T08:11:11.352301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352644+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:11.352644+0000","last_peered":"2024-09-17T08:11:11.352644+0000","last_clean":"2024-09-17T08:11:11.352644+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:11.352644+0000","last_undegraded":"2024-09-17T08:11:11.352644+0000","last_fullsized":"2024-09-17T08:11:11.352644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:11.352127+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:11.352127+0000","last_peered":"2024-09-17T08:11:11.352127+0000","last_clean":"2024-09-17T08:11:11.352127+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:11.352127+0000","last_undegraded":"2024-09-17T08:11:11.352127+0000","last_fullsized":"2024-09-17T08:11:11.352127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T08:11:15.637755+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:15.637755+0000","last_peered":"2024-09-17T08:11:15.637755+0000","last_clean":"2024-09-17T08:11:15.637755+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:15.637755+0000","last_undegraded":"2024-09-17T08:11:15.637755+0000","last_fullsized":"2024-09-17T08:11:15.637755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":401,"ondisk_log_size":401,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":81,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070488,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1277952,"data_stored":913439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.433},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.41999999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33900000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47799999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48499999999999999}]}]},{"osd":6,"up_from":50,"seq":214748364829,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32348,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712804,"statfs":{"total":95995035648,"available":95961911296,"internally_reserved":0,"allocated":1265664,"data_stored":913030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.69399999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67600000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.434},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66500000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.317,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.374},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.5}]},{"osd":5,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.38700000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.38},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.34799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39600000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.36199999999999999}]}]},{"osd":5,"up_from":41,"seq":176093659170,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31764,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713388,"statfs":{"total":95995035648,"available":95962509312,"internally_reserved":0,"allocated":667648,"data_stored":322007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.38},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33100000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.439}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.505}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41399999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39600000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52200000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953511,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27688,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717464,"statfs":{"total":95995035648,"available":95966683136,"internally_reserved":0,"allocated":688128,"data_stored":323240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.41599999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.443},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.374}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.432},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.314}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.435}]}]},{"osd":3,"up_from":27,"seq":115964117035,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27700,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":700416,"data_stored":322717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.214},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.25}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.28999999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.312},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.27900000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.374},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83299999999999996,"5min":0.873,"15min":0.873},"last":0.5}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27672,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717480,"statfs":{"total":95995035648,"available":95966699520,"internally_reserved":0,"allocated":671744,"data_stored":322789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.26200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.25},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38800000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.749,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.504},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":716800,"data_stored":323527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:27 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.25},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.30099999999999999},"min":{"1min":0.22800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.29199999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.373}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85099999999999998,"5min":1.329,"15min":1.329},"last":0.439}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.501},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1279999999999999,"5min":1.157,"15min":1.157},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47499999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28260,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716892,"statfs":{"total":95995035648,"available":95966097408,"internally_reserved":0,"allocated":1273856,"data_stored":913181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.314},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.24199999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.442},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.753,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:29.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 4.1b deep-scrub starts 2024-09-17T08:11:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 4.1b deep-scrub ok 2024-09-17T08:11:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:11:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:11:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:11:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:30 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:11:30.781 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 4.1b deep-scrub starts 2024-09-17T08:11:30.781 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 4.1b deep-scrub ok 2024-09-17T08:11:30.781 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:11:30.781 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 4.1b deep-scrub starts 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 4.1b deep-scrub ok 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:11:30.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:30 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:11:31.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:11:31.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:11:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:31.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:31 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:31 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:32.119 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:31 smithi079 bash[77546]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:11:32.434 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:32 smithi079 bash[77546]: Writing manifest to image destination 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-17T08:11:32.435 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T08:11:32.436 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:32.701 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:32 smithi079 podman[77546]: 2024-09-17 08:11:32.477042815 +0000 UTC m=+3.638612050 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:11:32.701 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:32 smithi079 podman[77546]: 2024-09-17 08:11:32.701662666 +0000 UTC m=+3.863231904 container create 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:32.701 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:11:32.701 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T08:11:32.702 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:32 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:32.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T08:11:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:32 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:33.238 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:33.239 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 podman[77546]: 2024-09-17 08:11:33.06304228 +0000 UTC m=+4.224611522 container init 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:33.239 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 podman[77546]: 2024-09-17 08:11:33.067892425 +0000 UTC m=+4.229461661 container start 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:33.239 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.073Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:11:33.239 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.073Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:11:33.239 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:11:33.240 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:11:33.241 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.074Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:11:33.242 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:11:33.243 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a[77805]: ts=2024-09-17T08:11:33.075Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:11:33.244 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 bash[77546]: 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 2024-09-17T08:11:33.244 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:11:33 smithi079 systemd[1]: Started Ceph node-exporter.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:33.249 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:33.251 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: Deploying daemon node-exporter.b on smithi173 2024-09-17T08:11:33.561 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: pgmap v46: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[40649]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: Deploying daemon node-exporter.b on smithi173 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: pgmap v46: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:33.562 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:33 smithi079 ceph-mon[29565]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: Deploying daemon node-exporter.b on smithi173 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: pgmap v46: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:11:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:33 smithi173 ceph-mon[33187]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:33.960 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-09-17T08:11:33.223286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1012,"num_read_kb":858,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":613,"ondisk_log_size":613,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239700,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749721516,"statfs":{"total":767960285184,"available":767714832384,"internally_reserved":0,"allocated":7335936,"data_stored":4401048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001365"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385201+0000","last_change":"2024-09-17T08:10:00.599431+0000","last_active":"2024-09-17T08:11:21.385201+0000","last_peered":"2024-09-17T08:11:21.385201+0000","last_clean":"2024-09-17T08:11:21.385201+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:21.385201+0000","last_undegraded":"2024-09-17T08:11:21.385201+0000","last_fullsized":"2024-09-17T08:11:21.385201+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430383+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:21.430383+0000","last_peered":"2024-09-17T08:11:21.430383+0000","last_clean":"2024-09-17T08:11:21.430383+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:21.430383+0000","last_undegraded":"2024-09-17T08:11:21.430383+0000","last_fullsized":"2024-09-17T08:11:21.430383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384992+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:21.384992+0000","last_peered":"2024-09-17T08:11:21.384992+0000","last_clean":"2024-09-17T08:11:21.384992+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:21.384992+0000","last_undegraded":"2024-09-17T08:11:21.384992+0000","last_fullsized":"2024-09-17T08:11:21.384992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777835+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:21.777835+0000","last_peered":"2024-09-17T08:11:21.777835+0000","last_clean":"2024-09-17T08:11:21.777835+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:21.777835+0000","last_undegraded":"2024-09-17T08:11:21.777835+0000","last_fullsized":"2024-09-17T08:11:21.777835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385931+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:21.385931+0000","last_peered":"2024-09-17T08:11:21.385931+0000","last_clean":"2024-09-17T08:11:21.385931+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:21.385931+0000","last_undegraded":"2024-09-17T08:11:21.385931+0000","last_fullsized":"2024-09-17T08:11:21.385931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"72'9","reported_seq":46,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.428910+0000","last_change":"2024-09-17T08:11:31.429094+0000","last_active":"2024-09-17T08:11:31.428910+0000","last_peered":"2024-09-17T08:11:21.777974+0000","last_clean":"2024-09-17T08:11:21.777974+0000","last_became_active":"2024-09-17T08:09:56.500692+0000","last_became_peered":"2024-09-17T08:09:56.500692+0000","last_unstale":"2024-09-17T08:11:31.428910+0000","last_undegraded":"2024-09-17T08:11:31.428910+0000","last_fullsized":"2024-09-17T08:11:31.428910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386281+0000","last_change":"2024-09-17T08:09:58.507646+0000","last_active":"2024-09-17T08:11:21.386281+0000","last_peered":"2024-09-17T08:11:21.386281+0000","last_clean":"2024-09-17T08:11:21.386281+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:21.386281+0000","last_undegraded":"2024-09-17T08:11:21.386281+0000","last_fullsized":"2024-09-17T08:11:21.386281+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778735+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:21.778735+0000","last_peered":"2024-09-17T08:11:21.778735+0000","last_clean":"2024-09-17T08:11:21.778735+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:21.778735+0000","last_undegraded":"2024-09-17T08:11:21.778735+0000","last_fullsized":"2024-09-17T08:11:21.778735+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.381594+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:11:22.381594+0000","last_peered":"2024-09-17T08:11:22.381594+0000","last_clean":"2024-09-17T08:11:22.381594+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:22.381594+0000","last_undegraded":"2024-09-17T08:11:22.381594+0000","last_fullsized":"2024-09-17T08:11:22.381594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:29.578149+0000","last_change":"2024-09-17T08:11:29.578149+0000","last_active":"2024-09-17T08:11:29.578149+0000","last_peered":"2024-09-17T08:11:29.578149+0000","last_clean":"2024-09-17T08:11:29.578149+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:29.578149+0000","last_undegraded":"2024-09-17T08:11:29.578149+0000","last_fullsized":"2024-09-17T08:11:29.578149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.578069+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.180514+0000","scrub_duration":0.0025270700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778552+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:21.778552+0000","last_peered":"2024-09-17T08:11:21.778552+0000","last_clean":"2024-09-17T08:11:21.778552+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:21.778552+0000","last_undegraded":"2024-09-17T08:11:21.778552+0000","last_fullsized":"2024-09-17T08:11:21.778552+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.086448+0000","last_change":"2024-09-17T08:11:31.086448+0000","last_active":"2024-09-17T08:11:31.086448+0000","last_peered":"2024-09-17T08:11:31.086448+0000","last_clean":"2024-09-17T08:11:31.086448+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:31.086448+0000","last_undegraded":"2024-09-17T08:11:31.086448+0000","last_fullsized":"2024-09-17T08:11:31.086448+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.086346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.086346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.290177+0000","scrub_duration":0.0013806619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.306565+0000","last_change":"2024-09-17T08:11:28.306565+0000","last_active":"2024-09-17T08:11:28.306565+0000","last_peered":"2024-09-17T08:11:28.306565+0000","last_clean":"2024-09-17T08:11:28.306565+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:28.306565+0000","last_undegraded":"2024-09-17T08:11:28.306565+0000","last_fullsized":"2024-09-17T08:11:28.306565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.306480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.306480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:53.967249+0000","scrub_duration":0.001181517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778919+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:21.778919+0000","last_peered":"2024-09-17T08:11:21.778919+0000","last_clean":"2024-09-17T08:11:21.778919+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:21.778919+0000","last_undegraded":"2024-09-17T08:11:21.778919+0000","last_fullsized":"2024-09-17T08:11:21.778919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.678933+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:11:21.678933+0000","last_peered":"2024-09-17T08:11:21.678933+0000","last_clean":"2024-09-17T08:11:21.678933+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:21.678933+0000","last_undegraded":"2024-09-17T08:11:21.678933+0000","last_fullsized":"2024-09-17T08:11:21.678933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.429751+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:11:21.429751+0000","last_peered":"2024-09-17T08:11:21.429751+0000","last_clean":"2024-09-17T08:11:21.429751+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:21.429751+0000","last_undegraded":"2024-09-17T08:11:21.429751+0000","last_fullsized":"2024-09-17T08:11:21.429751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778473+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:21.778473+0000","last_peered":"2024-09-17T08:11:21.778473+0000","last_clean":"2024-09-17T08:11:21.778473+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:21.778473+0000","last_undegraded":"2024-09-17T08:11:21.778473+0000","last_fullsized":"2024-09-17T08:11:21.778473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778470+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:11:21.778470+0000","last_peered":"2024-09-17T08:11:21.778470+0000","last_clean":"2024-09-17T08:11:21.778470+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:21.778470+0000","last_undegraded":"2024-09-17T08:11:21.778470+0000","last_fullsized":"2024-09-17T08:11:21.778470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.017724+0000","last_change":"2024-09-17T08:11:31.017724+0000","last_active":"2024-09-17T08:11:31.017724+0000","last_peered":"2024-09-17T08:11:31.017724+0000","last_clean":"2024-09-17T08:11:31.017724+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:31.017724+0000","last_undegraded":"2024-09-17T08:11:31.017724+0000","last_fullsized":"2024-09-17T08:11:31.017724+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.017640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.017640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.994183+0000","scrub_duration":0.0020059460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.097377+0000","last_change":"2024-09-17T08:11:28.097377+0000","last_active":"2024-09-17T08:11:28.097377+0000","last_peered":"2024-09-17T08:11:28.097377+0000","last_clean":"2024-09-17T08:11:28.097377+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:28.097377+0000","last_undegraded":"2024-09-17T08:11:28.097377+0000","last_fullsized":"2024-09-17T08:11:28.097377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.097289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.097289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.706372+0000","scrub_duration":0.001520696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778549+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:21.778549+0000","last_peered":"2024-09-17T08:11:21.778549+0000","last_clean":"2024-09-17T08:11:21.778549+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:21.778549+0000","last_undegraded":"2024-09-17T08:11:21.778549+0000","last_fullsized":"2024-09-17T08:11:21.778549+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385295+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:21.385295+0000","last_peered":"2024-09-17T08:11:21.385295+0000","last_clean":"2024-09-17T08:11:21.385295+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:21.385295+0000","last_undegraded":"2024-09-17T08:11:21.385295+0000","last_fullsized":"2024-09-17T08:11:21.385295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846374+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:11:27.846374+0000","last_peered":"2024-09-17T08:11:27.846374+0000","last_clean":"2024-09-17T08:11:27.846374+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:27.846374+0000","last_undegraded":"2024-09-17T08:11:27.846374+0000","last_fullsized":"2024-09-17T08:11:27.846374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:20.492012+0000","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779242+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:21.779242+0000","last_peered":"2024-09-17T08:11:21.779242+0000","last_clean":"2024-09-17T08:11:21.779242+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:21.779242+0000","last_undegraded":"2024-09-17T08:11:21.779242+0000","last_fullsized":"2024-09-17T08:11:21.779242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386028+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:21.386028+0000","last_peered":"2024-09-17T08:11:21.386028+0000","last_clean":"2024-09-17T08:11:21.386028+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:21.386028+0000","last_undegraded":"2024-09-17T08:11:21.386028+0000","last_fullsized":"2024-09-17T08:11:21.386028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386483+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:21.386483+0000","last_peered":"2024-09-17T08:11:21.386483+0000","last_clean":"2024-09-17T08:11:21.386483+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:21.386483+0000","last_undegraded":"2024-09-17T08:11:21.386483+0000","last_fullsized":"2024-09-17T08:11:21.386483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846689+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:11:27.846689+0000","last_peered":"2024-09-17T08:11:27.846689+0000","last_clean":"2024-09-17T08:11:27.846689+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:27.846689+0000","last_undegraded":"2024-09-17T08:11:27.846689+0000","last_fullsized":"2024-09-17T08:11:27.846689+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.126060+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:11:27.126060+0000","last_peered":"2024-09-17T08:11:27.126060+0000","last_clean":"2024-09-17T08:11:27.126060+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:27.126060+0000","last_undegraded":"2024-09-17T08:11:27.126060+0000","last_fullsized":"2024-09-17T08:11:27.126060+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.618404+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:11:25.618404+0000","last_peered":"2024-09-17T08:11:25.618404+0000","last_clean":"2024-09-17T08:11:25.618404+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:25.618404+0000","last_undegraded":"2024-09-17T08:11:25.618404+0000","last_fullsized":"2024-09-17T08:11:25.618404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.305742+0000","last_change":"2024-09-17T08:11:31.305742+0000","last_active":"2024-09-17T08:11:31.305742+0000","last_peered":"2024-09-17T08:11:31.305742+0000","last_clean":"2024-09-17T08:11:31.305742+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:31.305742+0000","last_undegraded":"2024-09-17T08:11:31.305742+0000","last_fullsized":"2024-09-17T08:11:31.305742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.305634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.305634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.476701+0000","scrub_duration":0.001519353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679426+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:21.679426+0000","last_peered":"2024-09-17T08:11:21.679426+0000","last_clean":"2024-09-17T08:11:21.679426+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:21.679426+0000","last_undegraded":"2024-09-17T08:11:21.679426+0000","last_fullsized":"2024-09-17T08:11:21.679426+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385338+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:21.385338+0000","last_peered":"2024-09-17T08:11:21.385338+0000","last_clean":"2024-09-17T08:11:21.385338+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:21.385338+0000","last_undegraded":"2024-09-17T08:11:21.385338+0000","last_fullsized":"2024-09-17T08:11:21.385338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778438+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:21.778438+0000","last_peered":"2024-09-17T08:11:21.778438+0000","last_clean":"2024-09-17T08:11:21.778438+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:21.778438+0000","last_undegraded":"2024-09-17T08:11:21.778438+0000","last_fullsized":"2024-09-17T08:11:21.778438+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383170+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:21.383170+0000","last_peered":"2024-09-17T08:11:21.383170+0000","last_clean":"2024-09-17T08:11:21.383170+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:21.383170+0000","last_undegraded":"2024-09-17T08:11:21.383170+0000","last_fullsized":"2024-09-17T08:11:21.383170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846375+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:27.846375+0000","last_peered":"2024-09-17T08:11:27.846375+0000","last_clean":"2024-09-17T08:11:27.846375+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:27.846375+0000","last_undegraded":"2024-09-17T08:11:27.846375+0000","last_fullsized":"2024-09-17T08:11:27.846375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778188+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:21.778188+0000","last_peered":"2024-09-17T08:11:21.778188+0000","last_clean":"2024-09-17T08:11:21.778188+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:21.778188+0000","last_undegraded":"2024-09-17T08:11:21.778188+0000","last_fullsized":"2024-09-17T08:11:21.778188+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384863+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:21.384863+0000","last_peered":"2024-09-17T08:11:21.384863+0000","last_clean":"2024-09-17T08:11:21.384863+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:21.384863+0000","last_undegraded":"2024-09-17T08:11:21.384863+0000","last_fullsized":"2024-09-17T08:11:21.384863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385597+0000","last_change":"2024-09-17T08:09:54.498648+0000","last_active":"2024-09-17T08:11:21.385597+0000","last_peered":"2024-09-17T08:11:21.385597+0000","last_clean":"2024-09-17T08:11:21.385597+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:21.385597+0000","last_undegraded":"2024-09-17T08:11:21.385597+0000","last_fullsized":"2024-09-17T08:11:21.385597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778204+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:21.778204+0000","last_peered":"2024-09-17T08:11:21.778204+0000","last_clean":"2024-09-17T08:11:21.778204+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:21.778204+0000","last_undegraded":"2024-09-17T08:11:21.778204+0000","last_fullsized":"2024-09-17T08:11:21.778204+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778835+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:21.778835+0000","last_peered":"2024-09-17T08:11:21.778835+0000","last_clean":"2024-09-17T08:11:21.778835+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:21.778835+0000","last_undegraded":"2024-09-17T08:11:21.778835+0000","last_fullsized":"2024-09-17T08:11:21.778835+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430043+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:21.430043+0000","last_peered":"2024-09-17T08:11:21.430043+0000","last_clean":"2024-09-17T08:11:21.430043+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:21.430043+0000","last_undegraded":"2024-09-17T08:11:21.430043+0000","last_fullsized":"2024-09-17T08:11:21.430043+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779949+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:21.779949+0000","last_peered":"2024-09-17T08:11:21.779949+0000","last_clean":"2024-09-17T08:11:21.779949+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:21.779949+0000","last_undegraded":"2024-09-17T08:11:21.779949+0000","last_fullsized":"2024-09-17T08:11:21.779949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.433770+0000","last_change":"2024-09-17T08:11:28.433770+0000","last_active":"2024-09-17T08:11:28.433770+0000","last_peered":"2024-09-17T08:11:28.433770+0000","last_clean":"2024-09-17T08:11:28.433770+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:28.433770+0000","last_undegraded":"2024-09-17T08:11:28.433770+0000","last_fullsized":"2024-09-17T08:11:28.433770+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.433666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.433666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.684654+0000","scrub_duration":0.001195822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386242+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:21.386242+0000","last_peered":"2024-09-17T08:11:21.386242+0000","last_clean":"2024-09-17T08:11:21.386242+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:21.386242+0000","last_undegraded":"2024-09-17T08:11:21.386242+0000","last_fullsized":"2024-09-17T08:11:21.386242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385176+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:11:21.385176+0000","last_peered":"2024-09-17T08:11:21.385176+0000","last_clean":"2024-09-17T08:11:21.385176+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:21.385176+0000","last_undegraded":"2024-09-17T08:11:21.385176+0000","last_fullsized":"2024-09-17T08:11:21.385176+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385294+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:11:21.385294+0000","last_peered":"2024-09-17T08:11:21.385294+0000","last_clean":"2024-09-17T08:11:21.385294+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:21.385294+0000","last_undegraded":"2024-09-17T08:11:21.385294+0000","last_fullsized":"2024-09-17T08:11:21.385294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846363+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:27.846363+0000","last_peered":"2024-09-17T08:11:27.846363+0000","last_clean":"2024-09-17T08:11:27.846363+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:27.846363+0000","last_undegraded":"2024-09-17T08:11:27.846363+0000","last_fullsized":"2024-09-17T08:11:27.846363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.080294+0000","last_change":"2024-09-17T08:11:32.080416+0000","last_active":"2024-09-17T08:11:32.080294+0000","last_peered":"2024-09-17T08:11:32.080294+0000","last_clean":"2024-09-17T08:11:32.080294+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:32.080294+0000","last_undegraded":"2024-09-17T08:11:32.080294+0000","last_fullsized":"2024-09-17T08:11:32.080294+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.080269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.080269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.073092+0000","scrub_duration":0.001335506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":107,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846378+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:27.846378+0000","last_peered":"2024-09-17T08:11:27.846378+0000","last_clean":"2024-09-17T08:11:27.846378+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:27.846378+0000","last_undegraded":"2024-09-17T08:11:27.846378+0000","last_fullsized":"2024-09-17T08:11:27.846378+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679157+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:21.679157+0000","last_peered":"2024-09-17T08:11:21.679157+0000","last_clean":"2024-09-17T08:11:21.679157+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:21.679157+0000","last_undegraded":"2024-09-17T08:11:21.679157+0000","last_fullsized":"2024-09-17T08:11:21.679157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430138+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:21.430138+0000","last_peered":"2024-09-17T08:11:21.430138+0000","last_clean":"2024-09-17T08:11:21.430138+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:21.430138+0000","last_undegraded":"2024-09-17T08:11:21.430138+0000","last_fullsized":"2024-09-17T08:11:21.430138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386001+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:21.386001+0000","last_peered":"2024-09-17T08:11:21.386001+0000","last_clean":"2024-09-17T08:11:21.386001+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:21.386001+0000","last_undegraded":"2024-09-17T08:11:21.386001+0000","last_fullsized":"2024-09-17T08:11:21.386001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:10:01.782568+0000","last_change":"2024-09-17T08:11:32.218042+0000","last_active":"2024-09-17T08:11:32.218042+0000","last_peered":"2024-09-17T08:10:01.782568+0000","last_clean":"2024-09-17T08:10:01.782568+0000","last_became_active":"2024-09-17T08:09:56.503476+0000","last_became_peered":"2024-09-17T08:09:56.503476+0000","last_unstale":"2024-09-17T08:10:01.782568+0000","last_undegraded":"2024-09-17T08:10:01.782568+0000","last_fullsized":"2024-09-17T08:10:01.782568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":141,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426382+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:31.426382+0000","last_peered":"2024-09-17T08:11:31.426382+0000","last_clean":"2024-09-17T08:11:31.426382+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:31.426382+0000","last_undegraded":"2024-09-17T08:11:31.426382+0000","last_fullsized":"2024-09-17T08:11:31.426382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.081976+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:11:23.081976+0000","last_peered":"2024-09-17T08:11:23.081976+0000","last_clean":"2024-09-17T08:11:23.081976+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:23.081976+0000","last_undegraded":"2024-09-17T08:11:23.081976+0000","last_fullsized":"2024-09-17T08:11:23.081976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.111013+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:11:27.111013+0000","last_peered":"2024-09-17T08:11:27.111013+0000","last_clean":"2024-09-17T08:11:27.111013+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:27.111013+0000","last_undegraded":"2024-09-17T08:11:27.111013+0000","last_fullsized":"2024-09-17T08:11:27.111013+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779787+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:11:21.779787+0000","last_peered":"2024-09-17T08:11:21.779787+0000","last_clean":"2024-09-17T08:11:21.779787+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:21.779787+0000","last_undegraded":"2024-09-17T08:11:21.779787+0000","last_fullsized":"2024-09-17T08:11:21.779787+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430304+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:21.430304+0000","last_peered":"2024-09-17T08:11:21.430304+0000","last_clean":"2024-09-17T08:11:21.430304+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:21.430304+0000","last_undegraded":"2024-09-17T08:11:21.430304+0000","last_fullsized":"2024-09-17T08:11:21.430304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.111674+0000","last_change":"2024-09-17T08:11:28.111674+0000","last_active":"2024-09-17T08:11:28.111674+0000","last_peered":"2024-09-17T08:11:28.111674+0000","last_clean":"2024-09-17T08:11:28.111674+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:28.111674+0000","last_undegraded":"2024-09-17T08:11:28.111674+0000","last_fullsized":"2024-09-17T08:11:28.111674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:11:28.111569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.111569+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.306471+0000","scrub_duration":0.0016978239999999999,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778974+0000","last_change":"2024-09-17T08:09:54.507600+0000","last_active":"2024-09-17T08:11:21.778974+0000","last_peered":"2024-09-17T08:11:21.778974+0000","last_clean":"2024-09-17T08:11:21.778974+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:21.778974+0000","last_undegraded":"2024-09-17T08:11:21.778974+0000","last_fullsized":"2024-09-17T08:11:21.778974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679450+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:11:21.679450+0000","last_peered":"2024-09-17T08:11:21.679450+0000","last_clean":"2024-09-17T08:11:21.679450+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:21.679450+0000","last_undegraded":"2024-09-17T08:11:21.679450+0000","last_fullsized":"2024-09-17T08:11:21.679450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384972+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:21.384972+0000","last_peered":"2024-09-17T08:11:21.384972+0000","last_clean":"2024-09-17T08:11:21.384972+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:21.384972+0000","last_undegraded":"2024-09-17T08:11:21.384972+0000","last_fullsized":"2024-09-17T08:11:21.384972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382705+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:21.382705+0000","last_peered":"2024-09-17T08:11:21.382705+0000","last_clean":"2024-09-17T08:11:21.382705+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:21.382705+0000","last_undegraded":"2024-09-17T08:11:21.382705+0000","last_fullsized":"2024-09-17T08:11:21.382705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385117+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:21.385117+0000","last_peered":"2024-09-17T08:11:21.385117+0000","last_clean":"2024-09-17T08:11:21.385117+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:21.385117+0000","last_undegraded":"2024-09-17T08:11:21.385117+0000","last_fullsized":"2024-09-17T08:11:21.385117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679306+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:11:21.679306+0000","last_peered":"2024-09-17T08:11:21.679306+0000","last_clean":"2024-09-17T08:11:21.679306+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:21.679306+0000","last_undegraded":"2024-09-17T08:11:21.679306+0000","last_fullsized":"2024-09-17T08:11:21.679306+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":292,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779059+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:21.779059+0000","last_peered":"2024-09-17T08:11:21.779059+0000","last_clean":"2024-09-17T08:11:21.779059+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:21.779059+0000","last_undegraded":"2024-09-17T08:11:21.779059+0000","last_fullsized":"2024-09-17T08:11:21.779059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779640+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:21.779640+0000","last_peered":"2024-09-17T08:11:21.779640+0000","last_clean":"2024-09-17T08:11:21.779640+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:21.779640+0000","last_undegraded":"2024-09-17T08:11:21.779640+0000","last_fullsized":"2024-09-17T08:11:21.779640+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679510+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:21.679510+0000","last_peered":"2024-09-17T08:11:21.679510+0000","last_clean":"2024-09-17T08:11:21.679510+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:21.679510+0000","last_undegraded":"2024-09-17T08:11:21.679510+0000","last_fullsized":"2024-09-17T08:11:21.679510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.389593+0000","last_change":"2024-09-17T08:11:27.389593+0000","last_active":"2024-09-17T08:11:27.389593+0000","last_peered":"2024-09-17T08:11:27.389593+0000","last_clean":"2024-09-17T08:11:27.389593+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:27.389593+0000","last_undegraded":"2024-09-17T08:11:27.389593+0000","last_fullsized":"2024-09-17T08:11:27.389593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:27.389490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.389490+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.830431+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:30.055058+0000","last_change":"2024-09-17T08:11:30.055058+0000","last_active":"2024-09-17T08:11:30.055058+0000","last_peered":"2024-09-17T08:11:30.055058+0000","last_clean":"2024-09-17T08:11:30.055058+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:30.055058+0000","last_undegraded":"2024-09-17T08:11:30.055058+0000","last_fullsized":"2024-09-17T08:11:30.055058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:30.054968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.054968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.938144+0000","scrub_duration":0.0018411460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777525+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:21.777525+0000","last_peered":"2024-09-17T08:11:21.777525+0000","last_clean":"2024-09-17T08:11:21.777525+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:21.777525+0000","last_undegraded":"2024-09-17T08:11:21.777525+0000","last_fullsized":"2024-09-17T08:11:21.777525+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.312687+0000","last_change":"2024-09-17T08:11:27.312687+0000","last_active":"2024-09-17T08:11:27.312687+0000","last_peered":"2024-09-17T08:11:27.312687+0000","last_clean":"2024-09-17T08:11:27.312687+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:27.312687+0000","last_undegraded":"2024-09-17T08:11:27.312687+0000","last_fullsized":"2024-09-17T08:11:27.312687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.312595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.312595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.098135+0000","scrub_duration":0.001815178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:30.115032+0000","last_change":"2024-09-17T08:11:30.115032+0000","last_active":"2024-09-17T08:11:30.115032+0000","last_peered":"2024-09-17T08:11:30.115032+0000","last_clean":"2024-09-17T08:11:30.115032+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:30.115032+0000","last_undegraded":"2024-09-17T08:11:30.115032+0000","last_fullsized":"2024-09-17T08:11:30.115032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:30.114921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.114921+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.519251+0000","scrub_duration":0.0019919109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386200+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:11:21.386200+0000","last_peered":"2024-09-17T08:11:21.386200+0000","last_clean":"2024-09-17T08:11:21.386200+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:21.386200+0000","last_undegraded":"2024-09-17T08:11:21.386200+0000","last_fullsized":"2024-09-17T08:11:21.386200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.124631+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:11:25.124631+0000","last_peered":"2024-09-17T08:11:25.124631+0000","last_clean":"2024-09-17T08:11:25.124631+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:25.124631+0000","last_undegraded":"2024-09-17T08:11:25.124631+0000","last_fullsized":"2024-09-17T08:11:25.124631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.073527+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:11:23.073527+0000","last_peered":"2024-09-17T08:11:23.073527+0000","last_clean":"2024-09-17T08:11:23.073527+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:23.073527+0000","last_undegraded":"2024-09-17T08:11:23.073527+0000","last_fullsized":"2024-09-17T08:11:23.073527+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.079173+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:11:23.079173+0000","last_peered":"2024-09-17T08:11:23.079173+0000","last_clean":"2024-09-17T08:11:23.079173+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:23.079173+0000","last_undegraded":"2024-09-17T08:11:23.079173+0000","last_fullsized":"2024-09-17T08:11:23.079173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777771+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:21.777771+0000","last_peered":"2024-09-17T08:11:21.777771+0000","last_clean":"2024-09-17T08:11:21.777771+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:21.777771+0000","last_undegraded":"2024-09-17T08:11:21.777771+0000","last_fullsized":"2024-09-17T08:11:21.777771+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846632+0000","last_change":"2024-09-17T08:09:58.513726+0000","last_active":"2024-09-17T08:11:27.846632+0000","last_peered":"2024-09-17T08:11:27.846632+0000","last_clean":"2024-09-17T08:11:27.846632+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:27.846632+0000","last_undegraded":"2024-09-17T08:11:27.846632+0000","last_fullsized":"2024-09-17T08:11:27.846632+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.089111+0000","last_change":"2024-09-17T08:11:32.089192+0000","last_active":"2024-09-17T08:11:32.089111+0000","last_peered":"2024-09-17T08:11:32.089111+0000","last_clean":"2024-09-17T08:11:32.089111+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:32.089111+0000","last_undegraded":"2024-09-17T08:11:32.089111+0000","last_fullsized":"2024-09-17T08:11:32.089111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.089096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.089096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.048086+0000","scrub_duration":0.0013942010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778331+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:21.778331+0000","last_peered":"2024-09-17T08:11:21.778331+0000","last_clean":"2024-09-17T08:11:21.778331+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:21.778331+0000","last_undegraded":"2024-09-17T08:11:21.778331+0000","last_fullsized":"2024-09-17T08:11:21.778331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778456+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:21.778456+0000","last_peered":"2024-09-17T08:11:21.778456+0000","last_clean":"2024-09-17T08:11:21.778456+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:21.778456+0000","last_undegraded":"2024-09-17T08:11:21.778456+0000","last_fullsized":"2024-09-17T08:11:21.778456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777344+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:21.777344+0000","last_peered":"2024-09-17T08:11:21.777344+0000","last_clean":"2024-09-17T08:11:21.777344+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:21.777344+0000","last_undegraded":"2024-09-17T08:11:21.777344+0000","last_fullsized":"2024-09-17T08:11:21.777344+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385339+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:21.385339+0000","last_peered":"2024-09-17T08:11:21.385339+0000","last_clean":"2024-09-17T08:11:21.385339+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:21.385339+0000","last_undegraded":"2024-09-17T08:11:21.385339+0000","last_fullsized":"2024-09-17T08:11:21.385339+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.027206+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:11:22.027206+0000","last_peered":"2024-09-17T08:11:22.027206+0000","last_clean":"2024-09-17T08:11:22.027206+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:22.027206+0000","last_undegraded":"2024-09-17T08:11:22.027206+0000","last_fullsized":"2024-09-17T08:11:22.027206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777417+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:21.777417+0000","last_peered":"2024-09-17T08:11:21.777417+0000","last_clean":"2024-09-17T08:11:21.777417+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:21.777417+0000","last_undegraded":"2024-09-17T08:11:21.777417+0000","last_fullsized":"2024-09-17T08:11:21.777417+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386023+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:21.386023+0000","last_peered":"2024-09-17T08:11:21.386023+0000","last_clean":"2024-09-17T08:11:21.386023+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:21.386023+0000","last_undegraded":"2024-09-17T08:11:21.386023+0000","last_fullsized":"2024-09-17T08:11:21.386023+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777424+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:21.777424+0000","last_peered":"2024-09-17T08:11:21.777424+0000","last_clean":"2024-09-17T08:11:21.777424+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:21.777424+0000","last_undegraded":"2024-09-17T08:11:21.777424+0000","last_fullsized":"2024-09-17T08:11:21.777424+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382190+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:11:21.382190+0000","last_peered":"2024-09-17T08:11:21.382190+0000","last_clean":"2024-09-17T08:11:21.382190+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:21.382190+0000","last_undegraded":"2024-09-17T08:11:21.382190+0000","last_fullsized":"2024-09-17T08:11:21.382190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.028718+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:11:23.028718+0000","last_peered":"2024-09-17T08:11:23.028718+0000","last_clean":"2024-09-17T08:11:23.028718+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:23.028718+0000","last_undegraded":"2024-09-17T08:11:23.028718+0000","last_fullsized":"2024-09-17T08:11:23.028718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679217+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:21.679217+0000","last_peered":"2024-09-17T08:11:21.679217+0000","last_clean":"2024-09-17T08:11:21.679217+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:21.679217+0000","last_undegraded":"2024-09-17T08:11:21.679217+0000","last_fullsized":"2024-09-17T08:11:21.679217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779725+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:21.779725+0000","last_peered":"2024-09-17T08:11:21.779725+0000","last_clean":"2024-09-17T08:11:21.779725+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:21.779725+0000","last_undegraded":"2024-09-17T08:11:21.779725+0000","last_fullsized":"2024-09-17T08:11:21.779725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:29.979585+0000","last_change":"2024-09-17T08:11:29.979585+0000","last_active":"2024-09-17T08:11:29.979585+0000","last_peered":"2024-09-17T08:11:29.979585+0000","last_clean":"2024-09-17T08:11:29.979585+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:29.979585+0000","last_undegraded":"2024-09-17T08:11:29.979585+0000","last_fullsized":"2024-09-17T08:11:29.979585+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:29.979461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.979461+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.112016+0000","scrub_duration":0.0029734470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.431193+0000","last_change":"2024-09-17T08:11:31.431465+0000","last_active":"2024-09-17T08:11:31.431193+0000","last_peered":"2024-09-17T08:11:21.679113+0000","last_clean":"2024-09-17T08:11:21.679113+0000","last_became_active":"2024-09-17T08:09:54.501179+0000","last_became_peered":"2024-09-17T08:09:54.501179+0000","last_unstale":"2024-09-17T08:11:31.431193+0000","last_undegraded":"2024-09-17T08:11:31.431193+0000","last_fullsized":"2024-09-17T08:11:31.431193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.453505+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846733+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:27.846733+0000","last_peered":"2024-09-17T08:11:27.846733+0000","last_clean":"2024-09-17T08:11:27.846733+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:27.846733+0000","last_undegraded":"2024-09-17T08:11:27.846733+0000","last_fullsized":"2024-09-17T08:11:27.846733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:24.057092+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:11:24.057092+0000","last_peered":"2024-09-17T08:11:24.057092+0000","last_clean":"2024-09-17T08:11:24.057092+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:24.057092+0000","last_undegraded":"2024-09-17T08:11:24.057092+0000","last_fullsized":"2024-09-17T08:11:24.057092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679320+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:21.679320+0000","last_peered":"2024-09-17T08:11:21.679320+0000","last_clean":"2024-09-17T08:11:21.679320+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:21.679320+0000","last_undegraded":"2024-09-17T08:11:21.679320+0000","last_fullsized":"2024-09-17T08:11:21.679320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430190+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:21.430190+0000","last_peered":"2024-09-17T08:11:21.430190+0000","last_clean":"2024-09-17T08:11:21.430190+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:21.430190+0000","last_undegraded":"2024-09-17T08:11:21.430190+0000","last_fullsized":"2024-09-17T08:11:21.430190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846621+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:27.846621+0000","last_peered":"2024-09-17T08:11:27.846621+0000","last_clean":"2024-09-17T08:11:27.846621+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:27.846621+0000","last_undegraded":"2024-09-17T08:11:27.846621+0000","last_fullsized":"2024-09-17T08:11:27.846621+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778316+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:21.778316+0000","last_peered":"2024-09-17T08:11:21.778316+0000","last_clean":"2024-09-17T08:11:21.778316+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:21.778316+0000","last_undegraded":"2024-09-17T08:11:21.778316+0000","last_fullsized":"2024-09-17T08:11:21.778316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385106+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:21.385106+0000","last_peered":"2024-09-17T08:11:21.385106+0000","last_clean":"2024-09-17T08:11:21.385106+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:21.385106+0000","last_undegraded":"2024-09-17T08:11:21.385106+0000","last_fullsized":"2024-09-17T08:11:21.385106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382583+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:21.382583+0000","last_peered":"2024-09-17T08:11:21.382583+0000","last_clean":"2024-09-17T08:11:21.382583+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:21.382583+0000","last_undegraded":"2024-09-17T08:11:21.382583+0000","last_fullsized":"2024-09-17T08:11:21.382583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.019796+0000","last_change":"2024-09-17T08:11:31.019796+0000","last_active":"2024-09-17T08:11:31.019796+0000","last_peered":"2024-09-17T08:11:31.019796+0000","last_clean":"2024-09-17T08:11:31.019796+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:31.019796+0000","last_undegraded":"2024-09-17T08:11:31.019796+0000","last_fullsized":"2024-09-17T08:11:31.019796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.019695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.019695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.782145+0000","scrub_duration":0.001088767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.429633+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:11:21.429633+0000","last_peered":"2024-09-17T08:11:21.429633+0000","last_clean":"2024-09-17T08:11:21.429633+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:21.429633+0000","last_undegraded":"2024-09-17T08:11:21.429633+0000","last_fullsized":"2024-09-17T08:11:21.429633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778458+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:21.778458+0000","last_peered":"2024-09-17T08:11:21.778458+0000","last_clean":"2024-09-17T08:11:21.778458+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:21.778458+0000","last_undegraded":"2024-09-17T08:11:21.778458+0000","last_fullsized":"2024-09-17T08:11:21.778458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.933581+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:11:26.933581+0000","last_peered":"2024-09-17T08:11:26.933581+0000","last_clean":"2024-09-17T08:11:26.933581+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:26.933581+0000","last_undegraded":"2024-09-17T08:11:26.933581+0000","last_fullsized":"2024-09-17T08:11:26.933581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384983+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:21.384983+0000","last_peered":"2024-09-17T08:11:21.384983+0000","last_clean":"2024-09-17T08:11:21.384983+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:21.384983+0000","last_undegraded":"2024-09-17T08:11:21.384983+0000","last_fullsized":"2024-09-17T08:11:21.384983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384740+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:21.384740+0000","last_peered":"2024-09-17T08:11:21.384740+0000","last_clean":"2024-09-17T08:11:21.384740+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:21.384740+0000","last_undegraded":"2024-09-17T08:11:21.384740+0000","last_fullsized":"2024-09-17T08:11:21.384740+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384767+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:21.384767+0000","last_peered":"2024-09-17T08:11:21.384767+0000","last_clean":"2024-09-17T08:11:21.384767+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:21.384767+0000","last_undegraded":"2024-09-17T08:11:21.384767+0000","last_fullsized":"2024-09-17T08:11:21.384767+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679371+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:21.679371+0000","last_peered":"2024-09-17T08:11:21.679371+0000","last_clean":"2024-09-17T08:11:21.679371+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:21.679371+0000","last_undegraded":"2024-09-17T08:11:21.679371+0000","last_fullsized":"2024-09-17T08:11:21.679371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.381919+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:21.381919+0000","last_peered":"2024-09-17T08:11:21.381919+0000","last_clean":"2024-09-17T08:11:21.381919+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:21.381919+0000","last_undegraded":"2024-09-17T08:11:21.381919+0000","last_fullsized":"2024-09-17T08:11:21.381919+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778483+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:21.778483+0000","last_peered":"2024-09-17T08:11:21.778483+0000","last_clean":"2024-09-17T08:11:21.778483+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:21.778483+0000","last_undegraded":"2024-09-17T08:11:21.778483+0000","last_fullsized":"2024-09-17T08:11:21.778483+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384919+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:21.384919+0000","last_peered":"2024-09-17T08:11:21.384919+0000","last_clean":"2024-09-17T08:11:21.384919+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:21.384919+0000","last_undegraded":"2024-09-17T08:11:21.384919+0000","last_fullsized":"2024-09-17T08:11:21.384919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383122+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:11:21.383122+0000","last_peered":"2024-09-17T08:11:21.383122+0000","last_clean":"2024-09-17T08:11:21.383122+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:21.383122+0000","last_undegraded":"2024-09-17T08:11:21.383122+0000","last_fullsized":"2024-09-17T08:11:21.383122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778595+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:21.778595+0000","last_peered":"2024-09-17T08:11:21.778595+0000","last_clean":"2024-09-17T08:11:21.778595+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:21.778595+0000","last_undegraded":"2024-09-17T08:11:21.778595+0000","last_fullsized":"2024-09-17T08:11:21.778595+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385277+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:21.385277+0000","last_peered":"2024-09-17T08:11:21.385277+0000","last_clean":"2024-09-17T08:11:21.385277+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:21.385277+0000","last_undegraded":"2024-09-17T08:11:21.385277+0000","last_fullsized":"2024-09-17T08:11:21.385277+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778140+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:21.778140+0000","last_peered":"2024-09-17T08:11:21.778140+0000","last_clean":"2024-09-17T08:11:21.778140+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:21.778140+0000","last_undegraded":"2024-09-17T08:11:21.778140+0000","last_fullsized":"2024-09-17T08:11:21.778140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.678796+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:21.678796+0000","last_peered":"2024-09-17T08:11:21.678796+0000","last_clean":"2024-09-17T08:11:21.678796+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:21.678796+0000","last_undegraded":"2024-09-17T08:11:21.678796+0000","last_fullsized":"2024-09-17T08:11:21.678796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.077750+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:11:25.077750+0000","last_peered":"2024-09-17T08:11:25.077750+0000","last_clean":"2024-09-17T08:11:25.077750+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:25.077750+0000","last_undegraded":"2024-09-17T08:11:25.077750+0000","last_fullsized":"2024-09-17T08:11:25.077750+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846285+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:11:27.846285+0000","last_peered":"2024-09-17T08:11:27.846285+0000","last_clean":"2024-09-17T08:11:27.846285+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:27.846285+0000","last_undegraded":"2024-09-17T08:11:27.846285+0000","last_fullsized":"2024-09-17T08:11:27.846285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385674+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:11:21.385674+0000","last_peered":"2024-09-17T08:11:21.385674+0000","last_clean":"2024-09-17T08:11:21.385674+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:21.385674+0000","last_undegraded":"2024-09-17T08:11:21.385674+0000","last_fullsized":"2024-09-17T08:11:21.385674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.016945+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:11:26.016945+0000","last_peered":"2024-09-17T08:11:26.016945+0000","last_clean":"2024-09-17T08:11:26.016945+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:26.016945+0000","last_undegraded":"2024-09-17T08:11:26.016945+0000","last_fullsized":"2024-09-17T08:11:26.016945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383090+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:21.383090+0000","last_peered":"2024-09-17T08:11:21.383090+0000","last_clean":"2024-09-17T08:11:21.383090+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:21.383090+0000","last_undegraded":"2024-09-17T08:11:21.383090+0000","last_fullsized":"2024-09-17T08:11:21.383090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430171+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:21.430171+0000","last_peered":"2024-09-17T08:11:21.430171+0000","last_clean":"2024-09-17T08:11:21.430171+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:21.430171+0000","last_undegraded":"2024-09-17T08:11:21.430171+0000","last_fullsized":"2024-09-17T08:11:21.430171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779316+0000","last_change":"2024-09-17T08:10:00.598000+0000","last_active":"2024-09-17T08:11:21.779316+0000","last_peered":"2024-09-17T08:11:21.779316+0000","last_clean":"2024-09-17T08:11:21.779316+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:21.779316+0000","last_undegraded":"2024-09-17T08:11:21.779316+0000","last_fullsized":"2024-09-17T08:11:21.779316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382413+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:21.382413+0000","last_peered":"2024-09-17T08:11:21.382413+0000","last_clean":"2024-09-17T08:11:21.382413+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:21.382413+0000","last_undegraded":"2024-09-17T08:11:21.382413+0000","last_fullsized":"2024-09-17T08:11:21.382413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"72'15","reported_seq":55,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.428010+0000","last_change":"2024-09-17T08:11:31.428155+0000","last_active":"2024-09-17T08:11:31.428010+0000","last_peered":"2024-09-17T08:11:21.777584+0000","last_clean":"2024-09-17T08:11:21.777584+0000","last_became_active":"2024-09-17T08:09:56.504114+0000","last_became_peered":"2024-09-17T08:09:56.504114+0000","last_unstale":"2024-09-17T08:11:31.428010+0000","last_undegraded":"2024-09-17T08:11:31.428010+0000","last_fullsized":"2024-09-17T08:11:31.428010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385933+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:21.385933+0000","last_peered":"2024-09-17T08:11:21.385933+0000","last_clean":"2024-09-17T08:11:21.385933+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:21.385933+0000","last_undegraded":"2024-09-17T08:11:21.385933+0000","last_fullsized":"2024-09-17T08:11:21.385933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386027+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:21.386027+0000","last_peered":"2024-09-17T08:11:21.386027+0000","last_clean":"2024-09-17T08:11:21.386027+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:21.386027+0000","last_undegraded":"2024-09-17T08:11:21.386027+0000","last_fullsized":"2024-09-17T08:11:21.386027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385560+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:21.385560+0000","last_peered":"2024-09-17T08:11:21.385560+0000","last_clean":"2024-09-17T08:11:21.385560+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:21.385560+0000","last_undegraded":"2024-09-17T08:11:21.385560+0000","last_fullsized":"2024-09-17T08:11:21.385560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385927+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:21.385927+0000","last_peered":"2024-09-17T08:11:21.385927+0000","last_clean":"2024-09-17T08:11:21.385927+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:21.385927+0000","last_undegraded":"2024-09-17T08:11:21.385927+0000","last_fullsized":"2024-09-17T08:11:21.385927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778521+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:21.778521+0000","last_peered":"2024-09-17T08:11:21.778521+0000","last_clean":"2024-09-17T08:11:21.778521+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:21.778521+0000","last_undegraded":"2024-09-17T08:11:21.778521+0000","last_fullsized":"2024-09-17T08:11:21.778521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070489,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1277952,"data_stored":913439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.495}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46300000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.443}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.372}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.434}]}]},{"osd":6,"up_from":50,"seq":214748364830,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32348,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712804,"statfs":{"total":95995035648,"available":95961911296,"internally_reserved":0,"allocated":1265664,"data_stored":913030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.69399999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67600000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.625},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66500000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.317,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.374},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.38700000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.34799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39500000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.372}]}]},{"osd":5,"up_from":41,"seq":176093659171,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31780,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713372,"statfs":{"total":95995035648,"available":95962492928,"internally_reserved":0,"allocated":679936,"data_stored":329860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.44}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.621}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51400000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717448,"statfs":{"total":95995035648,"available":95966666752,"internally_reserved":0,"allocated":700416,"data_stored":331093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.629}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.432},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.61099999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117036,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717436,"statfs":{"total":95995035648,"available":95966654464,"internally_reserved":0,"allocated":712704,"data_stored":330570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.37},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83299999999999996,"5min":0.873,"15min":0.873},"last":0.435}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55200000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717464,"statfs":{"total":95995035648,"available":95966683136,"internally_reserved":0,"allocated":684032,"data_stored":330642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.254},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.23499999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.497}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.749,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.441}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31828,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713324,"statfs":{"total":95995035648,"available":95962443776,"internally_reserved":0,"allocated":729088,"data_stored":331380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.191,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.39400000000000002,"5min":0.625,"15min":0.625},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.312,"5min":0.33300000000000002,"15min":0.30299999999999999},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46100000000000002,"5min":0.625,"15min":0.625},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85099999999999998,"5min":1.329,"15min":1.329},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1279999999999999,"5min":1.157,"15min":1.157},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.57099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28276,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716876,"statfs":{"total":95995035648,"available":95966081024,"internally_reserved":0,"allocated":1286144,"data_stored":921034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.216},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.23100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.315}]},{"osd":4,"last update":"Tue Sep 17 08:10:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.753,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47599999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.5},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:33.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:34.097 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:33 smithi173 systemd[1]: Starting Ceph node-exporter.b for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:34.168 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:34.437 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:34 smithi173 bash[66675]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T08:11:34.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: 4.18 deep-scrub starts 2024-09-17T08:11:34.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:11:34.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: 4.18 deep-scrub starts 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:11:34.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:34 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:11:34.723 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:34 smithi173 bash[66675]: Getting image source signatures 2024-09-17T08:11:34.724 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:34 smithi173 bash[66675]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T08:11:34.724 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:34 smithi173 bash[66675]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T08:11:34.724 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:34 smithi173 bash[66675]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T08:11:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: 4.18 deep-scrub starts 2024-09-17T08:11:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:11:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:11:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:11:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:11:34.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:34 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 4.18 deep-scrub ok 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 4.1c deep-scrub starts 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: 4.1c deep-scrub ok 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[40649]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:11:35.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 4.18 deep-scrub ok 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 4.1c deep-scrub starts 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: 4.1c deep-scrub ok 2024-09-17T08:11:35.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:35 smithi079 ceph-mon[29565]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:11:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 4.18 deep-scrub ok 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 4.1c deep-scrub starts 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: 4.1c deep-scrub ok 2024-09-17T08:11:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:35 smithi173 ceph-mon[33187]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:11:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:35.989 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:36.432 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:35 smithi173 bash[66675]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T08:11:36.432 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 bash[66675]: Writing manifest to image destination 2024-09-17T08:11:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:11:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:11:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[40649]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:11:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:11:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:36 smithi079 ceph-mon[29565]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:36.720 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:11:36.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:11:36.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-mon[33187]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:36.721 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 podman[66675]: 2024-09-17 08:11:36.385980332 +0000 UTC m=+2.381912229 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T08:11:36.721 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 podman[66675]: 2024-09-17 08:11:36.484744604 +0000 UTC m=+2.480676486 container create f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:11:36.721 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 podman[66675]: 2024-09-17 08:11:36.720757961 +0000 UTC m=+2.716689848 container init f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:11:36.726 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-17T08:11:35.223680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1012,"num_read_kb":858,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":613,"ondisk_log_size":613,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239700,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749721516,"statfs":{"total":767960285184,"available":767714832384,"internally_reserved":0,"allocated":7335936,"data_stored":4401048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001427"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385201+0000","last_change":"2024-09-17T08:10:00.599431+0000","last_active":"2024-09-17T08:11:21.385201+0000","last_peered":"2024-09-17T08:11:21.385201+0000","last_clean":"2024-09-17T08:11:21.385201+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:21.385201+0000","last_undegraded":"2024-09-17T08:11:21.385201+0000","last_fullsized":"2024-09-17T08:11:21.385201+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430383+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:21.430383+0000","last_peered":"2024-09-17T08:11:21.430383+0000","last_clean":"2024-09-17T08:11:21.430383+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:21.430383+0000","last_undegraded":"2024-09-17T08:11:21.430383+0000","last_fullsized":"2024-09-17T08:11:21.430383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384992+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:21.384992+0000","last_peered":"2024-09-17T08:11:21.384992+0000","last_clean":"2024-09-17T08:11:21.384992+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:21.384992+0000","last_undegraded":"2024-09-17T08:11:21.384992+0000","last_fullsized":"2024-09-17T08:11:21.384992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777835+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:21.777835+0000","last_peered":"2024-09-17T08:11:21.777835+0000","last_clean":"2024-09-17T08:11:21.777835+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:21.777835+0000","last_undegraded":"2024-09-17T08:11:21.777835+0000","last_fullsized":"2024-09-17T08:11:21.777835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385931+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:21.385931+0000","last_peered":"2024-09-17T08:11:21.385931+0000","last_clean":"2024-09-17T08:11:21.385931+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:21.385931+0000","last_undegraded":"2024-09-17T08:11:21.385931+0000","last_fullsized":"2024-09-17T08:11:21.385931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"72'9","reported_seq":46,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.428910+0000","last_change":"2024-09-17T08:11:31.429094+0000","last_active":"2024-09-17T08:11:31.428910+0000","last_peered":"2024-09-17T08:11:21.777974+0000","last_clean":"2024-09-17T08:11:21.777974+0000","last_became_active":"2024-09-17T08:09:56.500692+0000","last_became_peered":"2024-09-17T08:09:56.500692+0000","last_unstale":"2024-09-17T08:11:31.428910+0000","last_undegraded":"2024-09-17T08:11:31.428910+0000","last_fullsized":"2024-09-17T08:11:31.428910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386281+0000","last_change":"2024-09-17T08:09:58.507646+0000","last_active":"2024-09-17T08:11:21.386281+0000","last_peered":"2024-09-17T08:11:21.386281+0000","last_clean":"2024-09-17T08:11:21.386281+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:21.386281+0000","last_undegraded":"2024-09-17T08:11:21.386281+0000","last_fullsized":"2024-09-17T08:11:21.386281+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778735+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:21.778735+0000","last_peered":"2024-09-17T08:11:21.778735+0000","last_clean":"2024-09-17T08:11:21.778735+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:21.778735+0000","last_undegraded":"2024-09-17T08:11:21.778735+0000","last_fullsized":"2024-09-17T08:11:21.778735+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.381594+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:11:22.381594+0000","last_peered":"2024-09-17T08:11:22.381594+0000","last_clean":"2024-09-17T08:11:22.381594+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:22.381594+0000","last_undegraded":"2024-09-17T08:11:22.381594+0000","last_fullsized":"2024-09-17T08:11:22.381594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:29.578149+0000","last_change":"2024-09-17T08:11:29.578149+0000","last_active":"2024-09-17T08:11:29.578149+0000","last_peered":"2024-09-17T08:11:29.578149+0000","last_clean":"2024-09-17T08:11:29.578149+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:29.578149+0000","last_undegraded":"2024-09-17T08:11:29.578149+0000","last_fullsized":"2024-09-17T08:11:29.578149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.578069+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.180514+0000","scrub_duration":0.0025270700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778552+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:21.778552+0000","last_peered":"2024-09-17T08:11:21.778552+0000","last_clean":"2024-09-17T08:11:21.778552+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:21.778552+0000","last_undegraded":"2024-09-17T08:11:21.778552+0000","last_fullsized":"2024-09-17T08:11:21.778552+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.086448+0000","last_change":"2024-09-17T08:11:31.086448+0000","last_active":"2024-09-17T08:11:31.086448+0000","last_peered":"2024-09-17T08:11:31.086448+0000","last_clean":"2024-09-17T08:11:31.086448+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:31.086448+0000","last_undegraded":"2024-09-17T08:11:31.086448+0000","last_fullsized":"2024-09-17T08:11:31.086448+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.086346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.086346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.290177+0000","scrub_duration":0.0013806619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.306565+0000","last_change":"2024-09-17T08:11:28.306565+0000","last_active":"2024-09-17T08:11:28.306565+0000","last_peered":"2024-09-17T08:11:28.306565+0000","last_clean":"2024-09-17T08:11:28.306565+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:28.306565+0000","last_undegraded":"2024-09-17T08:11:28.306565+0000","last_fullsized":"2024-09-17T08:11:28.306565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.306480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.306480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:53.967249+0000","scrub_duration":0.001181517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778919+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:21.778919+0000","last_peered":"2024-09-17T08:11:21.778919+0000","last_clean":"2024-09-17T08:11:21.778919+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:21.778919+0000","last_undegraded":"2024-09-17T08:11:21.778919+0000","last_fullsized":"2024-09-17T08:11:21.778919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.678933+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:11:21.678933+0000","last_peered":"2024-09-17T08:11:21.678933+0000","last_clean":"2024-09-17T08:11:21.678933+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:21.678933+0000","last_undegraded":"2024-09-17T08:11:21.678933+0000","last_fullsized":"2024-09-17T08:11:21.678933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.429751+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:11:21.429751+0000","last_peered":"2024-09-17T08:11:21.429751+0000","last_clean":"2024-09-17T08:11:21.429751+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:21.429751+0000","last_undegraded":"2024-09-17T08:11:21.429751+0000","last_fullsized":"2024-09-17T08:11:21.429751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778473+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:21.778473+0000","last_peered":"2024-09-17T08:11:21.778473+0000","last_clean":"2024-09-17T08:11:21.778473+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:21.778473+0000","last_undegraded":"2024-09-17T08:11:21.778473+0000","last_fullsized":"2024-09-17T08:11:21.778473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778470+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:11:21.778470+0000","last_peered":"2024-09-17T08:11:21.778470+0000","last_clean":"2024-09-17T08:11:21.778470+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:21.778470+0000","last_undegraded":"2024-09-17T08:11:21.778470+0000","last_fullsized":"2024-09-17T08:11:21.778470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.017724+0000","last_change":"2024-09-17T08:11:31.017724+0000","last_active":"2024-09-17T08:11:31.017724+0000","last_peered":"2024-09-17T08:11:31.017724+0000","last_clean":"2024-09-17T08:11:31.017724+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:31.017724+0000","last_undegraded":"2024-09-17T08:11:31.017724+0000","last_fullsized":"2024-09-17T08:11:31.017724+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.017640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.017640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.994183+0000","scrub_duration":0.0020059460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.097377+0000","last_change":"2024-09-17T08:11:28.097377+0000","last_active":"2024-09-17T08:11:28.097377+0000","last_peered":"2024-09-17T08:11:28.097377+0000","last_clean":"2024-09-17T08:11:28.097377+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:28.097377+0000","last_undegraded":"2024-09-17T08:11:28.097377+0000","last_fullsized":"2024-09-17T08:11:28.097377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.097289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.097289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.706372+0000","scrub_duration":0.001520696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778549+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:21.778549+0000","last_peered":"2024-09-17T08:11:21.778549+0000","last_clean":"2024-09-17T08:11:21.778549+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:21.778549+0000","last_undegraded":"2024-09-17T08:11:21.778549+0000","last_fullsized":"2024-09-17T08:11:21.778549+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385295+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:21.385295+0000","last_peered":"2024-09-17T08:11:21.385295+0000","last_clean":"2024-09-17T08:11:21.385295+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:21.385295+0000","last_undegraded":"2024-09-17T08:11:21.385295+0000","last_fullsized":"2024-09-17T08:11:21.385295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846374+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:11:27.846374+0000","last_peered":"2024-09-17T08:11:27.846374+0000","last_clean":"2024-09-17T08:11:27.846374+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:27.846374+0000","last_undegraded":"2024-09-17T08:11:27.846374+0000","last_fullsized":"2024-09-17T08:11:27.846374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:20.492012+0000","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779242+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:21.779242+0000","last_peered":"2024-09-17T08:11:21.779242+0000","last_clean":"2024-09-17T08:11:21.779242+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:21.779242+0000","last_undegraded":"2024-09-17T08:11:21.779242+0000","last_fullsized":"2024-09-17T08:11:21.779242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386028+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:21.386028+0000","last_peered":"2024-09-17T08:11:21.386028+0000","last_clean":"2024-09-17T08:11:21.386028+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:21.386028+0000","last_undegraded":"2024-09-17T08:11:21.386028+0000","last_fullsized":"2024-09-17T08:11:21.386028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386483+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:21.386483+0000","last_peered":"2024-09-17T08:11:21.386483+0000","last_clean":"2024-09-17T08:11:21.386483+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:21.386483+0000","last_undegraded":"2024-09-17T08:11:21.386483+0000","last_fullsized":"2024-09-17T08:11:21.386483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846689+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:11:27.846689+0000","last_peered":"2024-09-17T08:11:27.846689+0000","last_clean":"2024-09-17T08:11:27.846689+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:27.846689+0000","last_undegraded":"2024-09-17T08:11:27.846689+0000","last_fullsized":"2024-09-17T08:11:27.846689+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.126060+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:11:27.126060+0000","last_peered":"2024-09-17T08:11:27.126060+0000","last_clean":"2024-09-17T08:11:27.126060+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:27.126060+0000","last_undegraded":"2024-09-17T08:11:27.126060+0000","last_fullsized":"2024-09-17T08:11:27.126060+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.618404+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:11:25.618404+0000","last_peered":"2024-09-17T08:11:25.618404+0000","last_clean":"2024-09-17T08:11:25.618404+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:25.618404+0000","last_undegraded":"2024-09-17T08:11:25.618404+0000","last_fullsized":"2024-09-17T08:11:25.618404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.305742+0000","last_change":"2024-09-17T08:11:31.305742+0000","last_active":"2024-09-17T08:11:31.305742+0000","last_peered":"2024-09-17T08:11:31.305742+0000","last_clean":"2024-09-17T08:11:31.305742+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:31.305742+0000","last_undegraded":"2024-09-17T08:11:31.305742+0000","last_fullsized":"2024-09-17T08:11:31.305742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.305634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.305634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.476701+0000","scrub_duration":0.001519353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679426+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:21.679426+0000","last_peered":"2024-09-17T08:11:21.679426+0000","last_clean":"2024-09-17T08:11:21.679426+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:21.679426+0000","last_undegraded":"2024-09-17T08:11:21.679426+0000","last_fullsized":"2024-09-17T08:11:21.679426+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385338+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:21.385338+0000","last_peered":"2024-09-17T08:11:21.385338+0000","last_clean":"2024-09-17T08:11:21.385338+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:21.385338+0000","last_undegraded":"2024-09-17T08:11:21.385338+0000","last_fullsized":"2024-09-17T08:11:21.385338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778438+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:21.778438+0000","last_peered":"2024-09-17T08:11:21.778438+0000","last_clean":"2024-09-17T08:11:21.778438+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:21.778438+0000","last_undegraded":"2024-09-17T08:11:21.778438+0000","last_fullsized":"2024-09-17T08:11:21.778438+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383170+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:21.383170+0000","last_peered":"2024-09-17T08:11:21.383170+0000","last_clean":"2024-09-17T08:11:21.383170+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:21.383170+0000","last_undegraded":"2024-09-17T08:11:21.383170+0000","last_fullsized":"2024-09-17T08:11:21.383170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846375+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:27.846375+0000","last_peered":"2024-09-17T08:11:27.846375+0000","last_clean":"2024-09-17T08:11:27.846375+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:27.846375+0000","last_undegraded":"2024-09-17T08:11:27.846375+0000","last_fullsized":"2024-09-17T08:11:27.846375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778188+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:21.778188+0000","last_peered":"2024-09-17T08:11:21.778188+0000","last_clean":"2024-09-17T08:11:21.778188+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:21.778188+0000","last_undegraded":"2024-09-17T08:11:21.778188+0000","last_fullsized":"2024-09-17T08:11:21.778188+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384863+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:21.384863+0000","last_peered":"2024-09-17T08:11:21.384863+0000","last_clean":"2024-09-17T08:11:21.384863+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:21.384863+0000","last_undegraded":"2024-09-17T08:11:21.384863+0000","last_fullsized":"2024-09-17T08:11:21.384863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385597+0000","last_change":"2024-09-17T08:09:54.498648+0000","last_active":"2024-09-17T08:11:21.385597+0000","last_peered":"2024-09-17T08:11:21.385597+0000","last_clean":"2024-09-17T08:11:21.385597+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:21.385597+0000","last_undegraded":"2024-09-17T08:11:21.385597+0000","last_fullsized":"2024-09-17T08:11:21.385597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778204+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:21.778204+0000","last_peered":"2024-09-17T08:11:21.778204+0000","last_clean":"2024-09-17T08:11:21.778204+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:21.778204+0000","last_undegraded":"2024-09-17T08:11:21.778204+0000","last_fullsized":"2024-09-17T08:11:21.778204+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778835+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:21.778835+0000","last_peered":"2024-09-17T08:11:21.778835+0000","last_clean":"2024-09-17T08:11:21.778835+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:21.778835+0000","last_undegraded":"2024-09-17T08:11:21.778835+0000","last_fullsized":"2024-09-17T08:11:21.778835+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430043+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:21.430043+0000","last_peered":"2024-09-17T08:11:21.430043+0000","last_clean":"2024-09-17T08:11:21.430043+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:21.430043+0000","last_undegraded":"2024-09-17T08:11:21.430043+0000","last_fullsized":"2024-09-17T08:11:21.430043+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779949+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:21.779949+0000","last_peered":"2024-09-17T08:11:21.779949+0000","last_clean":"2024-09-17T08:11:21.779949+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:21.779949+0000","last_undegraded":"2024-09-17T08:11:21.779949+0000","last_fullsized":"2024-09-17T08:11:21.779949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.433770+0000","last_change":"2024-09-17T08:11:28.433770+0000","last_active":"2024-09-17T08:11:28.433770+0000","last_peered":"2024-09-17T08:11:28.433770+0000","last_clean":"2024-09-17T08:11:28.433770+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:28.433770+0000","last_undegraded":"2024-09-17T08:11:28.433770+0000","last_fullsized":"2024-09-17T08:11:28.433770+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.433666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.433666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.684654+0000","scrub_duration":0.001195822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386242+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:21.386242+0000","last_peered":"2024-09-17T08:11:21.386242+0000","last_clean":"2024-09-17T08:11:21.386242+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:21.386242+0000","last_undegraded":"2024-09-17T08:11:21.386242+0000","last_fullsized":"2024-09-17T08:11:21.386242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385176+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:11:21.385176+0000","last_peered":"2024-09-17T08:11:21.385176+0000","last_clean":"2024-09-17T08:11:21.385176+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:21.385176+0000","last_undegraded":"2024-09-17T08:11:21.385176+0000","last_fullsized":"2024-09-17T08:11:21.385176+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385294+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:11:21.385294+0000","last_peered":"2024-09-17T08:11:21.385294+0000","last_clean":"2024-09-17T08:11:21.385294+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:21.385294+0000","last_undegraded":"2024-09-17T08:11:21.385294+0000","last_fullsized":"2024-09-17T08:11:21.385294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846363+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:27.846363+0000","last_peered":"2024-09-17T08:11:27.846363+0000","last_clean":"2024-09-17T08:11:27.846363+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:27.846363+0000","last_undegraded":"2024-09-17T08:11:27.846363+0000","last_fullsized":"2024-09-17T08:11:27.846363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.080294+0000","last_change":"2024-09-17T08:11:32.080416+0000","last_active":"2024-09-17T08:11:32.080294+0000","last_peered":"2024-09-17T08:11:32.080294+0000","last_clean":"2024-09-17T08:11:32.080294+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:32.080294+0000","last_undegraded":"2024-09-17T08:11:32.080294+0000","last_fullsized":"2024-09-17T08:11:32.080294+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.080269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.080269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.073092+0000","scrub_duration":0.001335506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":107,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846378+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:27.846378+0000","last_peered":"2024-09-17T08:11:27.846378+0000","last_clean":"2024-09-17T08:11:27.846378+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:27.846378+0000","last_undegraded":"2024-09-17T08:11:27.846378+0000","last_fullsized":"2024-09-17T08:11:27.846378+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679157+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:21.679157+0000","last_peered":"2024-09-17T08:11:21.679157+0000","last_clean":"2024-09-17T08:11:21.679157+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:21.679157+0000","last_undegraded":"2024-09-17T08:11:21.679157+0000","last_fullsized":"2024-09-17T08:11:21.679157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430138+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:21.430138+0000","last_peered":"2024-09-17T08:11:21.430138+0000","last_clean":"2024-09-17T08:11:21.430138+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:21.430138+0000","last_undegraded":"2024-09-17T08:11:21.430138+0000","last_fullsized":"2024-09-17T08:11:21.430138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386001+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:21.386001+0000","last_peered":"2024-09-17T08:11:21.386001+0000","last_clean":"2024-09-17T08:11:21.386001+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:21.386001+0000","last_undegraded":"2024-09-17T08:11:21.386001+0000","last_fullsized":"2024-09-17T08:11:21.386001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":2,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:10:01.782568+0000","last_change":"2024-09-17T08:11:32.218042+0000","last_active":"2024-09-17T08:11:32.218042+0000","last_peered":"2024-09-17T08:10:01.782568+0000","last_clean":"2024-09-17T08:10:01.782568+0000","last_became_active":"2024-09-17T08:09:56.503476+0000","last_became_peered":"2024-09-17T08:09:56.503476+0000","last_unstale":"2024-09-17T08:10:01.782568+0000","last_undegraded":"2024-09-17T08:10:01.782568+0000","last_fullsized":"2024-09-17T08:10:01.782568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":141,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426382+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:31.426382+0000","last_peered":"2024-09-17T08:11:31.426382+0000","last_clean":"2024-09-17T08:11:31.426382+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:31.426382+0000","last_undegraded":"2024-09-17T08:11:31.426382+0000","last_fullsized":"2024-09-17T08:11:31.426382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.081976+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:11:23.081976+0000","last_peered":"2024-09-17T08:11:23.081976+0000","last_clean":"2024-09-17T08:11:23.081976+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:23.081976+0000","last_undegraded":"2024-09-17T08:11:23.081976+0000","last_fullsized":"2024-09-17T08:11:23.081976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.111013+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:11:27.111013+0000","last_peered":"2024-09-17T08:11:27.111013+0000","last_clean":"2024-09-17T08:11:27.111013+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:27.111013+0000","last_undegraded":"2024-09-17T08:11:27.111013+0000","last_fullsized":"2024-09-17T08:11:27.111013+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779787+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:11:21.779787+0000","last_peered":"2024-09-17T08:11:21.779787+0000","last_clean":"2024-09-17T08:11:21.779787+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:21.779787+0000","last_undegraded":"2024-09-17T08:11:21.779787+0000","last_fullsized":"2024-09-17T08:11:21.779787+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430304+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:21.430304+0000","last_peered":"2024-09-17T08:11:21.430304+0000","last_clean":"2024-09-17T08:11:21.430304+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:21.430304+0000","last_undegraded":"2024-09-17T08:11:21.430304+0000","last_fullsized":"2024-09-17T08:11:21.430304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:28.111674+0000","last_change":"2024-09-17T08:11:28.111674+0000","last_active":"2024-09-17T08:11:28.111674+0000","last_peered":"2024-09-17T08:11:28.111674+0000","last_clean":"2024-09-17T08:11:28.111674+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:28.111674+0000","last_undegraded":"2024-09-17T08:11:28.111674+0000","last_fullsized":"2024-09-17T08:11:28.111674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:11:28.111569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.111569+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.306471+0000","scrub_duration":0.0016978239999999999,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778974+0000","last_change":"2024-09-17T08:09:54.507600+0000","last_active":"2024-09-17T08:11:21.778974+0000","last_peered":"2024-09-17T08:11:21.778974+0000","last_clean":"2024-09-17T08:11:21.778974+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:21.778974+0000","last_undegraded":"2024-09-17T08:11:21.778974+0000","last_fullsized":"2024-09-17T08:11:21.778974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679450+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:11:21.679450+0000","last_peered":"2024-09-17T08:11:21.679450+0000","last_clean":"2024-09-17T08:11:21.679450+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:21.679450+0000","last_undegraded":"2024-09-17T08:11:21.679450+0000","last_fullsized":"2024-09-17T08:11:21.679450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384972+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:21.384972+0000","last_peered":"2024-09-17T08:11:21.384972+0000","last_clean":"2024-09-17T08:11:21.384972+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:21.384972+0000","last_undegraded":"2024-09-17T08:11:21.384972+0000","last_fullsized":"2024-09-17T08:11:21.384972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382705+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:21.382705+0000","last_peered":"2024-09-17T08:11:21.382705+0000","last_clean":"2024-09-17T08:11:21.382705+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:21.382705+0000","last_undegraded":"2024-09-17T08:11:21.382705+0000","last_fullsized":"2024-09-17T08:11:21.382705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385117+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:21.385117+0000","last_peered":"2024-09-17T08:11:21.385117+0000","last_clean":"2024-09-17T08:11:21.385117+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:21.385117+0000","last_undegraded":"2024-09-17T08:11:21.385117+0000","last_fullsized":"2024-09-17T08:11:21.385117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679306+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:11:21.679306+0000","last_peered":"2024-09-17T08:11:21.679306+0000","last_clean":"2024-09-17T08:11:21.679306+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:21.679306+0000","last_undegraded":"2024-09-17T08:11:21.679306+0000","last_fullsized":"2024-09-17T08:11:21.679306+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":292,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779059+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:21.779059+0000","last_peered":"2024-09-17T08:11:21.779059+0000","last_clean":"2024-09-17T08:11:21.779059+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:21.779059+0000","last_undegraded":"2024-09-17T08:11:21.779059+0000","last_fullsized":"2024-09-17T08:11:21.779059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779640+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:21.779640+0000","last_peered":"2024-09-17T08:11:21.779640+0000","last_clean":"2024-09-17T08:11:21.779640+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:21.779640+0000","last_undegraded":"2024-09-17T08:11:21.779640+0000","last_fullsized":"2024-09-17T08:11:21.779640+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679510+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:21.679510+0000","last_peered":"2024-09-17T08:11:21.679510+0000","last_clean":"2024-09-17T08:11:21.679510+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:21.679510+0000","last_undegraded":"2024-09-17T08:11:21.679510+0000","last_fullsized":"2024-09-17T08:11:21.679510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.389593+0000","last_change":"2024-09-17T08:11:27.389593+0000","last_active":"2024-09-17T08:11:27.389593+0000","last_peered":"2024-09-17T08:11:27.389593+0000","last_clean":"2024-09-17T08:11:27.389593+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:27.389593+0000","last_undegraded":"2024-09-17T08:11:27.389593+0000","last_fullsized":"2024-09-17T08:11:27.389593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:27.389490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.389490+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.830431+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:30.055058+0000","last_change":"2024-09-17T08:11:30.055058+0000","last_active":"2024-09-17T08:11:30.055058+0000","last_peered":"2024-09-17T08:11:30.055058+0000","last_clean":"2024-09-17T08:11:30.055058+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:30.055058+0000","last_undegraded":"2024-09-17T08:11:30.055058+0000","last_fullsized":"2024-09-17T08:11:30.055058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:30.054968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.054968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.938144+0000","scrub_duration":0.0018411460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777525+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:21.777525+0000","last_peered":"2024-09-17T08:11:21.777525+0000","last_clean":"2024-09-17T08:11:21.777525+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:21.777525+0000","last_undegraded":"2024-09-17T08:11:21.777525+0000","last_fullsized":"2024-09-17T08:11:21.777525+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.312687+0000","last_change":"2024-09-17T08:11:27.312687+0000","last_active":"2024-09-17T08:11:27.312687+0000","last_peered":"2024-09-17T08:11:27.312687+0000","last_clean":"2024-09-17T08:11:27.312687+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:27.312687+0000","last_undegraded":"2024-09-17T08:11:27.312687+0000","last_fullsized":"2024-09-17T08:11:27.312687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.312595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.312595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.098135+0000","scrub_duration":0.001815178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:30.115032+0000","last_change":"2024-09-17T08:11:30.115032+0000","last_active":"2024-09-17T08:11:30.115032+0000","last_peered":"2024-09-17T08:11:30.115032+0000","last_clean":"2024-09-17T08:11:30.115032+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:30.115032+0000","last_undegraded":"2024-09-17T08:11:30.115032+0000","last_fullsized":"2024-09-17T08:11:30.115032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:30.114921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.114921+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.519251+0000","scrub_duration":0.0019919109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386200+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:11:21.386200+0000","last_peered":"2024-09-17T08:11:21.386200+0000","last_clean":"2024-09-17T08:11:21.386200+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:21.386200+0000","last_undegraded":"2024-09-17T08:11:21.386200+0000","last_fullsized":"2024-09-17T08:11:21.386200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.124631+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:11:25.124631+0000","last_peered":"2024-09-17T08:11:25.124631+0000","last_clean":"2024-09-17T08:11:25.124631+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:25.124631+0000","last_undegraded":"2024-09-17T08:11:25.124631+0000","last_fullsized":"2024-09-17T08:11:25.124631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.073527+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:11:23.073527+0000","last_peered":"2024-09-17T08:11:23.073527+0000","last_clean":"2024-09-17T08:11:23.073527+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:23.073527+0000","last_undegraded":"2024-09-17T08:11:23.073527+0000","last_fullsized":"2024-09-17T08:11:23.073527+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.079173+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:11:23.079173+0000","last_peered":"2024-09-17T08:11:23.079173+0000","last_clean":"2024-09-17T08:11:23.079173+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:23.079173+0000","last_undegraded":"2024-09-17T08:11:23.079173+0000","last_fullsized":"2024-09-17T08:11:23.079173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777771+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:21.777771+0000","last_peered":"2024-09-17T08:11:21.777771+0000","last_clean":"2024-09-17T08:11:21.777771+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:21.777771+0000","last_undegraded":"2024-09-17T08:11:21.777771+0000","last_fullsized":"2024-09-17T08:11:21.777771+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846632+0000","last_change":"2024-09-17T08:09:58.513726+0000","last_active":"2024-09-17T08:11:27.846632+0000","last_peered":"2024-09-17T08:11:27.846632+0000","last_clean":"2024-09-17T08:11:27.846632+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:27.846632+0000","last_undegraded":"2024-09-17T08:11:27.846632+0000","last_fullsized":"2024-09-17T08:11:27.846632+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.089111+0000","last_change":"2024-09-17T08:11:32.089192+0000","last_active":"2024-09-17T08:11:32.089111+0000","last_peered":"2024-09-17T08:11:32.089111+0000","last_clean":"2024-09-17T08:11:32.089111+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:32.089111+0000","last_undegraded":"2024-09-17T08:11:32.089111+0000","last_fullsized":"2024-09-17T08:11:32.089111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.089096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.089096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.048086+0000","scrub_duration":0.0013942010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778331+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:21.778331+0000","last_peered":"2024-09-17T08:11:21.778331+0000","last_clean":"2024-09-17T08:11:21.778331+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:21.778331+0000","last_undegraded":"2024-09-17T08:11:21.778331+0000","last_fullsized":"2024-09-17T08:11:21.778331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778456+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:21.778456+0000","last_peered":"2024-09-17T08:11:21.778456+0000","last_clean":"2024-09-17T08:11:21.778456+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:21.778456+0000","last_undegraded":"2024-09-17T08:11:21.778456+0000","last_fullsized":"2024-09-17T08:11:21.778456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777344+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:21.777344+0000","last_peered":"2024-09-17T08:11:21.777344+0000","last_clean":"2024-09-17T08:11:21.777344+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:21.777344+0000","last_undegraded":"2024-09-17T08:11:21.777344+0000","last_fullsized":"2024-09-17T08:11:21.777344+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385339+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:21.385339+0000","last_peered":"2024-09-17T08:11:21.385339+0000","last_clean":"2024-09-17T08:11:21.385339+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:21.385339+0000","last_undegraded":"2024-09-17T08:11:21.385339+0000","last_fullsized":"2024-09-17T08:11:21.385339+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:22.027206+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:11:22.027206+0000","last_peered":"2024-09-17T08:11:22.027206+0000","last_clean":"2024-09-17T08:11:22.027206+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:22.027206+0000","last_undegraded":"2024-09-17T08:11:22.027206+0000","last_fullsized":"2024-09-17T08:11:22.027206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777417+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:21.777417+0000","last_peered":"2024-09-17T08:11:21.777417+0000","last_clean":"2024-09-17T08:11:21.777417+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:21.777417+0000","last_undegraded":"2024-09-17T08:11:21.777417+0000","last_fullsized":"2024-09-17T08:11:21.777417+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386023+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:21.386023+0000","last_peered":"2024-09-17T08:11:21.386023+0000","last_clean":"2024-09-17T08:11:21.386023+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:21.386023+0000","last_undegraded":"2024-09-17T08:11:21.386023+0000","last_fullsized":"2024-09-17T08:11:21.386023+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.777424+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:21.777424+0000","last_peered":"2024-09-17T08:11:21.777424+0000","last_clean":"2024-09-17T08:11:21.777424+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:21.777424+0000","last_undegraded":"2024-09-17T08:11:21.777424+0000","last_fullsized":"2024-09-17T08:11:21.777424+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382190+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:11:21.382190+0000","last_peered":"2024-09-17T08:11:21.382190+0000","last_clean":"2024-09-17T08:11:21.382190+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:21.382190+0000","last_undegraded":"2024-09-17T08:11:21.382190+0000","last_fullsized":"2024-09-17T08:11:21.382190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:23.028718+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:11:23.028718+0000","last_peered":"2024-09-17T08:11:23.028718+0000","last_clean":"2024-09-17T08:11:23.028718+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:23.028718+0000","last_undegraded":"2024-09-17T08:11:23.028718+0000","last_fullsized":"2024-09-17T08:11:23.028718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679217+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:21.679217+0000","last_peered":"2024-09-17T08:11:21.679217+0000","last_clean":"2024-09-17T08:11:21.679217+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:21.679217+0000","last_undegraded":"2024-09-17T08:11:21.679217+0000","last_fullsized":"2024-09-17T08:11:21.679217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779725+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:21.779725+0000","last_peered":"2024-09-17T08:11:21.779725+0000","last_clean":"2024-09-17T08:11:21.779725+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:21.779725+0000","last_undegraded":"2024-09-17T08:11:21.779725+0000","last_fullsized":"2024-09-17T08:11:21.779725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:29.979585+0000","last_change":"2024-09-17T08:11:29.979585+0000","last_active":"2024-09-17T08:11:29.979585+0000","last_peered":"2024-09-17T08:11:29.979585+0000","last_clean":"2024-09-17T08:11:29.979585+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:29.979585+0000","last_undegraded":"2024-09-17T08:11:29.979585+0000","last_fullsized":"2024-09-17T08:11:29.979585+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:29.979461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.979461+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.112016+0000","scrub_duration":0.0029734470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.431193+0000","last_change":"2024-09-17T08:11:31.431465+0000","last_active":"2024-09-17T08:11:31.431193+0000","last_peered":"2024-09-17T08:11:21.679113+0000","last_clean":"2024-09-17T08:11:21.679113+0000","last_became_active":"2024-09-17T08:09:54.501179+0000","last_became_peered":"2024-09-17T08:09:54.501179+0000","last_unstale":"2024-09-17T08:11:31.431193+0000","last_undegraded":"2024-09-17T08:11:31.431193+0000","last_fullsized":"2024-09-17T08:11:31.431193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.453505+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846733+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:27.846733+0000","last_peered":"2024-09-17T08:11:27.846733+0000","last_clean":"2024-09-17T08:11:27.846733+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:27.846733+0000","last_undegraded":"2024-09-17T08:11:27.846733+0000","last_fullsized":"2024-09-17T08:11:27.846733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:24.057092+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:11:24.057092+0000","last_peered":"2024-09-17T08:11:24.057092+0000","last_clean":"2024-09-17T08:11:24.057092+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:24.057092+0000","last_undegraded":"2024-09-17T08:11:24.057092+0000","last_fullsized":"2024-09-17T08:11:24.057092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679320+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:21.679320+0000","last_peered":"2024-09-17T08:11:21.679320+0000","last_clean":"2024-09-17T08:11:21.679320+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:21.679320+0000","last_undegraded":"2024-09-17T08:11:21.679320+0000","last_fullsized":"2024-09-17T08:11:21.679320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430190+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:21.430190+0000","last_peered":"2024-09-17T08:11:21.430190+0000","last_clean":"2024-09-17T08:11:21.430190+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:21.430190+0000","last_undegraded":"2024-09-17T08:11:21.430190+0000","last_fullsized":"2024-09-17T08:11:21.430190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846621+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:27.846621+0000","last_peered":"2024-09-17T08:11:27.846621+0000","last_clean":"2024-09-17T08:11:27.846621+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:27.846621+0000","last_undegraded":"2024-09-17T08:11:27.846621+0000","last_fullsized":"2024-09-17T08:11:27.846621+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778316+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:21.778316+0000","last_peered":"2024-09-17T08:11:21.778316+0000","last_clean":"2024-09-17T08:11:21.778316+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:21.778316+0000","last_undegraded":"2024-09-17T08:11:21.778316+0000","last_fullsized":"2024-09-17T08:11:21.778316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385106+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:21.385106+0000","last_peered":"2024-09-17T08:11:21.385106+0000","last_clean":"2024-09-17T08:11:21.385106+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:21.385106+0000","last_undegraded":"2024-09-17T08:11:21.385106+0000","last_fullsized":"2024-09-17T08:11:21.385106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382583+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:21.382583+0000","last_peered":"2024-09-17T08:11:21.382583+0000","last_clean":"2024-09-17T08:11:21.382583+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:21.382583+0000","last_undegraded":"2024-09-17T08:11:21.382583+0000","last_fullsized":"2024-09-17T08:11:21.382583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.019796+0000","last_change":"2024-09-17T08:11:31.019796+0000","last_active":"2024-09-17T08:11:31.019796+0000","last_peered":"2024-09-17T08:11:31.019796+0000","last_clean":"2024-09-17T08:11:31.019796+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:31.019796+0000","last_undegraded":"2024-09-17T08:11:31.019796+0000","last_fullsized":"2024-09-17T08:11:31.019796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.019695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.019695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.782145+0000","scrub_duration":0.001088767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.429633+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:11:21.429633+0000","last_peered":"2024-09-17T08:11:21.429633+0000","last_clean":"2024-09-17T08:11:21.429633+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:21.429633+0000","last_undegraded":"2024-09-17T08:11:21.429633+0000","last_fullsized":"2024-09-17T08:11:21.429633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778458+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:21.778458+0000","last_peered":"2024-09-17T08:11:21.778458+0000","last_clean":"2024-09-17T08:11:21.778458+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:21.778458+0000","last_undegraded":"2024-09-17T08:11:21.778458+0000","last_fullsized":"2024-09-17T08:11:21.778458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.933581+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:11:26.933581+0000","last_peered":"2024-09-17T08:11:26.933581+0000","last_clean":"2024-09-17T08:11:26.933581+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:26.933581+0000","last_undegraded":"2024-09-17T08:11:26.933581+0000","last_fullsized":"2024-09-17T08:11:26.933581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384983+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:21.384983+0000","last_peered":"2024-09-17T08:11:21.384983+0000","last_clean":"2024-09-17T08:11:21.384983+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:21.384983+0000","last_undegraded":"2024-09-17T08:11:21.384983+0000","last_fullsized":"2024-09-17T08:11:21.384983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384740+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:21.384740+0000","last_peered":"2024-09-17T08:11:21.384740+0000","last_clean":"2024-09-17T08:11:21.384740+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:21.384740+0000","last_undegraded":"2024-09-17T08:11:21.384740+0000","last_fullsized":"2024-09-17T08:11:21.384740+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384767+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:21.384767+0000","last_peered":"2024-09-17T08:11:21.384767+0000","last_clean":"2024-09-17T08:11:21.384767+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:21.384767+0000","last_undegraded":"2024-09-17T08:11:21.384767+0000","last_fullsized":"2024-09-17T08:11:21.384767+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.679371+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:21.679371+0000","last_peered":"2024-09-17T08:11:21.679371+0000","last_clean":"2024-09-17T08:11:21.679371+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:21.679371+0000","last_undegraded":"2024-09-17T08:11:21.679371+0000","last_fullsized":"2024-09-17T08:11:21.679371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.381919+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:21.381919+0000","last_peered":"2024-09-17T08:11:21.381919+0000","last_clean":"2024-09-17T08:11:21.381919+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:21.381919+0000","last_undegraded":"2024-09-17T08:11:21.381919+0000","last_fullsized":"2024-09-17T08:11:21.381919+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778483+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:21.778483+0000","last_peered":"2024-09-17T08:11:21.778483+0000","last_clean":"2024-09-17T08:11:21.778483+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:21.778483+0000","last_undegraded":"2024-09-17T08:11:21.778483+0000","last_fullsized":"2024-09-17T08:11:21.778483+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.384919+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:21.384919+0000","last_peered":"2024-09-17T08:11:21.384919+0000","last_clean":"2024-09-17T08:11:21.384919+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:21.384919+0000","last_undegraded":"2024-09-17T08:11:21.384919+0000","last_fullsized":"2024-09-17T08:11:21.384919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383122+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:11:21.383122+0000","last_peered":"2024-09-17T08:11:21.383122+0000","last_clean":"2024-09-17T08:11:21.383122+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:21.383122+0000","last_undegraded":"2024-09-17T08:11:21.383122+0000","last_fullsized":"2024-09-17T08:11:21.383122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778595+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:21.778595+0000","last_peered":"2024-09-17T08:11:21.778595+0000","last_clean":"2024-09-17T08:11:21.778595+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:21.778595+0000","last_undegraded":"2024-09-17T08:11:21.778595+0000","last_fullsized":"2024-09-17T08:11:21.778595+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385277+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:21.385277+0000","last_peered":"2024-09-17T08:11:21.385277+0000","last_clean":"2024-09-17T08:11:21.385277+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:21.385277+0000","last_undegraded":"2024-09-17T08:11:21.385277+0000","last_fullsized":"2024-09-17T08:11:21.385277+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778140+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:21.778140+0000","last_peered":"2024-09-17T08:11:21.778140+0000","last_clean":"2024-09-17T08:11:21.778140+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:21.778140+0000","last_undegraded":"2024-09-17T08:11:21.778140+0000","last_fullsized":"2024-09-17T08:11:21.778140+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.678796+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:21.678796+0000","last_peered":"2024-09-17T08:11:21.678796+0000","last_clean":"2024-09-17T08:11:21.678796+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:21.678796+0000","last_undegraded":"2024-09-17T08:11:21.678796+0000","last_fullsized":"2024-09-17T08:11:21.678796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:25.077750+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:11:25.077750+0000","last_peered":"2024-09-17T08:11:25.077750+0000","last_clean":"2024-09-17T08:11:25.077750+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:25.077750+0000","last_undegraded":"2024-09-17T08:11:25.077750+0000","last_fullsized":"2024-09-17T08:11:25.077750+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:27.846285+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:11:27.846285+0000","last_peered":"2024-09-17T08:11:27.846285+0000","last_clean":"2024-09-17T08:11:27.846285+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:27.846285+0000","last_undegraded":"2024-09-17T08:11:27.846285+0000","last_fullsized":"2024-09-17T08:11:27.846285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385674+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:11:21.385674+0000","last_peered":"2024-09-17T08:11:21.385674+0000","last_clean":"2024-09-17T08:11:21.385674+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:21.385674+0000","last_undegraded":"2024-09-17T08:11:21.385674+0000","last_fullsized":"2024-09-17T08:11:21.385674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:26.016945+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:11:26.016945+0000","last_peered":"2024-09-17T08:11:26.016945+0000","last_clean":"2024-09-17T08:11:26.016945+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:26.016945+0000","last_undegraded":"2024-09-17T08:11:26.016945+0000","last_fullsized":"2024-09-17T08:11:26.016945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.383090+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:21.383090+0000","last_peered":"2024-09-17T08:11:21.383090+0000","last_clean":"2024-09-17T08:11:21.383090+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:21.383090+0000","last_undegraded":"2024-09-17T08:11:21.383090+0000","last_fullsized":"2024-09-17T08:11:21.383090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.430171+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:21.430171+0000","last_peered":"2024-09-17T08:11:21.430171+0000","last_clean":"2024-09-17T08:11:21.430171+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:21.430171+0000","last_undegraded":"2024-09-17T08:11:21.430171+0000","last_fullsized":"2024-09-17T08:11:21.430171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.779316+0000","last_change":"2024-09-17T08:10:00.598000+0000","last_active":"2024-09-17T08:11:21.779316+0000","last_peered":"2024-09-17T08:11:21.779316+0000","last_clean":"2024-09-17T08:11:21.779316+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:21.779316+0000","last_undegraded":"2024-09-17T08:11:21.779316+0000","last_fullsized":"2024-09-17T08:11:21.779316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.382413+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:21.382413+0000","last_peered":"2024-09-17T08:11:21.382413+0000","last_clean":"2024-09-17T08:11:21.382413+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:21.382413+0000","last_undegraded":"2024-09-17T08:11:21.382413+0000","last_fullsized":"2024-09-17T08:11:21.382413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"72'15","reported_seq":55,"reported_epoch":86,"state":"peering","last_fresh":"2024-09-17T08:11:31.428010+0000","last_change":"2024-09-17T08:11:31.428155+0000","last_active":"2024-09-17T08:11:31.428010+0000","last_peered":"2024-09-17T08:11:21.777584+0000","last_clean":"2024-09-17T08:11:21.777584+0000","last_became_active":"2024-09-17T08:09:56.504114+0000","last_became_peered":"2024-09-17T08:09:56.504114+0000","last_unstale":"2024-09-17T08:11:31.428010+0000","last_undegraded":"2024-09-17T08:11:31.428010+0000","last_fullsized":"2024-09-17T08:11:31.428010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385933+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:21.385933+0000","last_peered":"2024-09-17T08:11:21.385933+0000","last_clean":"2024-09-17T08:11:21.385933+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:21.385933+0000","last_undegraded":"2024-09-17T08:11:21.385933+0000","last_fullsized":"2024-09-17T08:11:21.385933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.386027+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:21.386027+0000","last_peered":"2024-09-17T08:11:21.386027+0000","last_clean":"2024-09-17T08:11:21.386027+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:21.386027+0000","last_undegraded":"2024-09-17T08:11:21.386027+0000","last_fullsized":"2024-09-17T08:11:21.386027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385560+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:21.385560+0000","last_peered":"2024-09-17T08:11:21.385560+0000","last_clean":"2024-09-17T08:11:21.385560+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:21.385560+0000","last_undegraded":"2024-09-17T08:11:21.385560+0000","last_fullsized":"2024-09-17T08:11:21.385560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.385927+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:21.385927+0000","last_peered":"2024-09-17T08:11:21.385927+0000","last_clean":"2024-09-17T08:11:21.385927+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:21.385927+0000","last_undegraded":"2024-09-17T08:11:21.385927+0000","last_fullsized":"2024-09-17T08:11:21.385927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T08:11:21.778521+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:21.778521+0000","last_peered":"2024-09-17T08:11:21.778521+0000","last_clean":"2024-09-17T08:11:21.778521+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:21.778521+0000","last_undegraded":"2024-09-17T08:11:21.778521+0000","last_fullsized":"2024-09-17T08:11:21.778521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070489,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1277952,"data_stored":913439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.495}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46300000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.443}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.372}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.434}]}]},{"osd":6,"up_from":50,"seq":214748364830,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32348,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712804,"statfs":{"total":95995035648,"available":95961911296,"internally_reserved":0,"allocated":1265664,"data_stored":913030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.69399999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67600000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.625},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66500000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.317,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.374},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.38700000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.34799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39500000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.372}]}]},{"osd":5,"up_from":41,"seq":176093659171,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31780,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713372,"statfs":{"total":95995035648,"available":95962492928,"internally_reserved":0,"allocated":679936,"data_stored":329860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.44}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.621}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51400000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717448,"statfs":{"total":95995035648,"available":95966666752,"internally_reserved":0,"allocated":700416,"data_stored":331093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.629}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.432},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.61099999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117036,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717436,"statfs":{"total":95995035648,"available":95966654464,"internally_reserved":0,"allocated":712704,"data_stored":330570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.37},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83299999999999996,"5min":0.873,"15min":0.873},"last":0.435}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55200000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717464,"statfs":{"total":95995035648,"available":95966683136,"internally_reserved":0,"allocated":684032,"data_stored":330642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.254},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.318},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.23499999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.497}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.749,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.441}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31828,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713324,"statfs":{"total":95995035648,"available":95962443776,"internally_reserved":0,"allocated":729088,"data_stored":331380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.191,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.39400000000000002,"5min":0.625,"15min":0.625},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.312,"5min":0.33300000000000002,"15min":0.30299999999999999},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46100000000000002,"5min":0.625,"15min":0.625},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85099999999999998,"5min":1.329,"15min":1.329},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1279999999999999,"5min":1.157,"15min":1.157},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.57099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28276,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716876,"statfs":{"total":95995035648,"available":95966081024,"internally_reserved":0,"allocated":1286144,"data_stored":921034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.216},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.23100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.315}]},{"osd":4,"last update":"Tue Sep 17 08:10:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.438},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.753,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47599999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.5},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:36.733 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-17T08:11:36.733 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.18', 'version': "72'9", 'reported_seq': 46, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-17T08:11:31.428910+0000', 'last_change': '2024-09-17T08:11:31.429094+0000', 'last_active': '2024-09-17T08:11:31.428910+0000', 'last_peered': '2024-09-17T08:11:21.777974+0000', 'last_clean': '2024-09-17T08:11:21.777974+0000', 'last_became_active': '2024-09-17T08:09:56.500692+0000', 'last_became_peered': '2024-09-17T08:09:56.500692+0000', 'last_unstale': '2024-09-17T08:11:31.428910+0000', 'last_undegraded': '2024-09-17T08:11:31.428910+0000', 'last_fullsized': '2024-09-17T08:11:31.428910+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 67, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_clean_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'objects_scrubbed': 0, 'log_size': 9, 'log_dups_size': 0, 'ondisk_log_size': 9, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'queued for scrub', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 110, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 12, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 17, 'num_read_kb': 11, 'num_write': 10, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6, 2], 'acting': [4, 6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2, 6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T08:11:36.733 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-17T08:11:36.733 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.7', 'version': "72'13", 'reported_seq': 2, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-17T08:10:01.782568+0000', 'last_change': '2024-09-17T08:11:32.218042+0000', 'last_active': '2024-09-17T08:11:32.218042+0000', 'last_peered': '2024-09-17T08:10:01.782568+0000', 'last_clean': '2024-09-17T08:10:01.782568+0000', 'last_became_active': '2024-09-17T08:09:56.503476+0000', 'last_became_peered': '2024-09-17T08:09:56.503476+0000', 'last_unstale': '2024-09-17T08:10:01.782568+0000', 'last_undegraded': '2024-09-17T08:10:01.782568+0000', 'last_fullsized': '2024-09-17T08:10:01.782568+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 67, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_clean_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'objects_scrubbed': 0, 'log_size': 13, 'log_dups_size': 0, 'ondisk_log_size': 13, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'queued for scrub', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 330, 'num_objects': 5, 'num_object_clones': 0, 'num_object_copies': 15, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 5, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5, 'num_whiteouts': 0, 'num_read': 29, 'num_read_kb': 18, 'num_write': 16, 'num_write_kb': 3, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5, 0], 'acting': [2, 5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T08:11:36.733 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-17T08:11:36.734 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 43, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-17T08:11:31.431193+0000', 'last_change': '2024-09-17T08:11:31.431465+0000', 'last_active': '2024-09-17T08:11:31.431193+0000', 'last_peered': '2024-09-17T08:11:21.679113+0000', 'last_clean': '2024-09-17T08:11:21.679113+0000', 'last_became_active': '2024-09-17T08:09:54.501179+0000', 'last_became_peered': '2024-09-17T08:09:54.501179+0000', 'last_unstale': '2024-09-17T08:11:31.431193+0000', 'last_undegraded': '2024-09-17T08:11:31.431193+0000', 'last_fullsized': '2024-09-17T08:11:31.431193+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 65, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T08:10:54.331799+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T08:09:53.481832+0000', 'last_clean_scrub_stamp': '2024-09-17T08:10:54.331799+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-17T08:12:14.453505+0000', 'scrub_duration': 0.002425285, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3, 2], 'acting': [5, 3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2, 3, 7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T08:11:36.734 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-17T08:11:36.734 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.1c', 'version': "72'15", 'reported_seq': 55, 'reported_epoch': 86, 'state': 'peering', 'last_fresh': '2024-09-17T08:11:31.428010+0000', 'last_change': '2024-09-17T08:11:31.428155+0000', 'last_active': '2024-09-17T08:11:31.428010+0000', 'last_peered': '2024-09-17T08:11:21.777584+0000', 'last_clean': '2024-09-17T08:11:21.777584+0000', 'last_became_active': '2024-09-17T08:09:56.504114+0000', 'last_became_peered': '2024-09-17T08:09:56.504114+0000', 'last_unstale': '2024-09-17T08:11:31.428010+0000', 'last_undegraded': '2024-09-17T08:11:31.428010+0000', 'last_fullsized': '2024-09-17T08:11:31.428010+0000', 'mapping_epoch': 86, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 67, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'last_clean_scrub_stamp': '2024-09-17T08:09:55.487348+0000', 'objects_scrubbed': 0, 'log_size': 15, 'log_dups_size': 0, 'ondisk_log_size': 15, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'queued for scrub', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 110, 'num_objects': 7, 'num_object_clones': 0, 'num_object_copies': 21, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7, 'num_whiteouts': 0, 'num_read': 26, 'num_read_kb': 17, 'num_write': 16, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5, 3], 'acting': [2, 5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 3, 5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T08:11:36.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-17T08:11:36.937 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 podman[66675]: 2024-09-17 08:11:36.725378726 +0000 UTC m=+2.721310606 container start f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.729Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T08:11:36.974 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T08:11:36.975 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T08:11:36.976 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T08:11:36.977 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T08:11:36.978 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b[66749]: ts=2024-09-17T08:11:36.730Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T08:11:36.979 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 bash[66675]: f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 2024-09-17T08:11:36.979 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:11:36 smithi173 systemd[1]: Started Ceph node-exporter.b for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:37.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:11:37.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:11:37.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: 6.1e deep-scrub starts 2024-09-17T08:11:37.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: 6.1e deep-scrub ok 2024-09-17T08:11:37.707 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:11:37.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:37 smithi173 ceph-mon[33187]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: 6.1e deep-scrub starts 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: 6.1e deep-scrub ok 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[40649]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: 6.1e deep-scrub starts 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: 6.1e deep-scrub ok 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:11:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:37 smithi079 ceph-mon[29565]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:38.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:38 smithi173 ceph-mon[33187]: 5.4 deep-scrub starts 2024-09-17T08:11:38.636 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:38 smithi173 ceph-mon[33187]: 5.4 deep-scrub ok 2024-09-17T08:11:38.637 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:38 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:11:38.637 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:38 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:11:38.854 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:38.854 INFO:teuthology.orchestra.run.smithi079.stdout:{"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1726560581,"num_in_osds":8,"osd_in_since":1726560564,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":132}],"num_pgs":132,"num_pools":6,"num_objects":196,"data_bytes":596246,"bytes_used":245563392,"bytes_avail":767714721792,"bytes_total":767960285184,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":68,"recovering_keys_per_sec":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"read_bytes_sec":1151,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.79:8443/","prometheus":"http://172.21.15.79:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-17T08:11:33.223331+0000","services":{"rgw":{"daemons":{"summary":"","24377":{"start_epoch":12,"start_stamp":"2024-09-17T08:10:01.618852+0000","gid":24377,"addr":"172.21.15.79:0/3352227429","metadata":{"arch":"x86_64","ceph_release":"reef","ceph_version":"ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)","ceph_version_short":"18.2.4-855-g19e2a2be","container_hostname":"smithi079","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi079","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"ae2fe8bf-c54d-4d41-8c71-5087859ddaac","zone_name":"default","zonegroup_id":"c396a67f-f134-4ebc-b8a4-5fdb3ed807ad","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{"a5b8f011-0737-4023-97d6-e3a4b933f5af":{"message":"Global Recovery Event (0s)\n [............................] ","progress":0,"add_to_ceph_s":true}}} 2024-09-17T08:11:38.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[40649]: 5.4 deep-scrub starts 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[40649]: 5.4 deep-scrub ok 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[29565]: 5.4 deep-scrub starts 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[29565]: 5.4 deep-scrub ok 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:11:38.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:38 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:11:39.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:11:39.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:11:39.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/554048618' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-17T08:11:39.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[40649]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:11:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:11:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/554048618' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-17T08:11:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:39 smithi079 ceph-mon[29565]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:39 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:11:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:39 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:11:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/554048618' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-17T08:11:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:39 smithi173 ceph-mon[33187]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T08:11:40.191 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-17T08:11:40.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:40 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:11:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:40 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:11:40.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:40 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:11:40.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:40 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:11:40.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:40 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:11:40.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:40 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s; 56 B/s, 1 objects/s recovering 2024-09-17T08:11:41.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s; 56 B/s, 1 objects/s recovering 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:41 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s; 56 B/s, 1 objects/s recovering 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:41 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:42.535 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:42.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:11:42.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[40649]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[29565]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T08:11:42.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:42 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:42.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:42 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:11:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:42 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:11:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:42 smithi173 ceph-mon[33187]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T08:11:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:42 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:43.502 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-09-17T08:11:41.452463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1017,"num_read_kb":863,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624,"ondisk_log_size":624,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239808,"kb_used_data":7284,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749721408,"statfs":{"total":767960285184,"available":767714721792,"internally_reserved":0,"allocated":7458816,"data_stored":4478842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.025931"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.012891+0000","last_change":"2024-09-17T08:11:34.012978+0000","last_active":"2024-09-17T08:11:34.012891+0000","last_peered":"2024-09-17T08:11:34.012891+0000","last_clean":"2024-09-17T08:11:34.012891+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:34.012891+0000","last_undegraded":"2024-09-17T08:11:34.012891+0000","last_fullsized":"2024-09-17T08:11:34.012891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.012877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.012877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.356338+0000","scrub_duration":0.001297653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431390+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:31.431390+0000","last_peered":"2024-09-17T08:11:31.431390+0000","last_clean":"2024-09-17T08:11:31.431390+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:31.431390+0000","last_undegraded":"2024-09-17T08:11:31.431390+0000","last_fullsized":"2024-09-17T08:11:31.431390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430247+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:31.430247+0000","last_peered":"2024-09-17T08:11:31.430247+0000","last_clean":"2024-09-17T08:11:31.430247+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:31.430247+0000","last_undegraded":"2024-09-17T08:11:31.430247+0000","last_fullsized":"2024-09-17T08:11:31.430247+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428597+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:31.428597+0000","last_peered":"2024-09-17T08:11:31.428597+0000","last_clean":"2024-09-17T08:11:31.428597+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:31.428597+0000","last_undegraded":"2024-09-17T08:11:31.428597+0000","last_fullsized":"2024-09-17T08:11:31.428597+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431137+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:31.431137+0000","last_peered":"2024-09-17T08:11:31.431137+0000","last_clean":"2024-09-17T08:11:31.431137+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:31.431137+0000","last_undegraded":"2024-09-17T08:11:31.431137+0000","last_fullsized":"2024-09-17T08:11:31.431137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:33.629214+0000","last_change":"2024-09-17T08:11:33.629288+0000","last_active":"2024-09-17T08:11:33.629214+0000","last_peered":"2024-09-17T08:11:33.629214+0000","last_clean":"2024-09-17T08:11:33.629214+0000","last_became_active":"2024-09-17T08:11:32.433797+0000","last_became_peered":"2024-09-17T08:11:32.433797+0000","last_unstale":"2024-09-17T08:11:33.629214+0000","last_undegraded":"2024-09-17T08:11:33.629214+0000","last_fullsized":"2024-09-17T08:11:33.629214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.629197+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.320003+0000","scrub_duration":0.003700662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.913295+0000","last_change":"2024-09-17T08:11:34.913426+0000","last_active":"2024-09-17T08:11:34.913295+0000","last_peered":"2024-09-17T08:11:34.913295+0000","last_clean":"2024-09-17T08:11:34.913295+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:34.913295+0000","last_undegraded":"2024-09-17T08:11:34.913295+0000","last_fullsized":"2024-09-17T08:11:34.913295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.913264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.913264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.562937+0000","scrub_duration":0.0017904069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428860+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:31.428860+0000","last_peered":"2024-09-17T08:11:31.428860+0000","last_clean":"2024-09-17T08:11:31.428860+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:31.428860+0000","last_undegraded":"2024-09-17T08:11:31.428860+0000","last_fullsized":"2024-09-17T08:11:31.428860+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.434234+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:11:31.434234+0000","last_peered":"2024-09-17T08:11:31.434234+0000","last_clean":"2024-09-17T08:11:31.434234+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:31.434234+0000","last_undegraded":"2024-09-17T08:11:31.434234+0000","last_fullsized":"2024-09-17T08:11:31.434234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428946+0000","last_change":"2024-09-17T08:11:29.578149+0000","last_active":"2024-09-17T08:11:31.428946+0000","last_peered":"2024-09-17T08:11:31.428946+0000","last_clean":"2024-09-17T08:11:31.428946+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:31.428946+0000","last_undegraded":"2024-09-17T08:11:31.428946+0000","last_fullsized":"2024-09-17T08:11:31.428946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.578069+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.180514+0000","scrub_duration":0.0025270700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433715+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:31.433715+0000","last_peered":"2024-09-17T08:11:31.433715+0000","last_clean":"2024-09-17T08:11:31.433715+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:31.433715+0000","last_undegraded":"2024-09-17T08:11:31.433715+0000","last_fullsized":"2024-09-17T08:11:31.433715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430036+0000","last_change":"2024-09-17T08:11:31.086448+0000","last_active":"2024-09-17T08:11:31.430036+0000","last_peered":"2024-09-17T08:11:31.430036+0000","last_clean":"2024-09-17T08:11:31.430036+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:31.430036+0000","last_undegraded":"2024-09-17T08:11:31.430036+0000","last_fullsized":"2024-09-17T08:11:31.430036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.086346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.086346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.290177+0000","scrub_duration":0.0013806619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.434032+0000","last_change":"2024-09-17T08:11:28.306565+0000","last_active":"2024-09-17T08:11:31.434032+0000","last_peered":"2024-09-17T08:11:31.434032+0000","last_clean":"2024-09-17T08:11:31.434032+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:31.434032+0000","last_undegraded":"2024-09-17T08:11:31.434032+0000","last_fullsized":"2024-09-17T08:11:31.434032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.306480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.306480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:53.967249+0000","scrub_duration":0.001181517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429245+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:31.429245+0000","last_peered":"2024-09-17T08:11:31.429245+0000","last_clean":"2024-09-17T08:11:31.429245+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:31.429245+0000","last_undegraded":"2024-09-17T08:11:31.429245+0000","last_fullsized":"2024-09-17T08:11:31.429245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430174+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:11:31.430174+0000","last_peered":"2024-09-17T08:11:31.430174+0000","last_clean":"2024-09-17T08:11:31.430174+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:31.430174+0000","last_undegraded":"2024-09-17T08:11:31.430174+0000","last_fullsized":"2024-09-17T08:11:31.430174+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430494+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:11:31.430494+0000","last_peered":"2024-09-17T08:11:31.430494+0000","last_clean":"2024-09-17T08:11:31.430494+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:31.430494+0000","last_undegraded":"2024-09-17T08:11:31.430494+0000","last_fullsized":"2024-09-17T08:11:31.430494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429798+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:31.429798+0000","last_peered":"2024-09-17T08:11:31.429798+0000","last_clean":"2024-09-17T08:11:31.429798+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:31.429798+0000","last_undegraded":"2024-09-17T08:11:31.429798+0000","last_fullsized":"2024-09-17T08:11:31.429798+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.434097+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:11:31.434097+0000","last_peered":"2024-09-17T08:11:31.434097+0000","last_clean":"2024-09-17T08:11:31.434097+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:31.434097+0000","last_undegraded":"2024-09-17T08:11:31.434097+0000","last_fullsized":"2024-09-17T08:11:31.434097+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426967+0000","last_change":"2024-09-17T08:11:31.017724+0000","last_active":"2024-09-17T08:11:31.426967+0000","last_peered":"2024-09-17T08:11:31.426967+0000","last_clean":"2024-09-17T08:11:31.426967+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:31.426967+0000","last_undegraded":"2024-09-17T08:11:31.426967+0000","last_fullsized":"2024-09-17T08:11:31.426967+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.017640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.017640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.994183+0000","scrub_duration":0.0020059460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427909+0000","last_change":"2024-09-17T08:11:28.097377+0000","last_active":"2024-09-17T08:11:31.427909+0000","last_peered":"2024-09-17T08:11:31.427909+0000","last_clean":"2024-09-17T08:11:31.427909+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:31.427909+0000","last_undegraded":"2024-09-17T08:11:31.427909+0000","last_fullsized":"2024-09-17T08:11:31.427909+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.097289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.097289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.706372+0000","scrub_duration":0.001520696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429227+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:31.429227+0000","last_peered":"2024-09-17T08:11:31.429227+0000","last_clean":"2024-09-17T08:11:31.429227+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:31.429227+0000","last_undegraded":"2024-09-17T08:11:31.429227+0000","last_fullsized":"2024-09-17T08:11:31.429227+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426590+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:31.426590+0000","last_peered":"2024-09-17T08:11:31.426590+0000","last_clean":"2024-09-17T08:11:31.426590+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:31.426590+0000","last_undegraded":"2024-09-17T08:11:31.426590+0000","last_fullsized":"2024-09-17T08:11:31.426590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434385+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:11:32.434385+0000","last_peered":"2024-09-17T08:11:32.434385+0000","last_clean":"2024-09-17T08:11:32.434385+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:32.434385+0000","last_undegraded":"2024-09-17T08:11:32.434385+0000","last_fullsized":"2024-09-17T08:11:32.434385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:20.492012+0000","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429758+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:31.429758+0000","last_peered":"2024-09-17T08:11:31.429758+0000","last_clean":"2024-09-17T08:11:31.429758+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:31.429758+0000","last_undegraded":"2024-09-17T08:11:31.429758+0000","last_fullsized":"2024-09-17T08:11:31.429758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427651+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:31.427651+0000","last_peered":"2024-09-17T08:11:31.427651+0000","last_clean":"2024-09-17T08:11:31.427651+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:31.427651+0000","last_undegraded":"2024-09-17T08:11:31.427651+0000","last_fullsized":"2024-09-17T08:11:31.427651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426739+0000","last_change":"2024-09-17T08:09:54.498740+0000","last_active":"2024-09-17T08:11:31.426739+0000","last_peered":"2024-09-17T08:11:31.426739+0000","last_clean":"2024-09-17T08:11:31.426739+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:31.426739+0000","last_undegraded":"2024-09-17T08:11:31.426739+0000","last_fullsized":"2024-09-17T08:11:31.426739+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430064+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:11:32.430064+0000","last_peered":"2024-09-17T08:11:32.430064+0000","last_clean":"2024-09-17T08:11:32.430064+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:32.430064+0000","last_undegraded":"2024-09-17T08:11:32.430064+0000","last_fullsized":"2024-09-17T08:11:32.430064+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430352+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:11:31.430352+0000","last_peered":"2024-09-17T08:11:31.430352+0000","last_clean":"2024-09-17T08:11:31.430352+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:31.430352+0000","last_undegraded":"2024-09-17T08:11:31.430352+0000","last_fullsized":"2024-09-17T08:11:31.430352+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429615+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:11:31.429615+0000","last_peered":"2024-09-17T08:11:31.429615+0000","last_clean":"2024-09-17T08:11:31.429615+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:31.429615+0000","last_undegraded":"2024-09-17T08:11:31.429615+0000","last_fullsized":"2024-09-17T08:11:31.429615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433580+0000","last_change":"2024-09-17T08:11:31.305742+0000","last_active":"2024-09-17T08:11:31.433580+0000","last_peered":"2024-09-17T08:11:31.433580+0000","last_clean":"2024-09-17T08:11:31.433580+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:31.433580+0000","last_undegraded":"2024-09-17T08:11:31.433580+0000","last_fullsized":"2024-09-17T08:11:31.433580+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.305634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.305634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.476701+0000","scrub_duration":0.001519353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431093+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:31.431093+0000","last_peered":"2024-09-17T08:11:31.431093+0000","last_clean":"2024-09-17T08:11:31.431093+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:31.431093+0000","last_undegraded":"2024-09-17T08:11:31.431093+0000","last_fullsized":"2024-09-17T08:11:31.431093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430835+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:31.430835+0000","last_peered":"2024-09-17T08:11:31.430835+0000","last_clean":"2024-09-17T08:11:31.430835+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:31.430835+0000","last_undegraded":"2024-09-17T08:11:31.430835+0000","last_fullsized":"2024-09-17T08:11:31.430835+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428733+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:31.428733+0000","last_peered":"2024-09-17T08:11:31.428733+0000","last_clean":"2024-09-17T08:11:31.428733+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:31.428733+0000","last_undegraded":"2024-09-17T08:11:31.428733+0000","last_fullsized":"2024-09-17T08:11:31.428733+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433100+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:31.433100+0000","last_peered":"2024-09-17T08:11:31.433100+0000","last_clean":"2024-09-17T08:11:31.433100+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:31.433100+0000","last_undegraded":"2024-09-17T08:11:31.433100+0000","last_fullsized":"2024-09-17T08:11:31.433100+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430420+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:32.430420+0000","last_peered":"2024-09-17T08:11:32.430420+0000","last_clean":"2024-09-17T08:11:32.430420+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:32.430420+0000","last_undegraded":"2024-09-17T08:11:32.430420+0000","last_fullsized":"2024-09-17T08:11:32.430420+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433705+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:31.433705+0000","last_peered":"2024-09-17T08:11:31.433705+0000","last_clean":"2024-09-17T08:11:31.433705+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:31.433705+0000","last_undegraded":"2024-09-17T08:11:31.433705+0000","last_fullsized":"2024-09-17T08:11:31.433705+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430071+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:31.430071+0000","last_peered":"2024-09-17T08:11:31.430071+0000","last_clean":"2024-09-17T08:11:31.430071+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:31.430071+0000","last_undegraded":"2024-09-17T08:11:31.430071+0000","last_fullsized":"2024-09-17T08:11:31.430071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:33.944540+0000","last_change":"2024-09-17T08:11:33.944624+0000","last_active":"2024-09-17T08:11:33.944540+0000","last_peered":"2024-09-17T08:11:33.944540+0000","last_clean":"2024-09-17T08:11:33.944540+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:33.944540+0000","last_undegraded":"2024-09-17T08:11:33.944540+0000","last_fullsized":"2024-09-17T08:11:33.944540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:33.944526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.944526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:03.130795+0000","scrub_duration":0.00148507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428578+0000","last_change":"2024-09-17T08:09:58.504741+0000","last_active":"2024-09-17T08:11:31.428578+0000","last_peered":"2024-09-17T08:11:31.428578+0000","last_clean":"2024-09-17T08:11:31.428578+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:31.428578+0000","last_undegraded":"2024-09-17T08:11:31.428578+0000","last_fullsized":"2024-09-17T08:11:31.428578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429143+0000","last_change":"2024-09-17T08:10:00.524679+0000","last_active":"2024-09-17T08:11:31.429143+0000","last_peered":"2024-09-17T08:11:31.429143+0000","last_clean":"2024-09-17T08:11:31.429143+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:31.429143+0000","last_undegraded":"2024-09-17T08:11:31.429143+0000","last_fullsized":"2024-09-17T08:11:31.429143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431955+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:31.431955+0000","last_peered":"2024-09-17T08:11:31.431955+0000","last_clean":"2024-09-17T08:11:31.431955+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:31.431955+0000","last_undegraded":"2024-09-17T08:11:31.431955+0000","last_fullsized":"2024-09-17T08:11:31.431955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.434334+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:31.434334+0000","last_peered":"2024-09-17T08:11:31.434334+0000","last_clean":"2024-09-17T08:11:31.434334+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:31.434334+0000","last_undegraded":"2024-09-17T08:11:31.434334+0000","last_fullsized":"2024-09-17T08:11:31.434334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432293+0000","last_change":"2024-09-17T08:11:28.433770+0000","last_active":"2024-09-17T08:11:31.432293+0000","last_peered":"2024-09-17T08:11:31.432293+0000","last_clean":"2024-09-17T08:11:31.432293+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:31.432293+0000","last_undegraded":"2024-09-17T08:11:31.432293+0000","last_fullsized":"2024-09-17T08:11:31.432293+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.433666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.433666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.684654+0000","scrub_duration":0.001195822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426815+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:31.426815+0000","last_peered":"2024-09-17T08:11:31.426815+0000","last_clean":"2024-09-17T08:11:31.426815+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:31.426815+0000","last_undegraded":"2024-09-17T08:11:31.426815+0000","last_fullsized":"2024-09-17T08:11:31.426815+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426472+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:11:31.426472+0000","last_peered":"2024-09-17T08:11:31.426472+0000","last_clean":"2024-09-17T08:11:31.426472+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:31.426472+0000","last_undegraded":"2024-09-17T08:11:31.426472+0000","last_fullsized":"2024-09-17T08:11:31.426472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427244+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:11:31.427244+0000","last_peered":"2024-09-17T08:11:31.427244+0000","last_clean":"2024-09-17T08:11:31.427244+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:31.427244+0000","last_undegraded":"2024-09-17T08:11:31.427244+0000","last_fullsized":"2024-09-17T08:11:31.427244+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434456+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:32.434456+0000","last_peered":"2024-09-17T08:11:32.434456+0000","last_clean":"2024-09-17T08:11:32.434456+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:32.434456+0000","last_undegraded":"2024-09-17T08:11:32.434456+0000","last_fullsized":"2024-09-17T08:11:32.434456+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.080416+0000","last_change":"2024-09-17T08:11:32.080416+0000","last_active":"2024-09-17T08:11:32.080416+0000","last_peered":"2024-09-17T08:11:32.080416+0000","last_clean":"2024-09-17T08:11:32.080416+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:32.080416+0000","last_undegraded":"2024-09-17T08:11:32.080416+0000","last_fullsized":"2024-09-17T08:11:32.080416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.080269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.080269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.073092+0000","scrub_duration":0.001335506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429748+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:32.429748+0000","last_peered":"2024-09-17T08:11:32.429748+0000","last_clean":"2024-09-17T08:11:32.429748+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:32.429748+0000","last_undegraded":"2024-09-17T08:11:32.429748+0000","last_fullsized":"2024-09-17T08:11:32.429748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430803+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:31.430803+0000","last_peered":"2024-09-17T08:11:31.430803+0000","last_clean":"2024-09-17T08:11:31.430803+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:31.430803+0000","last_undegraded":"2024-09-17T08:11:31.430803+0000","last_fullsized":"2024-09-17T08:11:31.430803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431345+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:31.431345+0000","last_peered":"2024-09-17T08:11:31.431345+0000","last_clean":"2024-09-17T08:11:31.431345+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:31.431345+0000","last_undegraded":"2024-09-17T08:11:31.431345+0000","last_fullsized":"2024-09-17T08:11:31.431345+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430987+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:31.430987+0000","last_peered":"2024-09-17T08:11:31.430987+0000","last_clean":"2024-09-17T08:11:31.430987+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:31.430987+0000","last_undegraded":"2024-09-17T08:11:31.430987+0000","last_fullsized":"2024-09-17T08:11:31.430987+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.025661+0000","last_change":"2024-09-17T08:11:34.025771+0000","last_active":"2024-09-17T08:11:34.025661+0000","last_peered":"2024-09-17T08:11:34.025661+0000","last_clean":"2024-09-17T08:11:34.025661+0000","last_became_active":"2024-09-17T08:11:32.435377+0000","last_became_peered":"2024-09-17T08:11:32.435377+0000","last_unstale":"2024-09-17T08:11:34.025661+0000","last_undegraded":"2024-09-17T08:11:34.025661+0000","last_fullsized":"2024-09-17T08:11:34.025661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:11:34.025633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.025633+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.265538+0000","scrub_duration":0.0023641140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":147,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:36.185377+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:36.185377+0000","last_peered":"2024-09-17T08:11:36.185377+0000","last_clean":"2024-09-17T08:11:36.185377+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:36.185377+0000","last_undegraded":"2024-09-17T08:11:36.185377+0000","last_fullsized":"2024-09-17T08:11:36.185377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":96,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427040+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:11:31.427040+0000","last_peered":"2024-09-17T08:11:31.427040+0000","last_clean":"2024-09-17T08:11:31.427040+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:31.427040+0000","last_undegraded":"2024-09-17T08:11:31.427040+0000","last_fullsized":"2024-09-17T08:11:31.427040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427595+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:11:31.427595+0000","last_peered":"2024-09-17T08:11:31.427595+0000","last_clean":"2024-09-17T08:11:31.427595+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:31.427595+0000","last_undegraded":"2024-09-17T08:11:31.427595+0000","last_fullsized":"2024-09-17T08:11:31.427595+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433906+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:11:31.433906+0000","last_peered":"2024-09-17T08:11:31.433906+0000","last_clean":"2024-09-17T08:11:31.433906+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:31.433906+0000","last_undegraded":"2024-09-17T08:11:31.433906+0000","last_fullsized":"2024-09-17T08:11:31.433906+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":65,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432054+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:31.432054+0000","last_peered":"2024-09-17T08:11:31.432054+0000","last_clean":"2024-09-17T08:11:31.432054+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:31.432054+0000","last_undegraded":"2024-09-17T08:11:31.432054+0000","last_fullsized":"2024-09-17T08:11:31.432054+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431518+0000","last_change":"2024-09-17T08:11:28.111674+0000","last_active":"2024-09-17T08:11:31.431518+0000","last_peered":"2024-09-17T08:11:31.431518+0000","last_clean":"2024-09-17T08:11:31.431518+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:31.431518+0000","last_undegraded":"2024-09-17T08:11:31.431518+0000","last_fullsized":"2024-09-17T08:11:31.431518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:11:28.111569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.111569+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.306471+0000","scrub_duration":0.0016978239999999999,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.356142+0000","last_change":"2024-09-17T08:11:34.356211+0000","last_active":"2024-09-17T08:11:34.356142+0000","last_peered":"2024-09-17T08:11:34.356142+0000","last_clean":"2024-09-17T08:11:34.356142+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:34.356142+0000","last_undegraded":"2024-09-17T08:11:34.356142+0000","last_fullsized":"2024-09-17T08:11:34.356142+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.356129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.356129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.828603+0000","scrub_duration":0.001774224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431178+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:11:31.431178+0000","last_peered":"2024-09-17T08:11:31.431178+0000","last_clean":"2024-09-17T08:11:31.431178+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:31.431178+0000","last_undegraded":"2024-09-17T08:11:31.431178+0000","last_fullsized":"2024-09-17T08:11:31.431178+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426900+0000","last_change":"2024-09-17T08:10:00.519217+0000","last_active":"2024-09-17T08:11:31.426900+0000","last_peered":"2024-09-17T08:11:31.426900+0000","last_clean":"2024-09-17T08:11:31.426900+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:31.426900+0000","last_undegraded":"2024-09-17T08:11:31.426900+0000","last_fullsized":"2024-09-17T08:11:31.426900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432840+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:31.432840+0000","last_peered":"2024-09-17T08:11:31.432840+0000","last_clean":"2024-09-17T08:11:31.432840+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:31.432840+0000","last_undegraded":"2024-09-17T08:11:31.432840+0000","last_fullsized":"2024-09-17T08:11:31.432840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430679+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:31.430679+0000","last_peered":"2024-09-17T08:11:31.430679+0000","last_clean":"2024-09-17T08:11:31.430679+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:31.430679+0000","last_undegraded":"2024-09-17T08:11:31.430679+0000","last_fullsized":"2024-09-17T08:11:31.430679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430681+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:11:31.430681+0000","last_peered":"2024-09-17T08:11:31.430681+0000","last_clean":"2024-09-17T08:11:31.430681+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:31.430681+0000","last_undegraded":"2024-09-17T08:11:31.430681+0000","last_fullsized":"2024-09-17T08:11:31.430681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":293,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.434008+0000","last_change":"2024-09-17T08:10:08.344236+0000","last_active":"2024-09-17T08:11:31.434008+0000","last_peered":"2024-09-17T08:11:31.434008+0000","last_clean":"2024-09-17T08:11:31.434008+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:31.434008+0000","last_undegraded":"2024-09-17T08:11:31.434008+0000","last_fullsized":"2024-09-17T08:11:31.434008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:10:08.344133+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433762+0000","last_change":"2024-09-17T08:09:58.517794+0000","last_active":"2024-09-17T08:11:31.433762+0000","last_peered":"2024-09-17T08:11:31.433762+0000","last_clean":"2024-09-17T08:11:31.433762+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:31.433762+0000","last_undegraded":"2024-09-17T08:11:31.433762+0000","last_fullsized":"2024-09-17T08:11:31.433762+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431268+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:31.431268+0000","last_peered":"2024-09-17T08:11:31.431268+0000","last_clean":"2024-09-17T08:11:31.431268+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:31.431268+0000","last_undegraded":"2024-09-17T08:11:31.431268+0000","last_fullsized":"2024-09-17T08:11:31.431268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432407+0000","last_change":"2024-09-17T08:11:27.389593+0000","last_active":"2024-09-17T08:11:31.432407+0000","last_peered":"2024-09-17T08:11:31.432407+0000","last_clean":"2024-09-17T08:11:31.432407+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:31.432407+0000","last_undegraded":"2024-09-17T08:11:31.432407+0000","last_fullsized":"2024-09-17T08:11:31.432407+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:27.389490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.389490+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.830431+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428291+0000","last_change":"2024-09-17T08:11:30.055058+0000","last_active":"2024-09-17T08:11:31.428291+0000","last_peered":"2024-09-17T08:11:31.428291+0000","last_clean":"2024-09-17T08:11:31.428291+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:31.428291+0000","last_undegraded":"2024-09-17T08:11:31.428291+0000","last_fullsized":"2024-09-17T08:11:31.428291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:30.054968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.054968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.938144+0000","scrub_duration":0.0018411460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427304+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:31.427304+0000","last_peered":"2024-09-17T08:11:31.427304+0000","last_clean":"2024-09-17T08:11:31.427304+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:31.427304+0000","last_undegraded":"2024-09-17T08:11:31.427304+0000","last_fullsized":"2024-09-17T08:11:31.427304+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433911+0000","last_change":"2024-09-17T08:11:27.312687+0000","last_active":"2024-09-17T08:11:31.433911+0000","last_peered":"2024-09-17T08:11:31.433911+0000","last_clean":"2024-09-17T08:11:31.433911+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:31.433911+0000","last_undegraded":"2024-09-17T08:11:31.433911+0000","last_fullsized":"2024-09-17T08:11:31.433911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.312595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.312595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.098135+0000","scrub_duration":0.001815178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431753+0000","last_change":"2024-09-17T08:11:30.115032+0000","last_active":"2024-09-17T08:11:31.431753+0000","last_peered":"2024-09-17T08:11:31.431753+0000","last_clean":"2024-09-17T08:11:31.431753+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:31.431753+0000","last_undegraded":"2024-09-17T08:11:31.431753+0000","last_fullsized":"2024-09-17T08:11:31.431753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:30.114921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.114921+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.519251+0000","scrub_duration":0.0019919109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427334+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:11:31.427334+0000","last_peered":"2024-09-17T08:11:31.427334+0000","last_clean":"2024-09-17T08:11:31.427334+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:31.427334+0000","last_undegraded":"2024-09-17T08:11:31.427334+0000","last_fullsized":"2024-09-17T08:11:31.427334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427173+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:11:31.427173+0000","last_peered":"2024-09-17T08:11:31.427173+0000","last_clean":"2024-09-17T08:11:31.427173+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:31.427173+0000","last_undegraded":"2024-09-17T08:11:31.427173+0000","last_fullsized":"2024-09-17T08:11:31.427173+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430724+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:11:31.430724+0000","last_peered":"2024-09-17T08:11:31.430724+0000","last_clean":"2024-09-17T08:11:31.430724+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:31.430724+0000","last_undegraded":"2024-09-17T08:11:31.430724+0000","last_fullsized":"2024-09-17T08:11:31.430724+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430056+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:11:31.430056+0000","last_peered":"2024-09-17T08:11:31.430056+0000","last_clean":"2024-09-17T08:11:31.430056+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:31.430056+0000","last_undegraded":"2024-09-17T08:11:31.430056+0000","last_fullsized":"2024-09-17T08:11:31.430056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427761+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:31.427761+0000","last_peered":"2024-09-17T08:11:31.427761+0000","last_clean":"2024-09-17T08:11:31.427761+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:31.427761+0000","last_undegraded":"2024-09-17T08:11:31.427761+0000","last_fullsized":"2024-09-17T08:11:31.427761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:36.379004+0000","last_change":"2024-09-17T08:11:36.379074+0000","last_active":"2024-09-17T08:11:36.379004+0000","last_peered":"2024-09-17T08:11:36.379004+0000","last_clean":"2024-09-17T08:11:36.379004+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:36.379004+0000","last_undegraded":"2024-09-17T08:11:36.379004+0000","last_fullsized":"2024-09-17T08:11:36.379004+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:36.378990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.378990+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.391768+0000","scrub_duration":0.001580082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.089192+0000","last_change":"2024-09-17T08:11:32.089192+0000","last_active":"2024-09-17T08:11:32.089192+0000","last_peered":"2024-09-17T08:11:32.089192+0000","last_clean":"2024-09-17T08:11:32.089192+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:32.089192+0000","last_undegraded":"2024-09-17T08:11:32.089192+0000","last_fullsized":"2024-09-17T08:11:32.089192+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.089096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.089096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.048086+0000","scrub_duration":0.0013942010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428904+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:31.428904+0000","last_peered":"2024-09-17T08:11:31.428904+0000","last_clean":"2024-09-17T08:11:31.428904+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:31.428904+0000","last_undegraded":"2024-09-17T08:11:31.428904+0000","last_fullsized":"2024-09-17T08:11:31.428904+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429545+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:31.429545+0000","last_peered":"2024-09-17T08:11:31.429545+0000","last_clean":"2024-09-17T08:11:31.429545+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:31.429545+0000","last_undegraded":"2024-09-17T08:11:31.429545+0000","last_fullsized":"2024-09-17T08:11:31.429545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426788+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:31.426788+0000","last_peered":"2024-09-17T08:11:31.426788+0000","last_clean":"2024-09-17T08:11:31.426788+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:31.426788+0000","last_undegraded":"2024-09-17T08:11:31.426788+0000","last_fullsized":"2024-09-17T08:11:31.426788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431175+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:31.431175+0000","last_peered":"2024-09-17T08:11:31.431175+0000","last_clean":"2024-09-17T08:11:31.431175+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:31.431175+0000","last_undegraded":"2024-09-17T08:11:31.431175+0000","last_fullsized":"2024-09-17T08:11:31.431175+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432050+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:11:31.432050+0000","last_peered":"2024-09-17T08:11:31.432050+0000","last_clean":"2024-09-17T08:11:31.432050+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:31.432050+0000","last_undegraded":"2024-09-17T08:11:31.432050+0000","last_fullsized":"2024-09-17T08:11:31.432050+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426935+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:31.426935+0000","last_peered":"2024-09-17T08:11:31.426935+0000","last_clean":"2024-09-17T08:11:31.426935+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:31.426935+0000","last_undegraded":"2024-09-17T08:11:31.426935+0000","last_fullsized":"2024-09-17T08:11:31.426935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431018+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:31.431018+0000","last_peered":"2024-09-17T08:11:31.431018+0000","last_clean":"2024-09-17T08:11:31.431018+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:31.431018+0000","last_undegraded":"2024-09-17T08:11:31.431018+0000","last_fullsized":"2024-09-17T08:11:31.431018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427408+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:31.427408+0000","last_peered":"2024-09-17T08:11:31.427408+0000","last_clean":"2024-09-17T08:11:31.427408+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:31.427408+0000","last_undegraded":"2024-09-17T08:11:31.427408+0000","last_fullsized":"2024-09-17T08:11:31.427408+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432118+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:11:31.432118+0000","last_peered":"2024-09-17T08:11:31.432118+0000","last_clean":"2024-09-17T08:11:31.432118+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:31.432118+0000","last_undegraded":"2024-09-17T08:11:31.432118+0000","last_fullsized":"2024-09-17T08:11:31.432118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431106+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:11:31.431106+0000","last_peered":"2024-09-17T08:11:31.431106+0000","last_clean":"2024-09-17T08:11:31.431106+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:31.431106+0000","last_undegraded":"2024-09-17T08:11:31.431106+0000","last_fullsized":"2024-09-17T08:11:31.431106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430737+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:31.430737+0000","last_peered":"2024-09-17T08:11:31.430737+0000","last_clean":"2024-09-17T08:11:31.430737+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:31.430737+0000","last_undegraded":"2024-09-17T08:11:31.430737+0000","last_fullsized":"2024-09-17T08:11:31.430737+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433885+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:31.433885+0000","last_peered":"2024-09-17T08:11:31.433885+0000","last_clean":"2024-09-17T08:11:31.433885+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:31.433885+0000","last_undegraded":"2024-09-17T08:11:31.433885+0000","last_fullsized":"2024-09-17T08:11:31.433885+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431459+0000","last_change":"2024-09-17T08:11:29.979585+0000","last_active":"2024-09-17T08:11:31.431459+0000","last_peered":"2024-09-17T08:11:31.431459+0000","last_clean":"2024-09-17T08:11:31.431459+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:31.431459+0000","last_undegraded":"2024-09-17T08:11:31.431459+0000","last_fullsized":"2024-09-17T08:11:31.431459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:29.979461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.979461+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.112016+0000","scrub_duration":0.0029734470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.437732+0000","last_change":"2024-09-17T08:11:32.437732+0000","last_active":"2024-09-17T08:11:32.437732+0000","last_peered":"2024-09-17T08:11:32.437732+0000","last_clean":"2024-09-17T08:11:32.437732+0000","last_became_active":"2024-09-17T08:11:32.437420+0000","last_became_peered":"2024-09-17T08:11:32.437420+0000","last_unstale":"2024-09-17T08:11:32.437732+0000","last_undegraded":"2024-09-17T08:11:32.437732+0000","last_fullsized":"2024-09-17T08:11:32.437732+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.232110+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433882+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:32.433882+0000","last_peered":"2024-09-17T08:11:32.433882+0000","last_clean":"2024-09-17T08:11:32.433882+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:32.433882+0000","last_undegraded":"2024-09-17T08:11:32.433882+0000","last_fullsized":"2024-09-17T08:11:32.433882+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431749+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:11:31.431749+0000","last_peered":"2024-09-17T08:11:31.431749+0000","last_clean":"2024-09-17T08:11:31.431749+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:31.431749+0000","last_undegraded":"2024-09-17T08:11:31.431749+0000","last_fullsized":"2024-09-17T08:11:31.431749+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430491+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:31.430491+0000","last_peered":"2024-09-17T08:11:31.430491+0000","last_clean":"2024-09-17T08:11:31.430491+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:31.430491+0000","last_undegraded":"2024-09-17T08:11:31.430491+0000","last_fullsized":"2024-09-17T08:11:31.430491+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431925+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:31.431925+0000","last_peered":"2024-09-17T08:11:31.431925+0000","last_clean":"2024-09-17T08:11:31.431925+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:31.431925+0000","last_undegraded":"2024-09-17T08:11:31.431925+0000","last_fullsized":"2024-09-17T08:11:31.431925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430851+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:32.430851+0000","last_peered":"2024-09-17T08:11:32.430851+0000","last_clean":"2024-09-17T08:11:32.430851+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:32.430851+0000","last_undegraded":"2024-09-17T08:11:32.430851+0000","last_fullsized":"2024-09-17T08:11:32.430851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429311+0000","last_change":"2024-09-17T08:10:00.527342+0000","last_active":"2024-09-17T08:11:31.429311+0000","last_peered":"2024-09-17T08:11:31.429311+0000","last_clean":"2024-09-17T08:11:31.429311+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:31.429311+0000","last_undegraded":"2024-09-17T08:11:31.429311+0000","last_fullsized":"2024-09-17T08:11:31.429311+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426396+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:31.426396+0000","last_peered":"2024-09-17T08:11:31.426396+0000","last_clean":"2024-09-17T08:11:31.426396+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:31.426396+0000","last_undegraded":"2024-09-17T08:11:31.426396+0000","last_fullsized":"2024-09-17T08:11:31.426396+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432767+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:31.432767+0000","last_peered":"2024-09-17T08:11:31.432767+0000","last_clean":"2024-09-17T08:11:31.432767+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:31.432767+0000","last_undegraded":"2024-09-17T08:11:31.432767+0000","last_fullsized":"2024-09-17T08:11:31.432767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431219+0000","last_change":"2024-09-17T08:11:31.019796+0000","last_active":"2024-09-17T08:11:31.431219+0000","last_peered":"2024-09-17T08:11:31.431219+0000","last_clean":"2024-09-17T08:11:31.431219+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:31.431219+0000","last_undegraded":"2024-09-17T08:11:31.431219+0000","last_fullsized":"2024-09-17T08:11:31.431219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.019695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.019695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.782145+0000","scrub_duration":0.001088767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430278+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:11:31.430278+0000","last_peered":"2024-09-17T08:11:31.430278+0000","last_clean":"2024-09-17T08:11:31.430278+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:31.430278+0000","last_undegraded":"2024-09-17T08:11:31.430278+0000","last_fullsized":"2024-09-17T08:11:31.430278+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.428882+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:31.428882+0000","last_peered":"2024-09-17T08:11:31.428882+0000","last_clean":"2024-09-17T08:11:31.428882+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:31.428882+0000","last_undegraded":"2024-09-17T08:11:31.428882+0000","last_fullsized":"2024-09-17T08:11:31.428882+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427289+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:11:31.427289+0000","last_peered":"2024-09-17T08:11:31.427289+0000","last_clean":"2024-09-17T08:11:31.427289+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:31.427289+0000","last_undegraded":"2024-09-17T08:11:31.427289+0000","last_fullsized":"2024-09-17T08:11:31.427289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426260+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:31.426260+0000","last_peered":"2024-09-17T08:11:31.426260+0000","last_clean":"2024-09-17T08:11:31.426260+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:31.426260+0000","last_undegraded":"2024-09-17T08:11:31.426260+0000","last_fullsized":"2024-09-17T08:11:31.426260+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429987+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:31.429987+0000","last_peered":"2024-09-17T08:11:31.429987+0000","last_clean":"2024-09-17T08:11:31.429987+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:31.429987+0000","last_undegraded":"2024-09-17T08:11:31.429987+0000","last_fullsized":"2024-09-17T08:11:31.429987+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429987+0000","last_change":"2024-09-17T08:09:56.506875+0000","last_active":"2024-09-17T08:11:31.429987+0000","last_peered":"2024-09-17T08:11:31.429987+0000","last_clean":"2024-09-17T08:11:31.429987+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:31.429987+0000","last_undegraded":"2024-09-17T08:11:31.429987+0000","last_fullsized":"2024-09-17T08:11:31.429987+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430909+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:31.430909+0000","last_peered":"2024-09-17T08:11:31.430909+0000","last_clean":"2024-09-17T08:11:31.430909+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:31.430909+0000","last_undegraded":"2024-09-17T08:11:31.430909+0000","last_fullsized":"2024-09-17T08:11:31.430909+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431922+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:31.431922+0000","last_peered":"2024-09-17T08:11:31.431922+0000","last_clean":"2024-09-17T08:11:31.431922+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:31.431922+0000","last_undegraded":"2024-09-17T08:11:31.431922+0000","last_fullsized":"2024-09-17T08:11:31.431922+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433540+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:31.433540+0000","last_peered":"2024-09-17T08:11:31.433540+0000","last_clean":"2024-09-17T08:11:31.433540+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:31.433540+0000","last_undegraded":"2024-09-17T08:11:31.433540+0000","last_fullsized":"2024-09-17T08:11:31.433540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430166+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:31.430166+0000","last_peered":"2024-09-17T08:11:31.430166+0000","last_clean":"2024-09-17T08:11:31.430166+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:31.430166+0000","last_undegraded":"2024-09-17T08:11:31.430166+0000","last_fullsized":"2024-09-17T08:11:31.430166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433058+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:11:31.433058+0000","last_peered":"2024-09-17T08:11:31.433058+0000","last_clean":"2024-09-17T08:11:31.433058+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:31.433058+0000","last_undegraded":"2024-09-17T08:11:31.433058+0000","last_fullsized":"2024-09-17T08:11:31.433058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433330+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:31.433330+0000","last_peered":"2024-09-17T08:11:31.433330+0000","last_clean":"2024-09-17T08:11:31.433330+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:31.433330+0000","last_undegraded":"2024-09-17T08:11:31.433330+0000","last_fullsized":"2024-09-17T08:11:31.433330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430760+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:31.430760+0000","last_peered":"2024-09-17T08:11:31.430760+0000","last_clean":"2024-09-17T08:11:31.430760+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:31.430760+0000","last_undegraded":"2024-09-17T08:11:31.430760+0000","last_fullsized":"2024-09-17T08:11:31.430760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433827+0000","last_change":"2024-09-17T08:10:00.527045+0000","last_active":"2024-09-17T08:11:31.433827+0000","last_peered":"2024-09-17T08:11:31.433827+0000","last_clean":"2024-09-17T08:11:31.433827+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:31.433827+0000","last_undegraded":"2024-09-17T08:11:31.433827+0000","last_fullsized":"2024-09-17T08:11:31.433827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430144+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:31.430144+0000","last_peered":"2024-09-17T08:11:31.430144+0000","last_clean":"2024-09-17T08:11:31.430144+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:31.430144+0000","last_undegraded":"2024-09-17T08:11:31.430144+0000","last_fullsized":"2024-09-17T08:11:31.430144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431895+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:11:31.431895+0000","last_peered":"2024-09-17T08:11:31.431895+0000","last_clean":"2024-09-17T08:11:31.431895+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:31.431895+0000","last_undegraded":"2024-09-17T08:11:31.431895+0000","last_fullsized":"2024-09-17T08:11:31.431895+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431914+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:11:32.431914+0000","last_peered":"2024-09-17T08:11:32.431914+0000","last_clean":"2024-09-17T08:11:32.431914+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:32.431914+0000","last_undegraded":"2024-09-17T08:11:32.431914+0000","last_fullsized":"2024-09-17T08:11:32.431914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.426900+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:11:31.426900+0000","last_peered":"2024-09-17T08:11:31.426900+0000","last_clean":"2024-09-17T08:11:31.426900+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:31.426900+0000","last_undegraded":"2024-09-17T08:11:31.426900+0000","last_fullsized":"2024-09-17T08:11:31.426900+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430472+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:11:31.430472+0000","last_peered":"2024-09-17T08:11:31.430472+0000","last_clean":"2024-09-17T08:11:31.430472+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:31.430472+0000","last_undegraded":"2024-09-17T08:11:31.430472+0000","last_fullsized":"2024-09-17T08:11:31.430472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.433057+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:31.433057+0000","last_peered":"2024-09-17T08:11:31.433057+0000","last_clean":"2024-09-17T08:11:31.433057+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:31.433057+0000","last_undegraded":"2024-09-17T08:11:31.433057+0000","last_fullsized":"2024-09-17T08:11:31.433057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431882+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:31.431882+0000","last_peered":"2024-09-17T08:11:31.431882+0000","last_clean":"2024-09-17T08:11:31.431882+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:31.431882+0000","last_undegraded":"2024-09-17T08:11:31.431882+0000","last_fullsized":"2024-09-17T08:11:31.431882+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:36.624779+0000","last_change":"2024-09-17T08:11:36.624844+0000","last_active":"2024-09-17T08:11:36.624779+0000","last_peered":"2024-09-17T08:11:36.624779+0000","last_clean":"2024-09-17T08:11:36.624779+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:36.624779+0000","last_undegraded":"2024-09-17T08:11:36.624779+0000","last_fullsized":"2024-09-17T08:11:36.624779+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.624764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:59.550403+0000","scrub_duration":0.00124591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.432485+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:31.432485+0000","last_peered":"2024-09-17T08:11:31.432485+0000","last_clean":"2024-09-17T08:11:31.432485+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:31.432485+0000","last_undegraded":"2024-09-17T08:11:31.432485+0000","last_fullsized":"2024-09-17T08:11:31.432485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.992139+0000","last_change":"2024-09-17T08:11:34.992216+0000","last_active":"2024-09-17T08:11:34.992139+0000","last_peered":"2024-09-17T08:11:34.992139+0000","last_clean":"2024-09-17T08:11:34.992139+0000","last_became_active":"2024-09-17T08:11:32.436438+0000","last_became_peered":"2024-09-17T08:11:32.436438+0000","last_unstale":"2024-09-17T08:11:34.992139+0000","last_undegraded":"2024-09-17T08:11:34.992139+0000","last_fullsized":"2024-09-17T08:11:34.992139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.992118+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.249927+0000","scrub_duration":0.0049126200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.427111+0000","last_change":"2024-09-17T08:09:58.507385+0000","last_active":"2024-09-17T08:11:31.427111+0000","last_peered":"2024-09-17T08:11:31.427111+0000","last_clean":"2024-09-17T08:11:31.427111+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:31.427111+0000","last_undegraded":"2024-09-17T08:11:31.427111+0000","last_fullsized":"2024-09-17T08:11:31.427111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430939+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:31.430939+0000","last_peered":"2024-09-17T08:11:31.430939+0000","last_clean":"2024-09-17T08:11:31.430939+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:31.430939+0000","last_undegraded":"2024-09-17T08:11:31.430939+0000","last_fullsized":"2024-09-17T08:11:31.430939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.431315+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:31.431315+0000","last_peered":"2024-09-17T08:11:31.431315+0000","last_clean":"2024-09-17T08:11:31.431315+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:31.431315+0000","last_undegraded":"2024-09-17T08:11:31.431315+0000","last_fullsized":"2024-09-17T08:11:31.431315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.430865+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:31.430865+0000","last_peered":"2024-09-17T08:11:31.430865+0000","last_clean":"2024-09-17T08:11:31.430865+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:31.430865+0000","last_undegraded":"2024-09-17T08:11:31.430865+0000","last_fullsized":"2024-09-17T08:11:31.430865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T08:11:31.429124+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:31.429124+0000","last_peered":"2024-09-17T08:11:31.429124+0000","last_clean":"2024-09-17T08:11:31.429124+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:31.429124+0000","last_undegraded":"2024-09-17T08:11:31.429124+0000","last_fullsized":"2024-09-17T08:11:31.429124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070490,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1302528,"data_stored":929053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.371},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33100000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.379}]},{"osd":6,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.31}]}]},{"osd":6,"up_from":50,"seq":214748364831,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1290240,"data_stored":928644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.69399999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67600000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.313}]},{"osd":1,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66500000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.438},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.498}]},{"osd":4,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.317,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.374},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34000000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.38700000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.34799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46300000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48699999999999999}]}]},{"osd":5,"up_from":41,"seq":176093659172,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31792,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713360,"statfs":{"total":95995035648,"available":95962480640,"internally_reserved":0,"allocated":696320,"data_stored":337731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.437}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.497}]},{"osd":7,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.50700000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953513,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27712,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":712704,"data_stored":338854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.66700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.746,"15min":0.746},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.628,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.59399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.33400000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72999999999999998,"5min":0.75,"15min":0.75},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.32800000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.316},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.25900000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.432},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35899999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.376}]}]},{"osd":3,"up_from":27,"seq":115964117037,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27724,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":724992,"data_stored":338331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.26400000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.312},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.24399999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.32100000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38300000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.438}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83299999999999996,"5min":0.873,"15min":0.873},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:10:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27712,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":712704,"data_stored":338843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.34300000000000003}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.315},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.749,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56399999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31820,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713332,"statfs":{"total":95995035648,"available":95962451968,"internally_reserved":0,"allocated":720896,"data_stored":338591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.191,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.39400000000000002,"5min":0.625,"15min":0.625},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.312,"5min":0.33300000000000002,"15min":0.30299999999999999},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46100000000000002,"5min":0.625,"15min":0.625},"last":0.255}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32600000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.30099999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.087,"5min":1.329,"15min":1.329},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.48899999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1279999999999999,"5min":1.157,"15min":1.157},"last":0.497}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1298432,"data_stored":928795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.248},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.307},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":0.874},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.443}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.378}]},{"osd":7,"last update":"Tue Sep 17 08:10:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:43 smithi173 ceph-mon[33187]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:43 smithi173 ceph-mon[33187]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:43.808 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:43 smithi079 ceph-mon[40649]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:43.808 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:43 smithi079 ceph-mon[40649]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:43.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:43 smithi079 ceph-mon[29565]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:43.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:43 smithi079 ceph-mon[29565]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:11:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:11:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: 5.1 deep-scrub starts 2024-09-17T08:11:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: 5.1 deep-scrub ok 2024-09-17T08:11:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:44 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: 5.1 deep-scrub starts 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: 5.1 deep-scrub ok 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: 5.1 deep-scrub starts 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: 5.1 deep-scrub ok 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:44 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:45.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:11:45.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:11:45.531 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: Reconfiguring daemon alertmanager.a on smithi079 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[40649]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:11:45.532 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: Reconfiguring daemon alertmanager.a on smithi079 2024-09-17T08:11:45.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:45 smithi079 ceph-mon[29565]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: Reconfiguring daemon alertmanager.a on smithi079 2024-09-17T08:11:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:45 smithi173 ceph-mon[33187]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-17T08:11:46.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:11:47.161 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:47.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:47.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:47.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:47.556 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:47.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:47.556 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:11:48.574 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:48 smithi079 ceph-mon[40649]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:48.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:48 smithi079 ceph-mon[29565]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:48.575 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:48 smithi079 systemd[1]: Stopping Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:48 smithi173 ceph-mon[33187]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:49.463 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:49 smithi079 ceph-mon[29565]: 6.e deep-scrub starts 2024-09-17T08:11:49.463 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:49 smithi079 ceph-mon[29565]: 6.e deep-scrub ok 2024-09-17T08:11:49.463 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:49 smithi079 ceph-mon[40649]: 6.e deep-scrub starts 2024-09-17T08:11:49.463 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:49 smithi079 ceph-mon[40649]: 6.e deep-scrub ok 2024-09-17T08:11:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:11:49.616 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:11:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:49 smithi173 ceph-mon[33187]: 6.e deep-scrub starts 2024-09-17T08:11:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:49 smithi173 ceph-mon[33187]: 6.e deep-scrub ok 2024-09-17T08:11:49.869 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:49 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[76373]: ts=2024-09-17T08:11:49.572Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:11:49.869 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:49 smithi079 podman[79568]: 2024-09-17 08:11:49.593427779 +0000 UTC m=+0.138618603 container died 8f6d38a3441dc9bddd53235cc498e648c90685b8d7e5581070e7323efa8afe60 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:50.131 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:50 smithi079 podman[79568]: 2024-09-17 08:11:50.113951819 +0000 UTC m=+0.659142636 container cleanup 8f6d38a3441dc9bddd53235cc498e648c90685b8d7e5581070e7323efa8afe60 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:50.131 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:50 smithi079 bash[79568]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T08:11:50.619 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:50 smithi079 podman[79589]: 2024-09-17 08:11:50.323574685 +0000 UTC m=+0.728120384 container remove 8f6d38a3441dc9bddd53235cc498e648c90685b8d7e5581070e7323efa8afe60 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:50.619 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:50 smithi079 podman[79589]: 2024-09-17 08:11:50.399131925 +0000 UTC m=+0.803677624 volume remove c38b119547caf28528fe7389d2f2e5e06c5986badcaf7cbad5895eddad14db03 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[29565]: 5.14 deep-scrub starts 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[29565]: 5.14 deep-scrub ok 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[29565]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[40649]: 5.14 deep-scrub starts 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[40649]: 5.14 deep-scrub ok 2024-09-17T08:11:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:50 smithi079 ceph-mon[40649]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:50 smithi173 ceph-mon[33187]: 5.14 deep-scrub starts 2024-09-17T08:11:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:50 smithi173 ceph-mon[33187]: 5.14 deep-scrub ok 2024-09-17T08:11:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:50 smithi173 ceph-mon[33187]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:51.032 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-17T08:11:49.227102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1027,"num_read_kb":873,"num_write":609,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624,"ondisk_log_size":624,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239916,"kb_used_data":7380,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749721300,"statfs":{"total":767960285184,"available":767714611200,"internally_reserved":0,"allocated":7557120,"data_stored":4541138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002369"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.012978+0000","last_change":"2024-09-17T08:11:34.012978+0000","last_active":"2024-09-17T08:11:34.012978+0000","last_peered":"2024-09-17T08:11:34.012978+0000","last_clean":"2024-09-17T08:11:34.012978+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:11:34.012978+0000","last_undegraded":"2024-09-17T08:11:34.012978+0000","last_fullsized":"2024-09-17T08:11:34.012978+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.012877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.012877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.356338+0000","scrub_duration":0.001297653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435496+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:11:32.435496+0000","last_peered":"2024-09-17T08:11:32.435496+0000","last_clean":"2024-09-17T08:11:32.435496+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:11:32.435496+0000","last_undegraded":"2024-09-17T08:11:32.435496+0000","last_fullsized":"2024-09-17T08:11:32.435496+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:05.431600+0000","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430656+0000","last_change":"2024-09-17T08:09:56.504496+0000","last_active":"2024-09-17T08:11:32.430656+0000","last_peered":"2024-09-17T08:11:32.430656+0000","last_clean":"2024-09-17T08:11:32.430656+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:11:32.430656+0000","last_undegraded":"2024-09-17T08:11:32.430656+0000","last_fullsized":"2024-09-17T08:11:32.430656+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:09:55.487348+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429108+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:11:32.429108+0000","last_peered":"2024-09-17T08:11:32.429108+0000","last_clean":"2024-09-17T08:11:32.429108+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:11:32.429108+0000","last_undegraded":"2024-09-17T08:11:32.429108+0000","last_fullsized":"2024-09-17T08:11:32.429108+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431655+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:11:32.431655+0000","last_peered":"2024-09-17T08:11:32.431655+0000","last_clean":"2024-09-17T08:11:32.431655+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:11:32.431655+0000","last_undegraded":"2024-09-17T08:11:32.431655+0000","last_fullsized":"2024-09-17T08:11:32.431655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:33.629288+0000","last_change":"2024-09-17T08:11:33.629288+0000","last_active":"2024-09-17T08:11:33.629288+0000","last_peered":"2024-09-17T08:11:33.629288+0000","last_clean":"2024-09-17T08:11:33.629288+0000","last_became_active":"2024-09-17T08:11:32.433797+0000","last_became_peered":"2024-09-17T08:11:32.433797+0000","last_unstale":"2024-09-17T08:11:33.629288+0000","last_undegraded":"2024-09-17T08:11:33.629288+0000","last_fullsized":"2024-09-17T08:11:33.629288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.629197+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.320003+0000","scrub_duration":0.003700662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.913426+0000","last_change":"2024-09-17T08:11:34.913426+0000","last_active":"2024-09-17T08:11:34.913426+0000","last_peered":"2024-09-17T08:11:34.913426+0000","last_clean":"2024-09-17T08:11:34.913426+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:11:34.913426+0000","last_undegraded":"2024-09-17T08:11:34.913426+0000","last_fullsized":"2024-09-17T08:11:34.913426+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.913264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.913264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.562937+0000","scrub_duration":0.0017904069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429697+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:11:32.429697+0000","last_peered":"2024-09-17T08:11:32.429697+0000","last_clean":"2024-09-17T08:11:32.429697+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:11:32.429697+0000","last_undegraded":"2024-09-17T08:11:32.429697+0000","last_fullsized":"2024-09-17T08:11:32.429697+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.389268+0000","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436792+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:11:32.436792+0000","last_peered":"2024-09-17T08:11:32.436792+0000","last_clean":"2024-09-17T08:11:32.436792+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:11:32.436792+0000","last_undegraded":"2024-09-17T08:11:32.436792+0000","last_fullsized":"2024-09-17T08:11:32.436792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429295+0000","last_change":"2024-09-17T08:11:29.578149+0000","last_active":"2024-09-17T08:11:32.429295+0000","last_peered":"2024-09-17T08:11:32.429295+0000","last_clean":"2024-09-17T08:11:32.429295+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:11:32.429295+0000","last_undegraded":"2024-09-17T08:11:32.429295+0000","last_fullsized":"2024-09-17T08:11:32.429295+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.578069+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.180514+0000","scrub_duration":0.0025270700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436048+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:11:32.436048+0000","last_peered":"2024-09-17T08:11:32.436048+0000","last_clean":"2024-09-17T08:11:32.436048+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:11:32.436048+0000","last_undegraded":"2024-09-17T08:11:32.436048+0000","last_fullsized":"2024-09-17T08:11:32.436048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.446528+0000","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433285+0000","last_change":"2024-09-17T08:11:31.086448+0000","last_active":"2024-09-17T08:11:32.433285+0000","last_peered":"2024-09-17T08:11:32.433285+0000","last_clean":"2024-09-17T08:11:32.433285+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:11:32.433285+0000","last_undegraded":"2024-09-17T08:11:32.433285+0000","last_fullsized":"2024-09-17T08:11:32.433285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.086346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.086346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.290177+0000","scrub_duration":0.0013806619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436429+0000","last_change":"2024-09-17T08:11:28.306565+0000","last_active":"2024-09-17T08:11:32.436429+0000","last_peered":"2024-09-17T08:11:32.436429+0000","last_clean":"2024-09-17T08:11:32.436429+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:11:32.436429+0000","last_undegraded":"2024-09-17T08:11:32.436429+0000","last_fullsized":"2024-09-17T08:11:32.436429+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.306480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.306480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:53.967249+0000","scrub_duration":0.001181517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429846+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:11:32.429846+0000","last_peered":"2024-09-17T08:11:32.429846+0000","last_clean":"2024-09-17T08:11:32.429846+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:11:32.429846+0000","last_undegraded":"2024-09-17T08:11:32.429846+0000","last_fullsized":"2024-09-17T08:11:32.429846+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:11.408617+0000","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433553+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:11:32.433553+0000","last_peered":"2024-09-17T08:11:32.433553+0000","last_clean":"2024-09-17T08:11:32.433553+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:11:32.433553+0000","last_undegraded":"2024-09-17T08:11:32.433553+0000","last_fullsized":"2024-09-17T08:11:32.433553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433506+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:11:32.433506+0000","last_peered":"2024-09-17T08:11:32.433506+0000","last_clean":"2024-09-17T08:11:32.433506+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:11:32.433506+0000","last_undegraded":"2024-09-17T08:11:32.433506+0000","last_fullsized":"2024-09-17T08:11:32.433506+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429896+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:11:32.429896+0000","last_peered":"2024-09-17T08:11:32.429896+0000","last_clean":"2024-09-17T08:11:32.429896+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:11:32.429896+0000","last_undegraded":"2024-09-17T08:11:32.429896+0000","last_fullsized":"2024-09-17T08:11:32.429896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436389+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:11:32.436389+0000","last_peered":"2024-09-17T08:11:32.436389+0000","last_clean":"2024-09-17T08:11:32.436389+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:11:32.436389+0000","last_undegraded":"2024-09-17T08:11:32.436389+0000","last_fullsized":"2024-09-17T08:11:32.436389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429146+0000","last_change":"2024-09-17T08:11:31.017724+0000","last_active":"2024-09-17T08:11:32.429146+0000","last_peered":"2024-09-17T08:11:32.429146+0000","last_clean":"2024-09-17T08:11:32.429146+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:11:32.429146+0000","last_undegraded":"2024-09-17T08:11:32.429146+0000","last_fullsized":"2024-09-17T08:11:32.429146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.017640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.017640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.994183+0000","scrub_duration":0.0020059460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434844+0000","last_change":"2024-09-17T08:11:28.097377+0000","last_active":"2024-09-17T08:11:32.434844+0000","last_peered":"2024-09-17T08:11:32.434844+0000","last_clean":"2024-09-17T08:11:32.434844+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:11:32.434844+0000","last_undegraded":"2024-09-17T08:11:32.434844+0000","last_fullsized":"2024-09-17T08:11:32.434844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.097289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.097289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.706372+0000","scrub_duration":0.001520696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429674+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:11:32.429674+0000","last_peered":"2024-09-17T08:11:32.429674+0000","last_clean":"2024-09-17T08:11:32.429674+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:11:32.429674+0000","last_undegraded":"2024-09-17T08:11:32.429674+0000","last_fullsized":"2024-09-17T08:11:32.429674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.907725+0000","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429468+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:11:32.429468+0000","last_peered":"2024-09-17T08:11:32.429468+0000","last_clean":"2024-09-17T08:11:32.429468+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:11:32.429468+0000","last_undegraded":"2024-09-17T08:11:32.429468+0000","last_fullsized":"2024-09-17T08:11:32.429468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.632543+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:11:41.632543+0000","last_peered":"2024-09-17T08:11:41.632543+0000","last_clean":"2024-09-17T08:11:41.632543+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:11:41.632543+0000","last_undegraded":"2024-09-17T08:11:41.632543+0000","last_fullsized":"2024-09-17T08:11:41.632543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:20.492012+0000","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429956+0000","last_change":"2024-09-17T08:10:00.523796+0000","last_active":"2024-09-17T08:11:32.429956+0000","last_peered":"2024-09-17T08:11:32.429956+0000","last_clean":"2024-09-17T08:11:32.429956+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:11:32.429956+0000","last_undegraded":"2024-09-17T08:11:32.429956+0000","last_fullsized":"2024-09-17T08:11:32.429956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429880+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:11:32.429880+0000","last_peered":"2024-09-17T08:11:32.429880+0000","last_clean":"2024-09-17T08:11:32.429880+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:11:32.429880+0000","last_undegraded":"2024-09-17T08:11:32.429880+0000","last_fullsized":"2024-09-17T08:11:32.429880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:43.931571+0000","last_change":"2024-09-17T08:11:43.931670+0000","last_active":"2024-09-17T08:11:43.931571+0000","last_peered":"2024-09-17T08:11:43.931571+0000","last_clean":"2024-09-17T08:11:43.931571+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:11:43.931571+0000","last_undegraded":"2024-09-17T08:11:43.931571+0000","last_fullsized":"2024-09-17T08:11:43.931571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:43.931558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.931558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:00.466096+0000","scrub_duration":0.0017476449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.454324+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:11:41.454324+0000","last_peered":"2024-09-17T08:11:41.454324+0000","last_clean":"2024-09-17T08:11:41.454324+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:11:41.454324+0000","last_undegraded":"2024-09-17T08:11:41.454324+0000","last_fullsized":"2024-09-17T08:11:41.454324+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433660+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:11:32.433660+0000","last_peered":"2024-09-17T08:11:32.433660+0000","last_clean":"2024-09-17T08:11:32.433660+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:11:32.433660+0000","last_undegraded":"2024-09-17T08:11:32.433660+0000","last_fullsized":"2024-09-17T08:11:32.433660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430056+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:11:32.430056+0000","last_peered":"2024-09-17T08:11:32.430056+0000","last_clean":"2024-09-17T08:11:32.430056+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:11:32.430056+0000","last_undegraded":"2024-09-17T08:11:32.430056+0000","last_fullsized":"2024-09-17T08:11:32.430056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435846+0000","last_change":"2024-09-17T08:11:31.305742+0000","last_active":"2024-09-17T08:11:32.435846+0000","last_peered":"2024-09-17T08:11:32.435846+0000","last_clean":"2024-09-17T08:11:32.435846+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:11:32.435846+0000","last_undegraded":"2024-09-17T08:11:32.435846+0000","last_fullsized":"2024-09-17T08:11:32.435846+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.305634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.305634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.476701+0000","scrub_duration":0.001519353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433558+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:11:32.433558+0000","last_peered":"2024-09-17T08:11:32.433558+0000","last_clean":"2024-09-17T08:11:32.433558+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:11:32.433558+0000","last_undegraded":"2024-09-17T08:11:32.433558+0000","last_fullsized":"2024-09-17T08:11:32.433558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430713+0000","last_change":"2024-09-17T08:10:00.598204+0000","last_active":"2024-09-17T08:11:32.430713+0000","last_peered":"2024-09-17T08:11:32.430713+0000","last_clean":"2024-09-17T08:11:32.430713+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:11:32.430713+0000","last_undegraded":"2024-09-17T08:11:32.430713+0000","last_fullsized":"2024-09-17T08:11:32.430713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429313+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:11:32.429313+0000","last_peered":"2024-09-17T08:11:32.429313+0000","last_clean":"2024-09-17T08:11:32.429313+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:11:32.429313+0000","last_undegraded":"2024-09-17T08:11:32.429313+0000","last_fullsized":"2024-09-17T08:11:32.429313+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432109+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:11:32.432109+0000","last_peered":"2024-09-17T08:11:32.432109+0000","last_clean":"2024-09-17T08:11:32.432109+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:11:32.432109+0000","last_undegraded":"2024-09-17T08:11:32.432109+0000","last_fullsized":"2024-09-17T08:11:32.432109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.406164+0000","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.632201+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:11:41.632201+0000","last_peered":"2024-09-17T08:11:41.632201+0000","last_clean":"2024-09-17T08:11:41.632201+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:11:41.632201+0000","last_undegraded":"2024-09-17T08:11:41.632201+0000","last_fullsized":"2024-09-17T08:11:41.632201+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436554+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:11:32.436554+0000","last_peered":"2024-09-17T08:11:32.436554+0000","last_clean":"2024-09-17T08:11:32.436554+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:11:32.436554+0000","last_undegraded":"2024-09-17T08:11:32.436554+0000","last_fullsized":"2024-09-17T08:11:32.436554+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:19.298629+0000","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430499+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:11:32.430499+0000","last_peered":"2024-09-17T08:11:32.430499+0000","last_clean":"2024-09-17T08:11:32.430499+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:11:32.430499+0000","last_undegraded":"2024-09-17T08:11:32.430499+0000","last_fullsized":"2024-09-17T08:11:32.430499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:21.039434+0000","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:33.944624+0000","last_change":"2024-09-17T08:11:33.944624+0000","last_active":"2024-09-17T08:11:33.944624+0000","last_peered":"2024-09-17T08:11:33.944624+0000","last_clean":"2024-09-17T08:11:33.944624+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:11:33.944624+0000","last_undegraded":"2024-09-17T08:11:33.944624+0000","last_fullsized":"2024-09-17T08:11:33.944624+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:33.944526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.944526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:03.130795+0000","scrub_duration":0.00148507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:43.596511+0000","last_change":"2024-09-17T08:11:43.596598+0000","last_active":"2024-09-17T08:11:43.596511+0000","last_peered":"2024-09-17T08:11:43.596511+0000","last_clean":"2024-09-17T08:11:43.596511+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:11:43.596511+0000","last_undegraded":"2024-09-17T08:11:43.596511+0000","last_fullsized":"2024-09-17T08:11:43.596511+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.596489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.205254+0000","scrub_duration":0.001427177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:38.545547+0000","last_change":"2024-09-17T08:11:38.545547+0000","last_active":"2024-09-17T08:11:38.545547+0000","last_peered":"2024-09-17T08:11:38.545547+0000","last_clean":"2024-09-17T08:11:38.545547+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:11:38.545547+0000","last_undegraded":"2024-09-17T08:11:38.545547+0000","last_fullsized":"2024-09-17T08:11:38.545547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:38.545458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:38.545458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.911421+0000","scrub_duration":0.0012391500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434370+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:11:32.434370+0000","last_peered":"2024-09-17T08:11:32.434370+0000","last_clean":"2024-09-17T08:11:32.434370+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:11:32.434370+0000","last_undegraded":"2024-09-17T08:11:32.434370+0000","last_fullsized":"2024-09-17T08:11:32.434370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.145593+0000","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436901+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:11:32.436901+0000","last_peered":"2024-09-17T08:11:32.436901+0000","last_clean":"2024-09-17T08:11:32.436901+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:11:32.436901+0000","last_undegraded":"2024-09-17T08:11:32.436901+0000","last_fullsized":"2024-09-17T08:11:32.436901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431578+0000","last_change":"2024-09-17T08:11:28.433770+0000","last_active":"2024-09-17T08:11:32.431578+0000","last_peered":"2024-09-17T08:11:32.431578+0000","last_clean":"2024-09-17T08:11:32.431578+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:11:32.431578+0000","last_undegraded":"2024-09-17T08:11:32.431578+0000","last_fullsized":"2024-09-17T08:11:32.431578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.433666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.433666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.684654+0000","scrub_duration":0.001195822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429386+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:11:32.429386+0000","last_peered":"2024-09-17T08:11:32.429386+0000","last_clean":"2024-09-17T08:11:32.429386+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:11:32.429386+0000","last_undegraded":"2024-09-17T08:11:32.429386+0000","last_fullsized":"2024-09-17T08:11:32.429386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429329+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:11:32.429329+0000","last_peered":"2024-09-17T08:11:32.429329+0000","last_clean":"2024-09-17T08:11:32.429329+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:11:32.429329+0000","last_undegraded":"2024-09-17T08:11:32.429329+0000","last_fullsized":"2024-09-17T08:11:32.429329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429839+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:11:32.429839+0000","last_peered":"2024-09-17T08:11:32.429839+0000","last_clean":"2024-09-17T08:11:32.429839+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:11:32.429839+0000","last_undegraded":"2024-09-17T08:11:32.429839+0000","last_fullsized":"2024-09-17T08:11:32.429839+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.454502+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:11:41.454502+0000","last_peered":"2024-09-17T08:11:41.454502+0000","last_clean":"2024-09-17T08:11:41.454502+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:11:41.454502+0000","last_undegraded":"2024-09-17T08:11:41.454502+0000","last_fullsized":"2024-09-17T08:11:41.454502+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433477+0000","last_change":"2024-09-17T08:11:32.080416+0000","last_active":"2024-09-17T08:11:32.433477+0000","last_peered":"2024-09-17T08:11:32.433477+0000","last_clean":"2024-09-17T08:11:32.433477+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:11:32.433477+0000","last_undegraded":"2024-09-17T08:11:32.433477+0000","last_fullsized":"2024-09-17T08:11:32.433477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.080269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.080269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.073092+0000","scrub_duration":0.001335506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":112,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.455001+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:11:41.455001+0000","last_peered":"2024-09-17T08:11:41.455001+0000","last_clean":"2024-09-17T08:11:41.455001+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:11:41.455001+0000","last_undegraded":"2024-09-17T08:11:41.455001+0000","last_fullsized":"2024-09-17T08:11:41.455001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434142+0000","last_change":"2024-09-17T08:10:56.273278+0000","last_active":"2024-09-17T08:11:32.434142+0000","last_peered":"2024-09-17T08:11:32.434142+0000","last_clean":"2024-09-17T08:11:32.434142+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:11:32.434142+0000","last_undegraded":"2024-09-17T08:11:32.434142+0000","last_fullsized":"2024-09-17T08:11:32.434142+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.273179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.273179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:59.432786+0000","scrub_duration":0.0029745990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435074+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:11:32.435074+0000","last_peered":"2024-09-17T08:11:32.435074+0000","last_clean":"2024-09-17T08:11:32.435074+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:11:32.435074+0000","last_undegraded":"2024-09-17T08:11:32.435074+0000","last_fullsized":"2024-09-17T08:11:32.435074+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431372+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:11:32.431372+0000","last_peered":"2024-09-17T08:11:32.431372+0000","last_clean":"2024-09-17T08:11:32.431372+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:11:32.431372+0000","last_undegraded":"2024-09-17T08:11:32.431372+0000","last_fullsized":"2024-09-17T08:11:32.431372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:14.667094+0000","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.025771+0000","last_change":"2024-09-17T08:11:34.025771+0000","last_active":"2024-09-17T08:11:34.025771+0000","last_peered":"2024-09-17T08:11:34.025771+0000","last_clean":"2024-09-17T08:11:34.025771+0000","last_became_active":"2024-09-17T08:11:32.435377+0000","last_became_peered":"2024-09-17T08:11:32.435377+0000","last_unstale":"2024-09-17T08:11:34.025771+0000","last_undegraded":"2024-09-17T08:11:34.025771+0000","last_fullsized":"2024-09-17T08:11:34.025771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:11:34.025633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.025633+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.265538+0000","scrub_duration":0.0023641140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":158,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:46.203267+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:11:46.203267+0000","last_peered":"2024-09-17T08:11:46.203267+0000","last_clean":"2024-09-17T08:11:46.203267+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:11:46.203267+0000","last_undegraded":"2024-09-17T08:11:46.203267+0000","last_fullsized":"2024-09-17T08:11:46.203267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":106,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432504+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:11:32.432504+0000","last_peered":"2024-09-17T08:11:32.432504+0000","last_clean":"2024-09-17T08:11:32.432504+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:11:32.432504+0000","last_undegraded":"2024-09-17T08:11:32.432504+0000","last_fullsized":"2024-09-17T08:11:32.432504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434635+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:11:32.434635+0000","last_peered":"2024-09-17T08:11:32.434635+0000","last_clean":"2024-09-17T08:11:32.434635+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:11:32.434635+0000","last_undegraded":"2024-09-17T08:11:32.434635+0000","last_fullsized":"2024-09-17T08:11:32.434635+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436172+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:11:32.436172+0000","last_peered":"2024-09-17T08:11:32.436172+0000","last_clean":"2024-09-17T08:11:32.436172+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:11:32.436172+0000","last_undegraded":"2024-09-17T08:11:32.436172+0000","last_fullsized":"2024-09-17T08:11:32.436172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435394+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:11:32.435394+0000","last_peered":"2024-09-17T08:11:32.435394+0000","last_clean":"2024-09-17T08:11:32.435394+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:11:32.435394+0000","last_undegraded":"2024-09-17T08:11:32.435394+0000","last_fullsized":"2024-09-17T08:11:32.435394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433600+0000","last_change":"2024-09-17T08:11:28.111674+0000","last_active":"2024-09-17T08:11:32.433600+0000","last_peered":"2024-09-17T08:11:32.433600+0000","last_clean":"2024-09-17T08:11:32.433600+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:11:32.433600+0000","last_undegraded":"2024-09-17T08:11:32.433600+0000","last_fullsized":"2024-09-17T08:11:32.433600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:11:28.111569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.111569+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.306471+0000","scrub_duration":0.0016978239999999999,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.356211+0000","last_change":"2024-09-17T08:11:34.356211+0000","last_active":"2024-09-17T08:11:34.356211+0000","last_peered":"2024-09-17T08:11:34.356211+0000","last_clean":"2024-09-17T08:11:34.356211+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:11:34.356211+0000","last_undegraded":"2024-09-17T08:11:34.356211+0000","last_fullsized":"2024-09-17T08:11:34.356211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.356129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.356129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.828603+0000","scrub_duration":0.001774224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433717+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:11:32.433717+0000","last_peered":"2024-09-17T08:11:32.433717+0000","last_clean":"2024-09-17T08:11:32.433717+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:11:32.433717+0000","last_undegraded":"2024-09-17T08:11:32.433717+0000","last_fullsized":"2024-09-17T08:11:32.433717+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:40.881867+0000","last_change":"2024-09-17T08:11:40.881867+0000","last_active":"2024-09-17T08:11:40.881867+0000","last_peered":"2024-09-17T08:11:40.881867+0000","last_clean":"2024-09-17T08:11:40.881867+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:11:40.881867+0000","last_undegraded":"2024-09-17T08:11:40.881867+0000","last_fullsized":"2024-09-17T08:11:40.881867+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:40.881773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.881773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:47.568816+0000","scrub_duration":0.001434877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431963+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:11:32.431963+0000","last_peered":"2024-09-17T08:11:32.431963+0000","last_clean":"2024-09-17T08:11:32.431963+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:11:32.431963+0000","last_undegraded":"2024-09-17T08:11:32.431963+0000","last_fullsized":"2024-09-17T08:11:32.431963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.847283+0000","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430686+0000","last_change":"2024-09-17T08:09:58.506228+0000","last_active":"2024-09-17T08:11:32.430686+0000","last_peered":"2024-09-17T08:11:32.430686+0000","last_clean":"2024-09-17T08:11:32.430686+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:11:32.430686+0000","last_undegraded":"2024-09-17T08:11:32.430686+0000","last_fullsized":"2024-09-17T08:11:32.430686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433776+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:11:32.433776+0000","last_peered":"2024-09-17T08:11:32.433776+0000","last_clean":"2024-09-17T08:11:32.433776+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:11:32.433776+0000","last_undegraded":"2024-09-17T08:11:32.433776+0000","last_fullsized":"2024-09-17T08:11:32.433776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":306,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.633372+0000","last_change":"2024-09-17T08:11:40.405063+0000","last_active":"2024-09-17T08:11:41.633372+0000","last_peered":"2024-09-17T08:11:41.633372+0000","last_clean":"2024-09-17T08:11:41.633372+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:11:41.633372+0000","last_undegraded":"2024-09-17T08:11:41.633372+0000","last_fullsized":"2024-09-17T08:11:41.633372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:11:40.404951+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.404951+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:09.296441+0000","scrub_duration":0.001835978,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:37.370612+0000","last_change":"2024-09-17T08:11:37.370612+0000","last_active":"2024-09-17T08:11:37.370612+0000","last_peered":"2024-09-17T08:11:37.370612+0000","last_clean":"2024-09-17T08:11:37.370612+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:11:37.370612+0000","last_undegraded":"2024-09-17T08:11:37.370612+0000","last_fullsized":"2024-09-17T08:11:37.370612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:11:37.370527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:55.452817+0000","scrub_duration":0.001376932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433735+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:11:32.433735+0000","last_peered":"2024-09-17T08:11:32.433735+0000","last_clean":"2024-09-17T08:11:32.433735+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:11:32.433735+0000","last_undegraded":"2024-09-17T08:11:32.433735+0000","last_fullsized":"2024-09-17T08:11:32.433735+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432075+0000","last_change":"2024-09-17T08:11:27.389593+0000","last_active":"2024-09-17T08:11:32.432075+0000","last_peered":"2024-09-17T08:11:32.432075+0000","last_clean":"2024-09-17T08:11:32.432075+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:11:32.432075+0000","last_undegraded":"2024-09-17T08:11:32.432075+0000","last_fullsized":"2024-09-17T08:11:32.432075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:27.389490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.389490+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.830431+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434301+0000","last_change":"2024-09-17T08:11:30.055058+0000","last_active":"2024-09-17T08:11:32.434301+0000","last_peered":"2024-09-17T08:11:32.434301+0000","last_clean":"2024-09-17T08:11:32.434301+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:11:32.434301+0000","last_undegraded":"2024-09-17T08:11:32.434301+0000","last_fullsized":"2024-09-17T08:11:32.434301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:30.054968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.054968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.938144+0000","scrub_duration":0.0018411460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432523+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:11:32.432523+0000","last_peered":"2024-09-17T08:11:32.432523+0000","last_clean":"2024-09-17T08:11:32.432523+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:11:32.432523+0000","last_undegraded":"2024-09-17T08:11:32.432523+0000","last_fullsized":"2024-09-17T08:11:32.432523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436319+0000","last_change":"2024-09-17T08:11:27.312687+0000","last_active":"2024-09-17T08:11:32.436319+0000","last_peered":"2024-09-17T08:11:32.436319+0000","last_clean":"2024-09-17T08:11:32.436319+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:11:32.436319+0000","last_undegraded":"2024-09-17T08:11:32.436319+0000","last_fullsized":"2024-09-17T08:11:32.436319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.312595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.312595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.098135+0000","scrub_duration":0.001815178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434286+0000","last_change":"2024-09-17T08:11:30.115032+0000","last_active":"2024-09-17T08:11:32.434286+0000","last_peered":"2024-09-17T08:11:32.434286+0000","last_clean":"2024-09-17T08:11:32.434286+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:11:32.434286+0000","last_undegraded":"2024-09-17T08:11:32.434286+0000","last_fullsized":"2024-09-17T08:11:32.434286+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:30.114921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.114921+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.519251+0000","scrub_duration":0.0019919109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430028+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:11:32.430028+0000","last_peered":"2024-09-17T08:11:32.430028+0000","last_clean":"2024-09-17T08:11:32.430028+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:11:32.430028+0000","last_undegraded":"2024-09-17T08:11:32.430028+0000","last_fullsized":"2024-09-17T08:11:32.430028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432733+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:11:32.432733+0000","last_peered":"2024-09-17T08:11:32.432733+0000","last_clean":"2024-09-17T08:11:32.432733+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:11:32.432733+0000","last_undegraded":"2024-09-17T08:11:32.432733+0000","last_fullsized":"2024-09-17T08:11:32.432733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433526+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:11:32.433526+0000","last_peered":"2024-09-17T08:11:32.433526+0000","last_clean":"2024-09-17T08:11:32.433526+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:11:32.433526+0000","last_undegraded":"2024-09-17T08:11:32.433526+0000","last_fullsized":"2024-09-17T08:11:32.433526+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433073+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:11:32.433073+0000","last_peered":"2024-09-17T08:11:32.433073+0000","last_clean":"2024-09-17T08:11:32.433073+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:11:32.433073+0000","last_undegraded":"2024-09-17T08:11:32.433073+0000","last_fullsized":"2024-09-17T08:11:32.433073+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434825+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:11:32.434825+0000","last_peered":"2024-09-17T08:11:32.434825+0000","last_clean":"2024-09-17T08:11:32.434825+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:11:32.434825+0000","last_undegraded":"2024-09-17T08:11:32.434825+0000","last_fullsized":"2024-09-17T08:11:32.434825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:10.718062+0000","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.632930+0000","last_change":"2024-09-17T08:11:36.379074+0000","last_active":"2024-09-17T08:11:41.632930+0000","last_peered":"2024-09-17T08:11:41.632930+0000","last_clean":"2024-09-17T08:11:41.632930+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:11:41.632930+0000","last_undegraded":"2024-09-17T08:11:41.632930+0000","last_fullsized":"2024-09-17T08:11:41.632930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:36.378990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.378990+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.391768+0000","scrub_duration":0.001580082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433266+0000","last_change":"2024-09-17T08:11:32.089192+0000","last_active":"2024-09-17T08:11:32.433266+0000","last_peered":"2024-09-17T08:11:32.433266+0000","last_clean":"2024-09-17T08:11:32.433266+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:11:32.433266+0000","last_undegraded":"2024-09-17T08:11:32.433266+0000","last_fullsized":"2024-09-17T08:11:32.433266+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.089096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.089096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.048086+0000","scrub_duration":0.0013942010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429187+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:11:32.429187+0000","last_peered":"2024-09-17T08:11:32.429187+0000","last_clean":"2024-09-17T08:11:32.429187+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:11:32.429187+0000","last_undegraded":"2024-09-17T08:11:32.429187+0000","last_fullsized":"2024-09-17T08:11:32.429187+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:16.111081+0000","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430382+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:11:32.430382+0000","last_peered":"2024-09-17T08:11:32.430382+0000","last_clean":"2024-09-17T08:11:32.430382+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:11:32.430382+0000","last_undegraded":"2024-09-17T08:11:32.430382+0000","last_fullsized":"2024-09-17T08:11:32.430382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:12.114155+0000","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432357+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:11:32.432357+0000","last_peered":"2024-09-17T08:11:32.432357+0000","last_clean":"2024-09-17T08:11:32.432357+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:11:32.432357+0000","last_undegraded":"2024-09-17T08:11:32.432357+0000","last_fullsized":"2024-09-17T08:11:32.432357+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430810+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:11:32.430810+0000","last_peered":"2024-09-17T08:11:32.430810+0000","last_clean":"2024-09-17T08:11:32.430810+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:11:32.430810+0000","last_undegraded":"2024-09-17T08:11:32.430810+0000","last_fullsized":"2024-09-17T08:11:32.430810+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:08.452803+0000","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435374+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:11:32.435374+0000","last_peered":"2024-09-17T08:11:32.435374+0000","last_clean":"2024-09-17T08:11:32.435374+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:11:32.435374+0000","last_undegraded":"2024-09-17T08:11:32.435374+0000","last_fullsized":"2024-09-17T08:11:32.435374+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432509+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:11:32.432509+0000","last_peered":"2024-09-17T08:11:32.432509+0000","last_clean":"2024-09-17T08:11:32.432509+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:11:32.432509+0000","last_undegraded":"2024-09-17T08:11:32.432509+0000","last_fullsized":"2024-09-17T08:11:32.432509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:06.518394+0000","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431218+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:11:32.431218+0000","last_peered":"2024-09-17T08:11:32.431218+0000","last_clean":"2024-09-17T08:11:32.431218+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:11:32.431218+0000","last_undegraded":"2024-09-17T08:11:32.431218+0000","last_fullsized":"2024-09-17T08:11:32.431218+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432417+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:11:32.432417+0000","last_peered":"2024-09-17T08:11:32.432417+0000","last_clean":"2024-09-17T08:11:32.432417+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:11:32.432417+0000","last_undegraded":"2024-09-17T08:11:32.432417+0000","last_fullsized":"2024-09-17T08:11:32.432417+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431819+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:11:32.431819+0000","last_peered":"2024-09-17T08:11:32.431819+0000","last_clean":"2024-09-17T08:11:32.431819+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:11:32.431819+0000","last_undegraded":"2024-09-17T08:11:32.431819+0000","last_fullsized":"2024-09-17T08:11:32.431819+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431419+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:11:32.431419+0000","last_peered":"2024-09-17T08:11:32.431419+0000","last_clean":"2024-09-17T08:11:32.431419+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:11:32.431419+0000","last_undegraded":"2024-09-17T08:11:32.431419+0000","last_fullsized":"2024-09-17T08:11:32.431419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433770+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:11:32.433770+0000","last_peered":"2024-09-17T08:11:32.433770+0000","last_clean":"2024-09-17T08:11:32.433770+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:11:32.433770+0000","last_undegraded":"2024-09-17T08:11:32.433770+0000","last_fullsized":"2024-09-17T08:11:32.433770+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.436044+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:11:32.436044+0000","last_peered":"2024-09-17T08:11:32.436044+0000","last_clean":"2024-09-17T08:11:32.436044+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:11:32.436044+0000","last_undegraded":"2024-09-17T08:11:32.436044+0000","last_fullsized":"2024-09-17T08:11:32.436044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431520+0000","last_change":"2024-09-17T08:11:29.979585+0000","last_active":"2024-09-17T08:11:32.431520+0000","last_peered":"2024-09-17T08:11:32.431520+0000","last_clean":"2024-09-17T08:11:32.431520+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:11:32.431520+0000","last_undegraded":"2024-09-17T08:11:32.431520+0000","last_fullsized":"2024-09-17T08:11:32.431520+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:29.979461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.979461+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.112016+0000","scrub_duration":0.0029734470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.438048+0000","last_change":"2024-09-17T08:11:32.437732+0000","last_active":"2024-09-17T08:11:32.438048+0000","last_peered":"2024-09-17T08:11:32.438048+0000","last_clean":"2024-09-17T08:11:32.438048+0000","last_became_active":"2024-09-17T08:11:32.437420+0000","last_became_peered":"2024-09-17T08:11:32.437420+0000","last_unstale":"2024-09-17T08:11:32.438048+0000","last_undegraded":"2024-09-17T08:11:32.438048+0000","last_fullsized":"2024-09-17T08:11:32.438048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:09.232110+0000","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.633579+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:11:41.633579+0000","last_peered":"2024-09-17T08:11:41.633579+0000","last_clean":"2024-09-17T08:11:41.633579+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:11:41.633579+0000","last_undegraded":"2024-09-17T08:11:41.633579+0000","last_fullsized":"2024-09-17T08:11:41.633579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435003+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:11:32.435003+0000","last_peered":"2024-09-17T08:11:32.435003+0000","last_clean":"2024-09-17T08:11:32.435003+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:11:32.435003+0000","last_undegraded":"2024-09-17T08:11:32.435003+0000","last_fullsized":"2024-09-17T08:11:32.435003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433982+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:11:32.433982+0000","last_peered":"2024-09-17T08:11:32.433982+0000","last_clean":"2024-09-17T08:11:32.433982+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:11:32.433982+0000","last_undegraded":"2024-09-17T08:11:32.433982+0000","last_fullsized":"2024-09-17T08:11:32.433982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.434589+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:11:32.434589+0000","last_peered":"2024-09-17T08:11:32.434589+0000","last_clean":"2024-09-17T08:11:32.434589+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:11:32.434589+0000","last_undegraded":"2024-09-17T08:11:32.434589+0000","last_fullsized":"2024-09-17T08:11:32.434589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"72'8","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.632752+0000","last_change":"2024-09-17T08:09:58.514156+0000","last_active":"2024-09-17T08:11:41.632752+0000","last_peered":"2024-09-17T08:11:41.632752+0000","last_clean":"2024-09-17T08:11:41.632752+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:11:41.632752+0000","last_undegraded":"2024-09-17T08:11:41.632752+0000","last_fullsized":"2024-09-17T08:11:41.632752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:44.613690+0000","last_change":"2024-09-17T08:11:44.613775+0000","last_active":"2024-09-17T08:11:44.613690+0000","last_peered":"2024-09-17T08:11:44.613690+0000","last_clean":"2024-09-17T08:11:44.613690+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:11:44.613690+0000","last_undegraded":"2024-09-17T08:11:44.613690+0000","last_fullsized":"2024-09-17T08:11:44.613690+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:44.613677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:44.613677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.055246+0000","scrub_duration":0.001294131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429277+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:11:32.429277+0000","last_peered":"2024-09-17T08:11:32.429277+0000","last_clean":"2024-09-17T08:11:32.429277+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:11:32.429277+0000","last_undegraded":"2024-09-17T08:11:32.429277+0000","last_fullsized":"2024-09-17T08:11:32.429277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:13.656091+0000","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432096+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:11:32.432096+0000","last_peered":"2024-09-17T08:11:32.432096+0000","last_clean":"2024-09-17T08:11:32.432096+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:11:32.432096+0000","last_undegraded":"2024-09-17T08:11:32.432096+0000","last_fullsized":"2024-09-17T08:11:32.432096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430901+0000","last_change":"2024-09-17T08:11:31.019796+0000","last_active":"2024-09-17T08:11:32.430901+0000","last_peered":"2024-09-17T08:11:32.430901+0000","last_clean":"2024-09-17T08:11:32.430901+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:11:32.430901+0000","last_undegraded":"2024-09-17T08:11:32.430901+0000","last_fullsized":"2024-09-17T08:11:32.430901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.019695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.019695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.782145+0000","scrub_duration":0.001088767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433359+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:11:32.433359+0000","last_peered":"2024-09-17T08:11:32.433359+0000","last_clean":"2024-09-17T08:11:32.433359+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:11:32.433359+0000","last_undegraded":"2024-09-17T08:11:32.433359+0000","last_fullsized":"2024-09-17T08:11:32.433359+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429426+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:11:32.429426+0000","last_peered":"2024-09-17T08:11:32.429426+0000","last_clean":"2024-09-17T08:11:32.429426+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:11:32.429426+0000","last_undegraded":"2024-09-17T08:11:32.429426+0000","last_fullsized":"2024-09-17T08:11:32.429426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429985+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:11:32.429985+0000","last_peered":"2024-09-17T08:11:32.429985+0000","last_clean":"2024-09-17T08:11:32.429985+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:11:32.429985+0000","last_undegraded":"2024-09-17T08:11:32.429985+0000","last_fullsized":"2024-09-17T08:11:32.429985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429141+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:11:32.429141+0000","last_peered":"2024-09-17T08:11:32.429141+0000","last_clean":"2024-09-17T08:11:32.429141+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:11:32.429141+0000","last_undegraded":"2024-09-17T08:11:32.429141+0000","last_fullsized":"2024-09-17T08:11:32.429141+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.590481+0000","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430552+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:11:32.430552+0000","last_peered":"2024-09-17T08:11:32.430552+0000","last_clean":"2024-09-17T08:11:32.430552+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:11:32.430552+0000","last_undegraded":"2024-09-17T08:11:32.430552+0000","last_fullsized":"2024-09-17T08:11:32.430552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:40.948461+0000","last_change":"2024-09-17T08:11:40.948461+0000","last_active":"2024-09-17T08:11:40.948461+0000","last_peered":"2024-09-17T08:11:40.948461+0000","last_clean":"2024-09-17T08:11:40.948461+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:11:40.948461+0000","last_undegraded":"2024-09-17T08:11:40.948461+0000","last_fullsized":"2024-09-17T08:11:40.948461+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:11:40.948380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.948380+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:05.132541+0000","scrub_duration":0.0015753480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433997+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:11:32.433997+0000","last_peered":"2024-09-17T08:11:32.433997+0000","last_clean":"2024-09-17T08:11:32.433997+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:11:32.433997+0000","last_undegraded":"2024-09-17T08:11:32.433997+0000","last_fullsized":"2024-09-17T08:11:32.433997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:17.132962+0000","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431526+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:11:32.431526+0000","last_peered":"2024-09-17T08:11:32.431526+0000","last_clean":"2024-09-17T08:11:32.431526+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:11:32.431526+0000","last_undegraded":"2024-09-17T08:11:32.431526+0000","last_fullsized":"2024-09-17T08:11:32.431526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435802+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:11:32.435802+0000","last_peered":"2024-09-17T08:11:32.435802+0000","last_clean":"2024-09-17T08:11:32.435802+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:11:32.435802+0000","last_undegraded":"2024-09-17T08:11:32.435802+0000","last_fullsized":"2024-09-17T08:11:32.435802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431714+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:11:32.431714+0000","last_peered":"2024-09-17T08:11:32.431714+0000","last_clean":"2024-09-17T08:11:32.431714+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:11:32.431714+0000","last_undegraded":"2024-09-17T08:11:32.431714+0000","last_fullsized":"2024-09-17T08:11:32.431714+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432114+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:11:32.432114+0000","last_peered":"2024-09-17T08:11:32.432114+0000","last_clean":"2024-09-17T08:11:32.432114+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:11:32.432114+0000","last_undegraded":"2024-09-17T08:11:32.432114+0000","last_fullsized":"2024-09-17T08:11:32.432114+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435557+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:11:32.435557+0000","last_peered":"2024-09-17T08:11:32.435557+0000","last_clean":"2024-09-17T08:11:32.435557+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:11:32.435557+0000","last_undegraded":"2024-09-17T08:11:32.435557+0000","last_fullsized":"2024-09-17T08:11:32.435557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430716+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:11:32.430716+0000","last_peered":"2024-09-17T08:11:32.430716+0000","last_clean":"2024-09-17T08:11:32.430716+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:11:32.430716+0000","last_undegraded":"2024-09-17T08:11:32.430716+0000","last_fullsized":"2024-09-17T08:11:32.430716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:43.441969+0000","last_change":"2024-09-17T08:11:43.442061+0000","last_active":"2024-09-17T08:11:43.441969+0000","last_peered":"2024-09-17T08:11:43.441969+0000","last_clean":"2024-09-17T08:11:43.441969+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:11:43.441969+0000","last_undegraded":"2024-09-17T08:11:43.441969+0000","last_fullsized":"2024-09-17T08:11:43.441969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:11:43.441954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.441954+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:08.765259+0000","scrub_duration":0.0019032389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.433493+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:11:32.433493+0000","last_peered":"2024-09-17T08:11:32.433493+0000","last_clean":"2024-09-17T08:11:32.433493+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:11:32.433493+0000","last_undegraded":"2024-09-17T08:11:32.433493+0000","last_fullsized":"2024-09-17T08:11:32.433493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435284+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:11:32.435284+0000","last_peered":"2024-09-17T08:11:32.435284+0000","last_clean":"2024-09-17T08:11:32.435284+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:11:32.435284+0000","last_undegraded":"2024-09-17T08:11:32.435284+0000","last_fullsized":"2024-09-17T08:11:32.435284+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T08:11:41.632691+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:11:41.632691+0000","last_peered":"2024-09-17T08:11:41.632691+0000","last_clean":"2024-09-17T08:11:41.632691+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:11:41.632691+0000","last_undegraded":"2024-09-17T08:11:41.632691+0000","last_fullsized":"2024-09-17T08:11:41.632691+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429534+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:11:32.429534+0000","last_peered":"2024-09-17T08:11:32.429534+0000","last_clean":"2024-09-17T08:11:32.429534+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:11:32.429534+0000","last_undegraded":"2024-09-17T08:11:32.429534+0000","last_fullsized":"2024-09-17T08:11:32.429534+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431248+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:11:32.431248+0000","last_peered":"2024-09-17T08:11:32.431248+0000","last_clean":"2024-09-17T08:11:32.431248+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:11:32.431248+0000","last_undegraded":"2024-09-17T08:11:32.431248+0000","last_fullsized":"2024-09-17T08:11:32.431248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432004+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:11:32.432004+0000","last_peered":"2024-09-17T08:11:32.432004+0000","last_clean":"2024-09-17T08:11:32.432004+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:11:32.432004+0000","last_undegraded":"2024-09-17T08:11:32.432004+0000","last_fullsized":"2024-09-17T08:11:32.432004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.435289+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:11:32.435289+0000","last_peered":"2024-09-17T08:11:32.435289+0000","last_clean":"2024-09-17T08:11:32.435289+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:11:32.435289+0000","last_undegraded":"2024-09-17T08:11:32.435289+0000","last_fullsized":"2024-09-17T08:11:32.435289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:15.016924+0000","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:36.624844+0000","last_change":"2024-09-17T08:11:36.624844+0000","last_active":"2024-09-17T08:11:36.624844+0000","last_peered":"2024-09-17T08:11:36.624844+0000","last_clean":"2024-09-17T08:11:36.624844+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:11:36.624844+0000","last_undegraded":"2024-09-17T08:11:36.624844+0000","last_fullsized":"2024-09-17T08:11:36.624844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.624764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:59.550403+0000","scrub_duration":0.00124591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.432177+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:11:32.432177+0000","last_peered":"2024-09-17T08:11:32.432177+0000","last_clean":"2024-09-17T08:11:32.432177+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:11:32.432177+0000","last_undegraded":"2024-09-17T08:11:32.432177+0000","last_fullsized":"2024-09-17T08:11:32.432177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":99,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:34.992216+0000","last_change":"2024-09-17T08:11:34.992216+0000","last_active":"2024-09-17T08:11:34.992216+0000","last_peered":"2024-09-17T08:11:34.992216+0000","last_clean":"2024-09-17T08:11:34.992216+0000","last_became_active":"2024-09-17T08:11:32.436438+0000","last_became_peered":"2024-09-17T08:11:32.436438+0000","last_unstale":"2024-09-17T08:11:34.992216+0000","last_undegraded":"2024-09-17T08:11:34.992216+0000","last_fullsized":"2024-09-17T08:11:34.992216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.992118+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.249927+0000","scrub_duration":0.0049126200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:38.885523+0000","last_change":"2024-09-17T08:11:38.885523+0000","last_active":"2024-09-17T08:11:38.885523+0000","last_peered":"2024-09-17T08:11:38.885523+0000","last_clean":"2024-09-17T08:11:38.885523+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:11:38.885523+0000","last_undegraded":"2024-09-17T08:11:38.885523+0000","last_fullsized":"2024-09-17T08:11:38.885523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:38.885436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:38.885436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:01.914843+0000","scrub_duration":0.0016837029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431199+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:11:32.431199+0000","last_peered":"2024-09-17T08:11:32.431199+0000","last_clean":"2024-09-17T08:11:32.431199+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:11:32.431199+0000","last_undegraded":"2024-09-17T08:11:32.431199+0000","last_fullsized":"2024-09-17T08:11:32.431199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.431290+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:11:32.431290+0000","last_peered":"2024-09-17T08:11:32.431290+0000","last_clean":"2024-09-17T08:11:32.431290+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:11:32.431290+0000","last_undegraded":"2024-09-17T08:11:32.431290+0000","last_fullsized":"2024-09-17T08:11:32.431290+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:02.055944+0000","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.430850+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:11:32.430850+0000","last_peered":"2024-09-17T08:11:32.430850+0000","last_clean":"2024-09-17T08:11:32.430850+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:11:32.430850+0000","last_undegraded":"2024-09-17T08:11:32.430850+0000","last_fullsized":"2024-09-17T08:11:32.430850+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T08:11:32.429639+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:11:32.429639+0000","last_peered":"2024-09-17T08:11:32.429639+0000","last_clean":"2024-09-17T08:11:32.429639+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:11:32.429639+0000","last_undegraded":"2024-09-17T08:11:32.429639+0000","last_fullsized":"2024-09-17T08:11:32.429639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070492,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32400,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712752,"statfs":{"total":95995035648,"available":95961858048,"internally_reserved":0,"allocated":1314816,"data_stored":936840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":1.008,"15min":1.008},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70899999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55800000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.497}]},{"osd":5,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.503,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.62}]},{"osd":6,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65700000000000003,"5min":0.75,"15min":0.75},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60799999999999998}]}]},{"osd":6,"up_from":50,"seq":214748364833,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1302528,"data_stored":936431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.69399999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67600000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54100000000000004}]},{"osd":1,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.747,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66500000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.317,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.374},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40100000000000002,"15min":0.38700000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.34799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.495},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57999999999999996}]}]},{"osd":5,"up_from":41,"seq":176093659174,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31808,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713344,"statfs":{"total":95995035648,"available":95962464256,"internally_reserved":0,"allocated":708608,"data_stored":345518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61499999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.625},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.504},"min":{"1min":0.40999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:10:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.30399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66000000000000003,"5min":0.752,"15min":0.752},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.48099999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27724,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":724992,"data_stored":346641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.5,"15min":0.5},"min":{"1min":0.35599999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.37},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.436}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.39100000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.432},"min":{"1min":0.38300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:11:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.497}]}]},{"osd":3,"up_from":27,"seq":115964117039,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717412,"statfs":{"total":95995035648,"available":95966629888,"internally_reserved":0,"allocated":737280,"data_stored":346118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36299999999999999,"15min":0.371},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.496,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":0.46899999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46899999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.316},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54100000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.26800000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69299999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83299999999999996,"5min":0.873,"15min":0.873},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Sep 17 08:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.505}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27724,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":724992,"data_stored":346630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33100000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55300000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.52300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.29999999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40300000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.35299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.627},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.749,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80400000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31832,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713320,"statfs":{"total":95995035648,"available":95962439680,"internally_reserved":0,"allocated":733184,"data_stored":346378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.191,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.39400000000000002,"5min":0.625,"15min":0.625},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.312,"5min":0.33300000000000002,"15min":0.30299999999999999},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46100000000000002,"5min":0.625,"15min":0.625},"last":0.19600000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.34899999999999998},"min":{"1min":0.20699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.27400000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.26500000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.087,"5min":1.329,"15min":1.329},"last":0.70599999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.139,"15min":1.139},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1279999999999999,"5min":1.157,"15min":1.157},"last":0.504}]},{"osd":7,"last update":"Tue Sep 17 08:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28300,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716852,"statfs":{"total":95995035648,"available":95966056448,"internally_reserved":0,"allocated":1310720,"data_stored":936582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.313},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.33700000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.39900000000000002,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51100000000000001,"5min":0.626,"15min":0.626},"last":0.372}]},{"osd":3,"last update":"Tue Sep 17 08:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.42399999999999999,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.626},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.44500000000000001,"15min":0.46500000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":0.874},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Sep 17 08:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.755,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:11:51.040 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T08:11:51.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump 2024-09-17T08:11:51.253 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:51.300 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T08:11:51.300 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 systemd[1]: Stopped Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:51.300 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 systemd[1]: Starting Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:51.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:51 smithi079 ceph-mon[29565]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:51.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:51 smithi079 ceph-mon[40649]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:51 smithi173 ceph-mon[33187]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:11:51.869 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 podman[79797]: 2024-09-17 08:11:51.519476468 +0000 UTC m=+0.017414227 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-17T08:11:51.869 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 podman[79797]: 2024-09-17 08:11:51.765893601 +0000 UTC m=+0.263831402 volume create 66e6d1332bfad7f857ee05c2db3ce8d20117ccb1148591158d1d39c76834fbbd 2024-09-17T08:11:51.869 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:51 smithi079 podman[79797]: 2024-09-17 08:11:51.849319725 +0000 UTC m=+0.347257483 container create 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:52.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:52 smithi079 ceph-mon[40649]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:52 smithi079 ceph-mon[29565]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:52 smithi173 ceph-mon[33187]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:11:52.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:11:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:11:53.309 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:11:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 podman[79797]: 2024-09-17 08:11:53.168306149 +0000 UTC m=+1.666243907 container init 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 podman[79797]: 2024-09-17 08:11:53.174018877 +0000 UTC m=+1.671956645 container start 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.185Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.186Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.186Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.79 port=9094 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.187Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T08:11:53.309 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.209Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:11:53.310 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.210Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T08:11:53.310 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.212Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T08:11:53.310 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:53.212Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T08:11:53.499 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 88 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-09-17T08:04:41.066270+0000 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2024-09-17T08:11:41.450969+0000 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:crush_version 18 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:full_ratio 0.95 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:backfillfull_ratio 0.9 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:nearfull_ratio 0.85 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:require_min_compat_client luminous 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:min_compat_client luminous 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:require_osd_release reef 2024-09-17T08:11:53.500 INFO:teuthology.orchestra.run.smithi079.stdout:stretch_mode_enabled false 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 7.89 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 66 flags hashpspool stripe_width 0 application rgw read_balance_score 1.50 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 3 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 69 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.65 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 68 flags hashpspool stripe_width 0 application rgw read_balance_score 2.25 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 70 flags hashpspool stripe_width 0 application rgw read_balance_score 1.25 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 72 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 1.75 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:max_osd 8 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 up in weight 1 up_from 8 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997] [v2:172.21.15.79:6804/2553823997,v1:172.21.15.79:6805/2553823997] exists,up 793e335a-ea62-499c-af46-e48c3190408a 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 up in weight 1 up_from 13 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614] [v2:172.21.15.79:6812/221772614,v1:172.21.15.79:6813/221772614] exists,up dda3e35f-7364-47fa-af06-610c50ac856d 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652] [v2:172.21.15.79:6820/383883652,v1:172.21.15.79:6821/383883652] exists,up e121b6c9-f67b-4802-a92b-9c709ff433c2 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 up in weight 1 up_from 27 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570] [v2:172.21.15.79:6828/3442203570,v1:172.21.15.79:6829/3442203570] exists,up 69af955a-9dbe-4d86-8278-27df098a3886 2024-09-17T08:11:53.501 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 up in weight 1 up_from 32 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597] [v2:172.21.15.173:6802/2625332597,v1:172.21.15.173:6803/2625332597] exists,up 2ab07ff0-d7f3-4fca-a97b-843f7e69fc35 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 up in weight 1 up_from 41 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432] [v2:172.21.15.173:6810/329100432,v1:172.21.15.173:6811/329100432] exists,up 2fc412d9-7b28-43b0-b7d3-80fff14855ad 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6 up in weight 1 up_from 50 up_thru 68 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206] [v2:172.21.15.173:6818/1332566206,v1:172.21.15.173:6819/1332566206] exists,up 19197da7-927a-4422-8d18-ca9918136919 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7 up in weight 1 up_from 59 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325] [v2:172.21.15.173:6826/777262325,v1:172.21.15.173:6827/777262325] exists,up b32fec83-5977-488d-b36c-bba34b9f3dc0 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 2.12 [7,2] 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.7 [1,2] 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.18 [1,2] 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:pg_upmap_items 4.1c [1,5] 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3019822778 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3759482241 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/2023211636 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6801/673928727 expires 2024-09-18T08:05:34.382508+0000 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/128416645 expires 2024-09-18T08:05:11.956861+0000 2024-09-17T08:11:53.502 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/2554615564 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3326230156 expires 2024-09-18T08:05:11.956861+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6800/673928727 expires 2024-09-18T08:05:34.382508+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/1830656502 expires 2024-09-18T08:05:34.382508+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/1850619208 expires 2024-09-18T08:05:11.956861+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6800/610659042 expires 2024-09-18T08:05:11.956861+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6801/610659042 expires 2024-09-18T08:05:11.956861+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/2454308049 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/107735338 expires 2024-09-18T08:05:34.382508+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:0/3951775509 expires 2024-09-18T08:05:34.382508+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6800/3253737241 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.503 INFO:teuthology.orchestra.run.smithi079.stdout:blocklist 172.21.15.79:6801/3253737241 expires 2024-09-18T08:10:31.196134+0000 2024-09-17T08:11:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:53.620 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 bash[79797]: 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce 2024-09-17T08:11:53.620 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:53 smithi079 systemd[1]: Started Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:11:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:11:54.212 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6802/2553823997,v1:172.21.15.79:6803/2553823997] [v2:172.21.15.79:6804/2553823997,v1:172.21.15.79:6805/2553823997] exists,up 793e335a-ea62-499c-af46-e48c3190408a', 'osd.1 up in weight 1 up_from 13 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6810/221772614,v1:172.21.15.79:6811/221772614] [v2:172.21.15.79:6812/221772614,v1:172.21.15.79:6813/221772614] exists,up dda3e35f-7364-47fa-af06-610c50ac856d', 'osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6818/383883652,v1:172.21.15.79:6819/383883652] [v2:172.21.15.79:6820/383883652,v1:172.21.15.79:6821/383883652] exists,up e121b6c9-f67b-4802-a92b-9c709ff433c2', 'osd.3 up in weight 1 up_from 27 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6826/3442203570,v1:172.21.15.79:6827/3442203570] [v2:172.21.15.79:6828/3442203570,v1:172.21.15.79:6829/3442203570] exists,up 69af955a-9dbe-4d86-8278-27df098a3886', 'osd.4 up in weight 1 up_from 32 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/2625332597,v1:172.21.15.173:6801/2625332597] [v2:172.21.15.173:6802/2625332597,v1:172.21.15.173:6803/2625332597] exists,up 2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'osd.5 up in weight 1 up_from 41 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/329100432,v1:172.21.15.173:6809/329100432] [v2:172.21.15.173:6810/329100432,v1:172.21.15.173:6811/329100432] exists,up 2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'osd.6 up in weight 1 up_from 50 up_thru 68 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1332566206,v1:172.21.15.173:6817/1332566206] [v2:172.21.15.173:6818/1332566206,v1:172.21.15.173:6819/1332566206] exists,up 19197da7-927a-4422-8d18-ca9918136919', 'osd.7 up in weight 1 up_from 59 up_thru 70 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/777262325,v1:172.21.15.173:6825/777262325] [v2:172.21.15.173:6826/777262325,v1:172.21.15.173:6827/777262325] exists,up b32fec83-5977-488d-b36c-bba34b9f3dc0'] 2024-09-17T08:11:54.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon mon.a config show 2024-09-17T08:11:54.413 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:54.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[40649]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:54.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/541794327' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-17T08:11:54.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[29565]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:54.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/541794327' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-17T08:11:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:54 smithi173 ceph-mon[33187]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:11:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:11:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/541794327' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-17T08:11:55.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:55 smithi079 ceph-mon[29565]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:11:55.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:55 smithi079 ceph-mon[29565]: Reconfiguring daemon prometheus.a on smithi173 2024-09-17T08:11:55.615 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:55 smithi079 ceph-mon[40649]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:11:55.615 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:55 smithi079 ceph-mon[40649]: Reconfiguring daemon prometheus.a on smithi173 2024-09-17T08:11:55.615 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:11:55 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:11:55.187Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000312155s 2024-09-17T08:11:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:55 smithi173 ceph-mon[33187]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T08:11:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:55 smithi173 ceph-mon[33187]: Reconfiguring daemon prometheus.a on smithi173 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mon.a", 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster": "ceph", 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket_mode": "", 2024-09-17T08:11:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_client_required": "cephx, none", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_cluster_required": "cephx", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_debug": "true", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_required": "cephx", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_supported": "", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio": "true", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T08:11:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_reap_max": "16", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_async_discard": "false", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_block_size": "4096", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio": "false", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_enable_discard": "false", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry": "3", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash": "0", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring": "false", 2024-09-17T08:11:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_hipri": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_type": "", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_allocator": "hybrid", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_buffered_io": "true", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T08:11:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery": "false", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_sync_write": "false", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T08:11:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocator": "hybrid", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_create": "true", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_create": "false", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_path": "", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_size": "0", 2024-09-17T08:11:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_path": "", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_size": "107374182400", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_create": "false", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_path": "", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs": "true", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T08:11:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune": "true", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size": "0", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_type": "2q", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T08:11:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_clone_cow": "true", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_mode": "none", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T08:11:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_freelist": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T08:11:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T08:11:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fail_eio": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T08:11:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T08:11:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size": "0", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T08:11:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T08:11:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T08:11:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T08:11:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T08:11:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_mem": "512", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_tracing": "false", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T08:11:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T08:11:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_version": "2", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_signatures": "false", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_version": "2", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_signatures": "false", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_version": "2", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_sign_messages": "true", 2024-09-17T08:11:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "chdir": "", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_acl_type": "", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_asio_thread_count": "2", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_mid": "0.750000", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_size": "16384", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_caps_release_delay": "5", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_check_pool_perm": "true", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_force_sync_read": "false", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_getattr_caps": "false", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_features": "", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T08:11:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_remount": "false", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_dirsize_rbytes": "true", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_force_lazyio": "false", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_fs": "", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_release_failure": "false", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_inline_size": "4096", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mds_namespace": "", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_metadata": "", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_gid": "-1", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_timeout": "300", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_uid": "-1", 2024-09-17T08:11:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mountpoint": "/", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_notify_timeout": "10", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc": "true", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_objects": "1000", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_size": "209715200", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_permissions": "true", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota": "true", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota_df": "true", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_bytes": "0", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_periods": "4", 2024-09-17T08:11:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_min": "131072", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_reconnect_stale": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_shutdown_timeout": "30", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_snapdir": ".snap", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_tick_interval": "1", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_trace": "", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_faked_inos": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_random_mds": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_port": "12201", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_monitors": "default=true", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog": "false", 2024-09-17T08:11:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_level": "info", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "-", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network": "", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network_interface": "", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_isal": "false", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_level": "5", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zstd_level": "1", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-17T08:11:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_seastar_smp": "1", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook": "", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook_timeout": "10", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "daemonize": "false", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "daos_pool": "tank", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_alienstore": "0/5", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok": "1/5", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok_assert_abort": "false", 2024-09-17T08:11:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_auth": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bdev": "1/3", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluefs": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluestore": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_buffer": "0/1", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephsqlite": "0/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_client": "0/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_compressor": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_context": "0/1", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crush": "1/1", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crypto": "1/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cyanstore": "0/5", 2024-09-17T08:11:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_dpdk": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_eventtrace": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filer": "0/1", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filestore": "1/3", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_finisher": "1/1", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_fuse": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_javaclient": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journal": "1/3", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journaler": "0/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_kstore": "1/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_leveldb": "4/5", 2024-09-17T08:11:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_lockdep": "0/1", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mclock": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_balancer": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_locker": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_migrator": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_memstore": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgr": "2/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgrc": "1/5", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mon": "20/20", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_monc": "0/10", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ms": "1/1", 2024-09-17T08:11:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_none": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objclass": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objectcacher": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objecter": "0/1", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_optracker": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_osd": "1/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_paxos": "20/20", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_perfcounter": "1/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_prioritycache": "1/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rados": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_replay": "0/5", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_refs": "0/0", 2024-09-17T08:11:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_reserver": "1/1", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_access": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_flight": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_sync": "1/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rocksdb": "4/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore": "0/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_backref": "0/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cache": "0/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_device": "0/5", 2024-09-17T08:11:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_epm": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_journal": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_lba": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_odata": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_omap": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_onode": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_t": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_tm": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_striper": "0/1", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_test": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_throttle": "1/1", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_timer": "0/1", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_tp": "0/5", 2024-09-17T08:11:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "device_failure_prediction_mode": "none", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_graylog": "false", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_journald": "false", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_stderr": "true", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_syslog": "false", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event_tracing": "false", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_http_port": "9926", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_prio_limit": "5", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sort_metrics": "true", 2024-09-17T08:11:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_stats_period": "5", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "fake_statfs_for_testing": "0", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "fatal_signal_handlers": "true", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_purge_ops": "10", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_truncate_ops": "128", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_blackhole": "false", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_snap": "true", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_caller_concurrency": "10", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_omap_check": "false", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-17T08:11:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_verify_split": "false", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_dump_file": "", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fadvise": "true", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fail_eio": "true", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_size": "128", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap": "false", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_inject_stall": "0", 2024-09-17T08:11:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_parallel": "false", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_trailing": "false", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_writeahead": "false", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_kill_at": "0", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T08:11:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_merge_threshold": "-10", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_odsync_write": "false", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend_path": "", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T08:11:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_threads": "2", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_punch_hole": "false", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_ops": "50", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T08:11:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_seek_data_hole": "false", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc": "false", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_splice": "false", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_multiple": "2", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_rand_factor": "20", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_update_to": "1000", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T08:11:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_xfs_extsize": "false", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_zfs_snap": "false", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fio_dir": "/tmp/fio", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_allow_other": "true", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_big_writes": "true", 2024-09-17T08:11:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_debug": "false", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_default_permissions": "false", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_disable_pagecache": "false", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_max_write": "0", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_multithreaded": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_require_active_mds": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_set_user_groups": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_move": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_read": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_write": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_target_name": "ceph", 2024-09-17T08:11:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_file": "", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_inject_failure": "0", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_interval": "5", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "host": "smithi079", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T08:11:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "inject_early_sigterm": "false", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_agent_port": "6799", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_tracing_enable": "false", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_aio": "true", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_align_min_size": "65536", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_align": "true", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_size": "4096", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_dio": "true", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_discard": "false", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_force_aio": "false", 2024-09-17T08:11:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_ignore_corruption": "false", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_entries": "100", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_replay_from": "0", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_write_header_frequency": "0", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_zero_on_create": "false", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prefetch_periods": "10", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prezero_periods": "5", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_write_head_interval": "15", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "keyfile": "", 2024-09-17T08:11:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_backend": "rocksdb", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_bytes": "67108864", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_ops": "512", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_onode_map_size": "1024", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_transaction": "false", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_block_size": "65536", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_bloom_size": "0", 2024-09-17T08:11:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_cache_size": "536870912", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_compact_on_mount": "false", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_compression": "false", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_log": "", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_max_open_files": "0", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_paranoid": "false", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "librados_thread_count": "2", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep": "false", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep_force_backtrace": "false", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "log_coarse_timestamps": "true", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T08:11:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "log_flush_on_exit": "false", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_port": "12201", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_new": "1000", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_recent": "10000", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stderr_prefix": "", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": "true", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_graylog": "false", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_journald": "false", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_stderr": "false", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_syslog": "false", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T08:11:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_action_on_write_error": "1", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_alternate_name_max": "8192", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_asio_thread_count": "2", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_export_pin": "true", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_interval": "10", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max": "-1", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max_until": "-1", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_merge_size": "50", 2024-09-17T08:11:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_mode": "0", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_bits": "3", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_size": "10000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T08:11:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_mid": "0.700000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_interval": "1", 2024-09-17T08:11:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_auth_pins": "false", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_frag": "false", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_scatterstat": "false", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_subtrees": "false", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T08:11:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_default_dir_hash": "2", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_defer_session_stale": "true", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_entries": "0", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_prefetch": "true", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_early_reply": "true", 2024-09-17T08:11:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enable_op_tracker": "true", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enforce_unique_name": "true", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T08:11:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_health_dummy": "false", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_join_fs": "", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_journal_format": "1", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_create_at": "0", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_export_at": "0", 2024-09-17T08:11:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_import_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_link_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_openc_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_rename_at": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_events": "-1", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_segments": "128", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_pause": "false", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_segment_size": "0", 2024-09-17T08:11:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_requests": "100000", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_export_size": "20971520", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_file_recover": "32", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_files": "64", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops": "8192", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_metrics_update_interval": "2", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_per_client": "100", 2024-09-17T08:11:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_numa_node": "-1", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_duration": "600", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_size": "20", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_log_threshold": "5", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_grace": "15", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_interval": "5", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T08:11:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_caps": "30000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_interval": "1.000000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_gid": "0", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_uid": "0", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T08:11:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_shutdown_check": "0", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_skip_ino": "0", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T08:11:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_min_uid": "0", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_rstat": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_symlink_recovery": "true", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_exports": "0", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_fragments": "0", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_tick_interval": "5.000000", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_valgrind_exit": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_backtrace": "1", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_scatter": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_sessions": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool_debug": "false", 2024-09-17T08:11:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_set": "false", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_size": "65536", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_bytes": "134217728", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_messages": "512", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_disabled_modules": "", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T08:11:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_messages": "128", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_messages": "128", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_messages": "8192", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_pool": "true", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_standby_modules": "true", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_period": "5", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_threshold": "5", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_tick_period": "2", 2024-09-17T08:11:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_delete": "true", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_size_one": "false", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_bytes": "104857600", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_directed_command_retry": "2", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T08:11:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file_level": "debug", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_file": "false", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_journald": "true", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T08:11:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_start": "false", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_trim": "true", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cpu_threads": "4", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_crit": "5", 2024-09-17T08:11:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_warn": "5", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_json": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_extra_checks": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_quincy": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T08:11:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_down_mkfs_grace": "60", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_default_strategy": "1", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_timeout": "5.000000", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_enable_op_tracker": "true", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_fake_pool_delete": "false", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_force_quorum_join": "false", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_detail_to_clog": "true", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_log_update_period": "5", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_max_detail": "50", 2024-09-17T08:11:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog": "true", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host": "[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0]", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host_override": "", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_initial_members": "", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease": "5.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T08:11:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_full_interval": "50", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max": "10000", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max_summary": "50", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_epochs": "500", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_osd": "10000", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T08:11:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_autotune": "true", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_target": "2147483648", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_digest_period": "5", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_min_osdmap_epochs": "50", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_complaint_time": "30", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_duration": "600", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_size": "20", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T08:11:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_log_threshold": "5", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size": "500", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T08:11:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T08:11:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_report_timeout": "900", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T08:11:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_osds": "4", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T08:11:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_interval": "86400", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_max_keys": "100", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_timeout": "300", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_session_timeout": "300", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_smart_report_timeout": "5", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_size": "4", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_debug": "false", 2024-09-17T08:11:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T08:11:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T08:11:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T08:11:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T08:11:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T08:11:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T08:11:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_tick_interval": "5", 2024-09-17T08:11:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T08:11:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_older_version": "true", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": "", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T08:11:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_tracing_enabled": "false", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_op_threads": "3", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_cm": "false", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_device_name": "", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T08:11:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_sl": "3", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_type": "ib", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_reap_threshold": "5", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_before_connect": "false", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv4": "true", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv6": "false", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr1": "true", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr2": "true", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_max": "7568", 2024-09-17T08:11:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_min": "6800", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_count": "3", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_delay": "5", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_client": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mds": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mgr": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mon": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_osd": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_client_mode": "crc secure", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_type": "", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_compress_secure": "false", 2024-09-17T08:11:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_data": "true", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_header": "true", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bug": "true", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_old_message": "true", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T08:11:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hugepages": "", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_lro": "true", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_pmd": "", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_port_id": "0", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_on_send": "false", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T08:11:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_type": "", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_network_congestion": "0", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_socket_failures": "0", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_accept_failures": "4", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_backoff": "15.000000", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T08:11:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_mode": "none", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_min_cost": "65536", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_public_type": "", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_service_mode": "crc secure", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_nodelay": "true", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_type": "async+posix", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "no_config_file": "false", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T08:11:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_ops": "1024", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_timeout": "10.000000", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "objectstore_blackhole": "false", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "openssl_engine_opts": "", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T08:11:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_ops": "4", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_slop": "0.020000", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_asio_thread_count": "2", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_max": "512", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_min": "64", 2024-09-17T08:11:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_peering": "false", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_beacon_report_interval": "300", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_duration": "30", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blkin_trace_all": "false", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T08:11:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_update_on_start": "true", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_cap": "256", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_op_priority": "63", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_watch_timeout": "30", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_command_max_records": "256", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_compact_on_start": "false", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T08:11:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_on_start": "true", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_op_order": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T08:11:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_shutdown": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T08:11:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_notify_timeout": "30", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_enable_op_tracker": "true", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T08:11:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_info": "true", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown": "true", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_function_tracing": "false", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_grace": "20", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_interval": "6", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T08:11:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_stale": "600", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_size": "5120", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_kill_backfill_at": "0", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_cache_size": "50", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_dedup": "true", 2024-09-17T08:11:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max": "40", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_name_len": "100", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_size": "0", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_backfills": "1", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_count": "5", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_period": "600", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_name_len": "2048", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_size": "134217728", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T08:11:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pgls": "1024", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_cost": "8388608", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_objects": "10", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_scrubs": "3", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_size": "90", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T08:11:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_profile": "balanced", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T08:11:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_base": "805306368", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target": "4294967296", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_autotune": "false", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_recovery_priority": "0", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_interval": "5", 2024-09-17T08:11:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_cache_shards": "32", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_node": "-1", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objecter_finishers": "1", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore": "bluestore", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_fuse": "false", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_tracing": "false", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T08:11:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_duration": "600", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_size": "20", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_log_threshold": "5", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards": "0", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T08:11:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_timeout": "15", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_open_classes_on_start": "true", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_os_flags": "0", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_peering_op_priority": "255", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_priority": "5", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T08:11:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crimson": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T08:11:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flags": "0", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_min_size": "0", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_size": "3", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_type": "replicated", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T08:11:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_cost": "20971520", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active": "0", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_priority": "3", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_priority": "5", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T08:11:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_repair_during_recovery": "false", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_cost": "52428800", 2024-09-17T08:11:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_hour": "0", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_priority": "5", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T08:11:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_skip_data_digest": "false", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_smart_report_timeout": "5", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_priority": "5", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T08:11:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_transaction_size": "30", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tracing": "false", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_use_stale_snap": "false", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T08:11:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_kill_at": "0", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_max_join_drift": "10", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min": "500", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min_wait": "0.050000", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max": "500", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_min": "10", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_stash_full_interval": "25", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_max": "500", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_min": "250", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "perf": "true", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "pid_file": "", 2024-09-17T08:11:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "-", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrv": "", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "public_bind_addr": "-", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network": "", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network_interface": "", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_enabled": "false", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_mon_op_timeout": "0", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_osd_op_timeout": "0", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_tracing": "false", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_atime_update_interval": "60", 2024-09-17T08:11:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache": "true", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_policy": "writearound", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_size": "33554432", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T08:11:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_compression_hint": "none", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_clone_format": "auto", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_data_pool": "", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_features": "61", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_format": "2", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_map_options": "", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_order": "22", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_pool": "rbd", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T08:11:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_count": "0", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler": "simple", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T08:11:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_order": "24", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_pool": "", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_splay_width": "4", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T08:11:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T08:11:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T08:11:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_threads": "1", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_plugins": "", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T08:11:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T08:11:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_tracing": "false", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_names": "true", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_pool": "true", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_base_url": "", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_log_level": "", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_admin_entry": "admin", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_backend_store": "rados", 2024-09-17T08:11:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_barbican_url": "", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_beast_enable_async": "true", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_enabled": "true", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_lru_size": "10000", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_config_store": "rados", 2024-09-17T08:11:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_content_length_compat": "false", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cross_domain_policy": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T08:11:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T08:11:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T08:11:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T08:11:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_window": "30", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T08:11:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T08:11:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_name": "", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_s3website_name": "", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_ops_log": "false", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_static_website": "false", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_usage_log": "false", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_expose_bucket": "false", 2024-09-17T08:11:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_extended_http_attrs": "", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_filter": "none", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_objs": "32", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T08:11:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_init_timeout": "300", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_project": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_user": "", 2024-09-17T08:11:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_api_version": "2", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_url": "", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T08:11:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_objs": "32", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_worker": "3", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T08:11:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_http_headers": "", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_size": "0", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T08:11:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_listing_results": "1000", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_notify_retries": "10", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T08:11:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T08:11:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_control_oids": "8", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_numa_node": "-1", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_tracing": "false", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_token": "", 2024-09-17T08:11:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_url": "", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_rados": "false", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_continue": "true", 2024-09-17T08:11:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_tracing": "false", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm": "", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_id": "", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region": "", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T08:11:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_request_uri": "", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_resolve_cname": "false", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_run_sync_thread": "true", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T08:11:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_script_uri": "", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_service_provider_name": "", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_id": "", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_secret": "", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_entry": "sts", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_key": "sts", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T08:11:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_url": "", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_custom_header": "", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_need_stats": "true", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_tenant_name": "", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url": "", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T08:11:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_lease_period": "120", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_thread_pool_size": "512", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_comment": "", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_createby": "", 2024-09-17T08:11:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_encoding": "", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_flag": "false", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_origin": "", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_tracker": "", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_shards": "32", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_use_opa_authz": "false", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T08:11:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_unique_email": "true", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_verify_ssl": "true", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone": "", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_id": "", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup": "", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_id": "", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T08:11:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_block_size": "4096", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T08:11:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_partition_filters": "false", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_perf": "false", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "run_dir": "/var/run/ceph", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_block_create": "true", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T08:11:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_device_size": "53687091200", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_main_device_type": "SSD", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T08:11:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_segment_size": "67108864", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "set_keepcaps": "false", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "setgroup": "ceph", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser": "ceph", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser_match_path": "", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "thp": "false", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_default_timeout": "60", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "throttler_perf_counter": "true" 2024-09-17T08:11:56.198 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:11:56.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:56 smithi079 ceph-mon[29565]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:56.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:56 smithi079 ceph-mon[40649]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:56.660 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:56 smithi173 ceph-mon[33187]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:11:57.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T08:11:57.402 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:57.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:57 smithi079 ceph-mon[29565]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:11:57.524 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:57 smithi079 ceph-mon[40649]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:11:57.666 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:57 smithi173 ceph-mon[33187]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:11:59.010 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-17T08:11:59.010 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:11:59.018 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-17T08:11:59.019 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:11:59.027 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-17T08:11:59.027 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:11:59.178 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:58 smithi173 systemd[1]: Stopping Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.178Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T08:11:59.461 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.179Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T08:11:59.462 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.179Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T08:11:59.462 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.179Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T08:11:59.462 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[64413]: ts=2024-09-17T08:11:59.179Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T08:11:59.462 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 podman[67787]: 2024-09-17 08:11:59.189102039 +0000 UTC m=+0.121366490 container died d16f348816828155afc548b70a5719106d7cfb3809b8c2995d2e77575dd6eb1c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:59.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:11:59.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:11:59.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[29565]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T08:11:59.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/970389575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:11:59.616 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[29565]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:11:59.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:11:59.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:11:59.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[40649]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T08:11:59.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/970389575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:11:59.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:11:59 smithi079 ceph-mon[40649]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:11:59.618 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-17T08:11:59.629 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-17T08:11:59.658 INFO:tasks.radosbench:Beginning radosbench... 2024-09-17T08:11:59.658 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-17T08:11:59.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_0 16 2024-09-17T08:11:59.658 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T08:11:59.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:11:59.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:11:59.659 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T08:11:59.659 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:11:59.659 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:11:59.659 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-17T08:11:59.660 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T08:11:59.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:11:59.660 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T08:11:59.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:11:59.661 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T08:11:59.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:11:59.662 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T08:11:59.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:11:59.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:11:59.687 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:11:59.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:11:59.723 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 podman[67787]: 2024-09-17 08:11:59.559637327 +0000 UTC m=+0.491901770 container cleanup d16f348816828155afc548b70a5719106d7cfb3809b8c2995d2e77575dd6eb1c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:11:59.723 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 bash[67787]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T08:11:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:11:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:11:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-mon[33187]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T08:11:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/970389575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:11:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:11:59 smithi173 ceph-mon[33187]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:11:59.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:11:59.817 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:11:59.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:11:59.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:11:59.874 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:11:59.895 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:59.895 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:59.899 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:59.905 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:11:59.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:11:59.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:11:59.948 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:11:59.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:00.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:00.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:00.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:00.119 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:11:59 smithi173 podman[67804]: 2024-09-17 08:11:59.794253729 +0000 UTC m=+0.603469179 container remove d16f348816828155afc548b70a5719106d7cfb3809b8c2995d2e77575dd6eb1c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:12:00.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:00.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:00.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:00.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:00.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:00.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:00.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:00.442 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:00.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:00.544 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:00.565 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:00.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:00.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:00 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:12:00.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:00 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:12:00.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:00 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:12:00.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:00 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:12:00.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:00.688 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:00.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:00.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:00 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:12:00.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:00 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:12:00.724 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:00 smithi173 podman[67886]: 2024-09-17 08:12:00.372772365 +0000 UTC m=+0.557914006 container died 7e86f5198eb2781394583a717985c24663a2d89aa5165a613f2dc8fe971c2632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=dreamy_borg, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T08:12:00.724 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:00 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-17T08:12:00.724 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:00 smithi173 systemd[1]: Stopped Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:12:00.724 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:00 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.198s CPU time. 2024-09-17T08:12:00.724 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:00 smithi173 systemd[1]: Starting Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:12:00.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:00.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:00.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:00.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:00.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:00.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:00.936 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:01.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:01.058 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:01.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:01.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:01.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:01.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:01.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:01.306 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:01.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:01.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:01.429 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:01.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:01.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:01.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:01.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:01.588 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:12:01.588 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T08:12:01.588 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:12:01.589 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:01 smithi173 podman[68186]: 2024-09-17 08:12:01.363208417 +0000 UTC m=+0.018496351 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T08:12:01.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:01.677 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:01.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:01.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:01.800 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:01.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:01.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:01.924 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:01.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:01.925 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:02.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:02.048 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:02.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:02.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:02.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:02.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:02.274 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:02.295 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:02.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:02.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:12:02.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:12:02.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[29565]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:12:02.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:02.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:12:02.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:12:02.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[40649]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:12:02.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:02.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:02.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:02.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:02.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:02.474 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 podman[68186]: 2024-09-17 08:12:02.359542011 +0000 UTC m=+1.014829986 container create 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:12:02.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:12:02.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:12:02.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-mon[33187]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:12:02.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:02.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:02.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:02.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:02.706 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:02.728 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:02.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:02.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:02.850 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:02.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:02.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:02 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:02] ENGINE Bus STOPPING 2024-09-17T08:12:02.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:02.973 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 podman[68186]: 2024-09-17 08:12:02.586608405 +0000 UTC m=+1.241896347 container init 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:12:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 podman[68186]: 2024-09-17 08:12:02.59117553 +0000 UTC m=+1.246463466 container start 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:12:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.615Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.615Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.615Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi173 (none))" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.615Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.615Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.617Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.618Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.619Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.619Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.620Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.620Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.379µs 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.620Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.621Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-17T08:12:02.975 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.621Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.622Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.385µs wal_replay_duration=1.195398ms wbl_replay_duration=144ns total_replay_duration=1.22908ms 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.623Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.623Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.623Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.641Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.726111ms db_storage=1.215µs remote_storage=1.028µs web_handler=368ns query_engine=655ns scrape=205.672µs scrape_sd=95.402µs notify=10.788µs notify_sd=13.935µs rules=18.095739ms tracing=5.13µs 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.641Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:12:02.641Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 bash[68186]: 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 2024-09-17T08:12:02.976 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:12:02 smithi173 systemd[1]: Started Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:12:02.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:02.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:03.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:03.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:03.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:03.204 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:03.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:03.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:03.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:03.348 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:03.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:03.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:03.456 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:12:03.190Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003323727s 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STOPPED 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STARTING 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Serving on http://:::9283 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STARTED 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STOPPING 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:03.457 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:03.458 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:03.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:12:03.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:03.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:03.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:03.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:03.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:03.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:12:03.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:03.595 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:03.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.596 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:03.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:03.718 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:03.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:03.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:03.841 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:03.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:03.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:03.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:03.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:03.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:04.068 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:04.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:04.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:04.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:12:04.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STOPPED 2024-09-17T08:12:04.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STARTING 2024-09-17T08:12:04.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Serving on http://:::9283 2024-09-17T08:12:04.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STARTED 2024-09-17T08:12:04.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:03] ENGINE Bus STOPPING 2024-09-17T08:12:04.187 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_0' created 2024-09-17T08:12:04.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:04.214 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:04.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:04.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:04.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:04.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:04.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:04.460 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:04.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2582767196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.506 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:12:04.506 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:04] ENGINE Bus STOPPED 2024-09-17T08:12:04.506 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:04] ENGINE Bus STARTING 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2582767196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi079.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi173.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi173.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2582767196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-17T08:12:04.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:04.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.584 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:04.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:04.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:04.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:04.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:04.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:04.829 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:04.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:04.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:04] ENGINE Serving on http://:::9283 2024-09-17T08:12:04.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:04 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:12:04] ENGINE Bus STARTED 2024-09-17T08:12:04.931 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:04.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:04.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:04.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:05.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:05.077 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:05.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:05.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:05.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:05 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-17T08:12:05.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:05 smithi173 ceph-mon[33187]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T08:12:05.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:05.319 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:05 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-17T08:12:05.319 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:05 smithi079 ceph-mon[40649]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T08:12:05.320 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:05 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-17T08:12:05.320 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:05 smithi079 ceph-mon[29565]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T08:12:05.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:05.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:05.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:05.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:05.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.578 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:05.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:05.699 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:05.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:05.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-17T08:12:05.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:05.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:05.924 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:05.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:05.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:05.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:06.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:06.069 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:06.170 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:06.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:06.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:06.204 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:12:06.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:06.317 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:06.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:06.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:06 smithi173 ceph-mon[33187]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T08:12:06.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:06 smithi173 ceph-mon[33187]: pgmap v67: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:06.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:06.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:06.564 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:06.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[29565]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T08:12:06.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[29565]: pgmap v67: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[40649]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T08:12:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[40649]: pgmap v67: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:06.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:06.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:06.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:06.810 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:06.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:06.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:06.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:06.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:07.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:07.056 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:07.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:07.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:07.181 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:07.221 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:12:07.221 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:12:07.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:07.305 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:07.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:07.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:07.424 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:12:07.425 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:12:07.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:07.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:07.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T08:12:07.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.426 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:07 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.429 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:07.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1654268259' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:07 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:07.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:07.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:07.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:07.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:07.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.676 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:07.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:07.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:07.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.799 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:07.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:07.862 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:12:07.862 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:12:07.862 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-17T08:12:07.863 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:12:07.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:07.921 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:07.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:07.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:08.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:08.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.046 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:08.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:08.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:08.170 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:08.272 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:08.295 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:08.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:08.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:08 smithi079 ceph-mon[40649]: pgmap v69: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-17T08:12:08.317 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:08 smithi079 ceph-mon[29565]: pgmap v69: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-17T08:12:08.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:08.419 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:08.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:08.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:08 smithi173 ceph-mon[33187]: pgmap v69: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-17T08:12:08.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:08.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:08.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:08.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:08.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:12:08.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.672 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:08.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:08.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:08.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:08.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:08.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:08.920 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:08.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:08.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:08.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:12:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:12:08.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:12:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:12:08.957 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:12:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:12:08.975 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:12:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:12:08.993 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:12:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:12:09.013 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:12:09.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:09.023 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:09.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:09.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:09.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:09.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:09.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:09.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:09.287 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:09.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:09.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:09.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:09.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:09.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:09.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1449211100' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:09.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1449211100' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:09.478 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1449211100' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:09.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:09.554 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:09.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:09.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:09.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:09.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:09.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:09.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:09.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:09.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:09.924 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:09.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:09.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:10.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:10.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:10.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.053 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:10.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:10.175 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:10.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:10.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:10.298 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:10.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:10.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:10 smithi079 ceph-mon[40649]: pgmap v70: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:10.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:10 smithi079 ceph-mon[29565]: pgmap v70: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:10.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:10.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:10.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:10.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:10 smithi173 ceph-mon[33187]: pgmap v70: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:10.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:10.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:10.577 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:10.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:10.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:10.779 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:10.805 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:10.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:10.907 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:10.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:10.933 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:10.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:11.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:11.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:11.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:11.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:11.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.186 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:11.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:11.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:11.308 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:11.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:11.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:11.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:11.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:11.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:11.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:11.655 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:11.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:11.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:11.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:11.808 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:11.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:11.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:11.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:11.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:11.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:12.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:12.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:12.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:12.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:12:12.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:12.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.192 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:12.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:12.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:12.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:12.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:12.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:12.443 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:12.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:12.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:12 smithi173 ceph-mon[33187]: pgmap v71: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 726 B/s rd, 0 op/s 2024-09-17T08:12:12.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:12.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:12 smithi079 ceph-mon[40649]: pgmap v71: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 726 B/s rd, 0 op/s 2024-09-17T08:12:12.522 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:12 smithi079 ceph-mon[29565]: pgmap v71: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 726 B/s rd, 0 op/s 2024-09-17T08:12:12.545 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:12.569 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:12.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:12.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:12.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.693 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:12.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:12.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:12.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:12.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:12.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:12.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:12.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:12.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:13.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:13.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:13.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:13.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:13.198 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:13.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:13.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:13.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:13.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:13.366 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:13.366 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:12:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:12:13.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:13.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.452 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:13.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:13.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:13.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:13.575 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:13.672 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:12:13.672 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:12:13.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:13.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:13.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:13.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:13.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:13.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:13.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:13.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:13.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:13.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:14.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:14.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.077 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:14.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:14.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:14.199 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:14.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:14.250 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:14 smithi079 ceph-mon[40649]: pgmap v72: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:14.251 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:14 smithi079 ceph-mon[29565]: pgmap v72: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:14.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:12:14.301 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:14.327 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:14.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:14.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:14.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:14.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:14.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:14 smithi173 ceph-mon[33187]: pgmap v72: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:14.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:14.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:14.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:14.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:14.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:14.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.704 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:14.805 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:14.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:14.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:14.884 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:14.910 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:14.932 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:14.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:14.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:14.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:15.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:15.082 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:15.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:15.183 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:15.190 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:12:15.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:15.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:15.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:15.331 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:15.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:15.390 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:15.390 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:15.432 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:15.456 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:15.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:15.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:15.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:15.580 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:15.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:15.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:15.705 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:15.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:15.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:15.830 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:15.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:15.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:15.959 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:15.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:15.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:16.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:16.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:16.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:16.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:16.201 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-17T08:12:16.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:16.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:16.308 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:16.334 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:16.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:16.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:16.457 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:16.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T08:12:16.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: pgmap v74: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:16.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3549844449' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:16 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.559 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:16.585 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:16.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: pgmap v74: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3549844449' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2274426897' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T08:12:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: pgmap v74: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:12:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3549844449' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:16 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:12:16.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:16.712 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:16.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:16.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:16.838 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:16.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:16.940 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:16.963 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:16.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:16.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:17.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:17.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:17.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:17.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:17.211 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[40649]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[29565]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:17.212 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:17.215 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:17.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:17.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:17.338 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:17.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:17.440 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:17.457 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:17 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:12:17.457 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:17 smithi173 ceph-mon[33187]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T08:12:17.457 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:17.457 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:17.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:17.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:17.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:17.590 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:17.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.591 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:17.692 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:17.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:17.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:17.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:17.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:17.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:17.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:17.959 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:17.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:17.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:17.974 DEBUG:teuthology.orchestra.run.smithi079:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-17T08:12:17.974 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-17T08:12:17.975 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-17T08:12:17.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:12:17.977 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-17T08:12:17.977 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-17T08:12:18.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:18.070 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:hints = 1 2024-09-17T08:12:18.070 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-17T08:12:18.070 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object prefix: benchmark_data_smithi079_83551 2024-09-17T08:12:18.070 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:12:18.071 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 0 0 0 0 0 0 - 0 2024-09-17T08:12:18.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:18.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:18.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:18.222 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:18.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:18.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[29565]: pgmap v76: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[29565]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[40649]: pgmap v76: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:18.336 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:18 smithi079 ceph-mon[40649]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T08:12:18.363 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:18.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:18.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:12:18.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:18.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:18 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:18.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:18 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:18.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:18 smithi173 ceph-mon[33187]: pgmap v76: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:12:18.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:18 smithi173 ceph-mon[33187]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T08:12:18.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:18.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:18.608 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:18.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:18.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:18.736 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:18.768 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:18.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:18.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:18.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:18.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:18.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:19.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:19.036 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:19.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:19.070 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 1 16 3423 3407 212.926 212.938 0.00356649 0.00467933 2024-09-17T08:12:19.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:19.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:19.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:19.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.329 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:19.380 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:19 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:12:19.380 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:19 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:12:19.431 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:19.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.462 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:19.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:19.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:19 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:12:19.564 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:19.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:19.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:19.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:19.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:19.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:19.861 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:19.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:19.964 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:19.994 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:19.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:19.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:20.071 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 2 15 6669 6654 207.86 202.938 0.00689961 0.00479969 2024-09-17T08:12:20.096 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:20.125 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:20.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:20.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:20.268 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:20.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:20.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:20.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.408 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:20.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:20.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:20 smithi173 ceph-mon[33187]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:20.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:20 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2187020680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:20.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:20.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:20.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:20.611 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:12:20.611 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:12:20.611 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:12:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:12:20.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:12:20.614 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:20 smithi079 ceph-mon[40649]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:20.614 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:20 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2187020680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:20.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:20 smithi079 ceph-mon[29565]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:12:20.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:20 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2187020680' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:20.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:20.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:20.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:20.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:20.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:20.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:20.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:20.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:20.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:20.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:21.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:21.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 3 16 9614 9598 199.87 184 0.00548002 0.00499422 2024-09-17T08:12:21.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:21.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:21.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:21.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:21.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:21.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:21.352 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:21.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.354 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:21.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:21.494 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:21.531 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:21.533 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:21.548 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:21 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:21.548 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:21 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:21.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:21 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:21.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:21 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:21.596 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:21.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:21.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:21.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:21 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:21.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:21 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:21.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:21.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:21.868 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:21.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:21.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:21.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:21.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:22.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:22.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:22.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 4 16 12041 12025 187.824 151.688 0.00403755 0.00531515 2024-09-17T08:12:22.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:22.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:22.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:22.306 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:22.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:22.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:22.437 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:22.539 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:22.566 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[40649]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[40649]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[29565]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-09-17T08:12:22.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:22 smithi079 ceph-mon[29565]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T08:12:22.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:22.699 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:22.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:22.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:22 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:22 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:22 smithi173 ceph-mon[33187]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-09-17T08:12:22.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:22 smithi173 ceph-mon[33187]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T08:12:22.801 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:22.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:22.932 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:22.962 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:22.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:22.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:23.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:23.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 5 15 14870 14855 185.632 176.875 0.00945785 0.00537992 2024-09-17T08:12:23.092 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:23.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:23.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:23.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:23.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:23.249 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:12:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:12:23.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:12:23.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:23.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:23.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:23.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:23.536 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:23.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:23.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:23.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:23.677 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:23.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:23.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:23.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:23.812 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:23.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:23.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:23.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:23.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:7388 7.6 7:651c83e5:::benchmark_data_smithi079_83551_object7387:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:20.289972+0000", 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7269490360000002, 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016333329000000001, 2024-09-17T08:12:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7388 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.289972+0000", 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.289972+0000", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.289971+0000", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.289992+0000", 2024-09-17T08:12:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0653999999999999e-05 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.290004+0000", 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139e-05 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.290011+0000", 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5140000000000004e-06 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.292048+0000", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020364210000000001 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.292501+0000", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045328499999999999 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.294416+0000", 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019152699999999999 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.296249+0000", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018325730000000001 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.296279+0000", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0630000000000003e-05 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.306227+0000", 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099477860000000001 2024-09-17T08:12:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.306269+0000", 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1820000000000003e-05 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:20.306305+0000", 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6056999999999997e-05 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9883 7.6 7:65bde47f:::benchmark_data_smithi079_83551_object9882:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.169899+0000", 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8470221040000001, 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016426311999999998, 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9883 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169899+0000", 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169899+0000", 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169897+0000", 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169943+0000", 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5318999999999998e-05 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169955+0000", 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2245999999999999e-05 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.169962+0000", 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.092e-06 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.173692+0000", 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037299500000000001 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.173893+0000", 2024-09-17T08:12:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020069400000000001 2024-09-17T08:12:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.174017+0000", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012420899999999999 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.177914+0000", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038967030000000001 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.177921+0000", 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7719999999999994e-06 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186251+0000", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083299380000000003 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186292+0000", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1204000000000001e-05 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186325+0000", 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2495e-05 2024-09-17T08:12:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9900 7.6 7:654e80c7:::benchmark_data_smithi079_83551_object9899:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.181332+0000", 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8355891409999998, 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022099450999999999, 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9900 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181332+0000", 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181332+0000", 2024-09-17T08:12:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181331+0000", 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181361+0000", 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0843000000000003e-05 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181372+0000", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0132000000000001e-05 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.181378+0000", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3300000000000004e-06 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186634+0000", 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052559099999999999 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186742+0000", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108027 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186888+0000", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014636099999999999 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.188994+0000", 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021054390000000002 2024-09-17T08:12:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.194134+0000", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051402499999999999 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203143+0000", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090091709999999998 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203197+0000", 2024-09-17T08:12:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4231000000000003e-05 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203431+0000", 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023373600000000001 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9914 7.6 7:6bf03333:::benchmark_data_smithi079_83551_object9913:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.186938+0000", 2024-09-17T08:12:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8299831769999999, 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016690379000000002, 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9914 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186938+0000", 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186938+0000", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186937+0000", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186963+0000", 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6075999999999999e-05 2024-09-17T08:12:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186971+0000", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3329999999999997e-06 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.186977+0000", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4600000000000002e-06 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.187843+0000", 2024-09-17T08:12:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086581099999999997 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.187948+0000", 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105191 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.188058+0000", 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109663 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.192757+0000", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046999099999999999 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.194170+0000", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001412878 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203476+0000", 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093058600000000009 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203594+0000", 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117664 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.203628+0000", 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4020000000000003e-05 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9917 7.9 7:9ece3d7a:::benchmark_data_smithi079_83551_object9916:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.190159+0000", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8267616159999998, 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015927107999999999, 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9917 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190159+0000", 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190159+0000", 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190157+0000", 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190186+0000", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8977000000000001e-05 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190201+0000", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4399000000000001e-05 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190209+0000", 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4630000000000001e-06 2024-09-17T08:12:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190411+0000", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000201936 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.190529+0000", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117337 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.191207+0000", 2024-09-17T08:12:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067894299999999999 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.195528+0000", 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004320361 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.205866+0000", 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010337766 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.205999+0000", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132998 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.206048+0000", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9122999999999997e-05 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.206086+0000", 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8482000000000002e-05 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:9940 7.e e96/91 7:718a6188:::benchmark_data_smithi079_83551_object9939:head v 96'632, mlcod=96'629)", 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.213789+0000", 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8031316319999999, 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016295397999999999, 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213789+0000", 2024-09-17T08:12:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213789+0000", 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213788+0000", 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213820+0000", 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1610999999999999e-05 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213832+0000", 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2167000000000001e-05 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213838+0000", 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5250000000000002e-06 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214709+0000", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087062299999999995 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214724+0000", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5257e-05 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230058+0000", 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015333632 2024-09-17T08:12:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230084+0000", 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6568e-05 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9950 7.9 7:93364d15:::benchmark_data_smithi079_83551_object9949:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.213907+0000", 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8030133589999999, 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016258291000000001, 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9950 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213907+0000", 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213907+0000", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213906+0000", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213931+0000", 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5479000000000001e-05 2024-09-17T08:12:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213940+0000", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9469999999999992e-06 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213946+0000", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9519999999999999e-06 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.216257+0000", 2024-09-17T08:12:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023110729999999999 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.216388+0000", 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000130932 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.216527+0000", 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013864699999999999 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.220081+0000", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035540670000000002 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223706+0000", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036248780000000002 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230070+0000", 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063637570000000003 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230119+0000", 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9193999999999997e-05 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230166+0000", 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6771999999999999e-05 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9953 7.9 7:98bcbe4f:::benchmark_data_smithi079_83551_object9952:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.213951+0000", 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8029696790000003, 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016287553, 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9953 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213951+0000", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213951+0000", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213951+0000", 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213970+0000", 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9083e-05 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213976+0000", 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8749999999999997e-06 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213980+0000", 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.217179+0000", 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031994329999999998 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.217406+0000", 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022710300000000001 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.217540+0000", 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013388300000000001 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.220112+0000", 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717320000000002 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223738+0000", 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036258660000000002 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230174+0000", 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064357099999999999 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230210+0000", 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5976000000000002e-05 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230239+0000", 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9068999999999999e-05 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:9949 7.d e96/91 7:b273f7db:::benchmark_data_smithi079_83551_object9948:head v 96'614, mlcod=96'612)", 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.213993+0000", 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8029280449999998, 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016008272, 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213993+0000", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213993+0000", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.213992+0000", 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214030+0000", 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8022000000000003e-05 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214037+0000", 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2600000000000002e-06 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214041+0000", 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0589999999999996e-06 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214114+0000", 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2885000000000004e-05 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214132+0000", 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8161000000000001e-05 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.229974+0000", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015842449000000002 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230001+0000", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6781999999999999e-05 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:9952 7.5 e96/91 7:afa86358:::benchmark_data_smithi079_83551_object9951:head v 96'641, mlcod=96'639)", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.214500+0000", 2024-09-17T08:12:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8024210529999998, 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015522391999999999, 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214500+0000", 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214500+0000", 2024-09-17T08:12:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214499+0000", 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214531+0000", 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1831000000000001e-05 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214561+0000", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0899999999999999e-05 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.214569+0000", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4050000000000003e-06 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.217985+0000", 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034165620000000002 2024-09-17T08:12:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.218005+0000", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199e-05 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230005+0000", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012000008 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.230022+0000", 2024-09-17T08:12:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7433000000000001e-05 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9963 7.4 7:22306f6f:::benchmark_data_smithi079_83551_object9962:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.223208+0000", 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7937129199999999, 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017369025999999999, 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9963 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223208+0000", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223208+0000", 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223207+0000", 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223267+0000", 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9726999999999999e-05 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223274+0000", 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4450000000000001e-06 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223278+0000", 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5260000000000004e-06 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223322+0000", 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3492999999999998e-05 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223432+0000", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110933 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.223580+0000", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001473 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.233714+0000", 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010134386 2024-09-17T08:12:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.233807+0000", 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2509000000000005e-05 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.240488+0000", 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066817580000000003 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.240538+0000", 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0071000000000002e-05 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.240577+0000", 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8336e-05 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9984 7.f 7:f6ad8bdf:::benchmark_data_smithi079_83551_object9983:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.243808+0000", 2024-09-17T08:12:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.773112276, 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017348881999999999, 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9984 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243808+0000", 2024-09-17T08:12:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243808+0000", 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243807+0000", 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243836+0000", 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9306999999999999e-05 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243851+0000", 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4749999999999999e-05 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243858+0000", 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1890000000000001e-06 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.245300+0000", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001442189 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.245402+0000", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101826 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.245525+0000", 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012320000000000001 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.247028+0000", 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015022830000000001 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.247772+0000", 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074498100000000003 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261061+0000", 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013288392 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261112+0000", 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1205000000000002e-05 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261157+0000", 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5223999999999999e-05 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9985 7.4 7:20cd8e06:::benchmark_data_smithi079_83551_object9984:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.243868+0000", 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7730529640000001, 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016381586, 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9985 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243868+0000", 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243868+0000", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243867+0000", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243893+0000", 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.533e-05 2024-09-17T08:12:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243900+0000", 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5689999999999999e-06 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243905+0000", 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1159999999999998e-06 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.244093+0000", 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000187839 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.244200+0000", 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107299 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.244333+0000", 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013285400000000001 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.246297+0000", 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019641799999999998 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.247761+0000", 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00146361 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260157+0000", 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012396038999999999 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260203+0000", 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6328000000000001e-05 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260249+0000", 2024-09-17T08:12:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5998999999999998e-05 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9986 7.9 7:9582af29:::benchmark_data_smithi079_83551_object9985:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.243919+0000", 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7730016969999998, 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016251788999999999, 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9986 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243919+0000", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243919+0000", 2024-09-17T08:12:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243918+0000", 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243944+0000", 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-05 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243951+0000", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4109999999999999e-06 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.243956+0000", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.246028+0000", 2024-09-17T08:12:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002071275 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.246148+0000", 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012078000000000001 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.246299+0000", 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015085700000000001 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.247293+0000", 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099379500000000001 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.247735+0000", 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044187499999999999 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260090+0000", 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012354703999999999 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260135+0000", 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5107e-05 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260171+0000", 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6186999999999998e-05 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.140 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 6 15 17411 17396 181.16 158.812 0.00517352 0.00551428 2024-09-17T08:12:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9989 7.4 7:2cbcd6c9:::benchmark_data_smithi079_83551_object9988:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.249272+0000", 2024-09-17T08:12:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7676482890000003, 2024-09-17T08:12:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015292274999999999, 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9989 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249272+0000", 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249272+0000", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249270+0000", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249313+0000", 2024-09-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391000000000001e-05 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249328+0000", 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5031000000000001e-05 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249336+0000", 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8059999999999995e-06 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.252174+0000", 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028383739999999998 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.252292+0000", 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118293 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.252426+0000", 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134058 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.257007+0000", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045810499999999997 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.260290+0000", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032821289999999999 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264487+0000", 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041973540000000004 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264530+0000", 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.333e-05 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264565+0000", 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4483000000000002e-05 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9990 7.f 7:fb5589dd:::benchmark_data_smithi079_83551_object9989:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.249349+0000", 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7675715190000001, 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023111357999999999, 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9990 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249349+0000", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249349+0000", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249349+0000", 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249373+0000", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4646000000000001e-05 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249381+0000", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4490000000000002e-06 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249615+0000", 2024-09-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023418500000000001 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261444+0000", 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011829226999999999 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261555+0000", 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110535 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.261725+0000", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017056100000000001 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264261+0000", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002535748 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264267+0000", 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3659999999999997e-06 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272373+0000", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081051030000000007 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272424+0000", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1208999999999997e-05 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272461+0000", 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6829000000000003e-05 2024-09-17T08:12:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:9994 7.f 7:f82fd3e8:::benchmark_data_smithi079_83551_object9993:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.249686+0000", 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7672345580000002, 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022875348, 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9994 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249686+0000", 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249686+0000", 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249686+0000", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249708+0000", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2385999999999999e-05 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249715+0000", 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4189999999999996e-06 2024-09-17T08:12:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.249721+0000", 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2399999999999998e-06 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.262249+0000", 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012528035999999999 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.262349+0000", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9971999999999998e-05 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.262469+0000", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011998999999999999 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.264272+0000", 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018029179999999999 2024-09-17T08:12:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.265984+0000", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017121720000000001 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272496+0000", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065117760000000004 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272531+0000", 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5725000000000001e-05 2024-09-17T08:12:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.272562+0000", 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0273999999999998e-05 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:11695 7.f 7:f92ca32e:::benchmark_data_smithi079_83551_object11694:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:21.941415+0000", 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.075505884, 2024-09-17T08:12:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015485519999999999, 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11695 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941415+0000", 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941415+0000", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941414+0000", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941438+0000", 2024-09-17T08:12:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4023e-05 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941449+0000", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1141000000000001e-05 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.941455+0000", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5659999999999997e-06 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.946615+0000", 2024-09-17T08:12:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051595709999999999 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.947226+0000", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061101000000000005 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.948271+0000", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001045795 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.949602+0000", 2024-09-17T08:12:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013304020000000001 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.949805+0000", 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203618 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.956821+0000", 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070156690000000004 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.956867+0000", 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6377e-05 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:21.956900+0000", 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2867999999999998e-05 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:15161 7.f 7:fd6caf34:::benchmark_data_smithi079_83551_object15160:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:12:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:23.170451+0000", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.84646975899999999, 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015087229000000001, 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15161 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170451+0000", 2024-09-17T08:12:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170451+0000", 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170450+0000", 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170552+0000", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010194100000000001 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170574+0000", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1455999999999999e-05 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.170581+0000", 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.849e-06 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.178127+0000", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075467520000000003 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.178239+0000", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111307 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.178355+0000", 2024-09-17T08:12:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011642300000000001 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.180168+0000", 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001813376 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.180549+0000", 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000380942 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.185421+0000", 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048714450000000003 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.185464+0000", 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2861000000000002e-05 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.185538+0000", 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4472000000000001e-05 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:15962 7.9 7:9e6fcff5:::benchmark_data_smithi079_83551_object15961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:23.478953+0000", 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.53796817299999999, 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016149356, 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:12:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15962 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.478953+0000", 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.478953+0000", 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.478951+0000", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.479041+0000", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0171000000000001e-05 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.479132+0000", 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0649999999999994e-05 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.479139+0000", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6299999999999998e-06 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.479418+0000", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027903699999999999 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.480482+0000", 2024-09-17T08:12:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010633260000000001 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.480580+0000", 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7991000000000004e-05 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.484241+0000", 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003661498 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.486097+0000", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001855806 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.495027+0000", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089299900000000005 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.495068+0000", 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467999999999997e-05 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:12:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:23.495102+0000", 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3461999999999999e-05 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:12:24.178 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:12:24.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.180 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:24.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:24.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:24.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:24.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:24.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:24 smithi079 ceph-mon[29565]: pgmap v81: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:12:24.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:24 smithi079 ceph-mon[40649]: pgmap v81: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:12:24.412 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:24.441 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:24.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:24.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:24.582 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:24.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.583 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:24.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:24.718 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:24.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:24 smithi173 ceph-mon[33187]: pgmap v81: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:12:24.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:24.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:24.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:24.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:24.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:25.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:25.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:25.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 7 16 19966 19950 178.08 159.625 0.00431353 0.0056097 2024-09-17T08:12:25.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:25.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:25.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:25.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:25.282 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:25.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:25.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:25.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:25.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:25.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:25.547 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:25.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:25.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:25.690 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:25.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:25.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:25.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:25.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:25.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:25.967 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:25.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:25.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:26.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:26.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 8 16 22449 22433 175.217 155.188 0.00184506 0.00569685 2024-09-17T08:12:26.090 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:26.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:26.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:26.228 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:26.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:26.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:26.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:26.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:26.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:26.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:26.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:26.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: pgmap v82: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:12:26.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: pgmap v82: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:12:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:26.623 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:26.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: pgmap v82: 148 pgs: 148 active+clean; 776 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:12:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:12:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:12:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:26.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:26.763 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:26.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:26.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:26.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:26.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:26.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:26.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:27.027 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:27.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:27.072 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 9 16 24781 24765 171.942 145.75 0.00520728 0.00581002 2024-09-17T08:12:27.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:27.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:12:27.181 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:27.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:27.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:27.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:27.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:27.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:27.440 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:27.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:27.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:27.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.587 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:27.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:27.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:27.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:27.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:27.779 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:27.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:27.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:27.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:27.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:28.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:28.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.033 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:28.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:28.039 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:12:28.042 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:12:28.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 10 16 26689 26673 166.666 119.25 0.00889673 0.00599398 2024-09-17T08:12:28.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:28.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:28.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:28.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:28.289 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:12:28.321 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:28.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:28.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:28.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:28.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:28.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:28.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:28.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[29565]: pgmap v83: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:12:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2681365430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[40649]: pgmap v83: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:12:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2681365430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:28 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:28.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:28.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:28 smithi173 ceph-mon[33187]: pgmap v83: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:12:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:28 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2681365430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:28 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:12:28.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:28.847 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:28.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:28.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:28.983 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:28.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:28.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:29.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 11 16 28819 28803 163.616 133.125 0.00384601 0.00610616 2024-09-17T08:12:29.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:29.117 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:29.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:29.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:29.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:29.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:29.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:29.398 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:29.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:12:29.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:29 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:12:29.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:29.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.536 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:29.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:12:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:12:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T08:12:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:12:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:12:29.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:12:29.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:12:29.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:12:29.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:29.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:12:29.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T08:12:29.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:12:29.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:12:29.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:12:29.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:29 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:12:29.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:29.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:29.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:29.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:29.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:29.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:29.913 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:29.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:29.945 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:29.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:30.046 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:30.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:30.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:30.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 12 16 30840 30824 160.507 126.312 0.00182667 0.00620375 2024-09-17T08:12:30.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:30.196 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:30.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:30.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:30.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:30.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:30.466 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:30.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:30.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:30.598 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:30.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:30.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[29565]: pgmap v85: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-17T08:12:30.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:12:30.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:12:30.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[40649]: pgmap v85: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-17T08:12:30.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:12:30.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:30 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:12:30.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:30 smithi173 ceph-mon[33187]: pgmap v85: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 196 MiB/s wr, 3.14k op/s 2024-09-17T08:12:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:30 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:12:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:30 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:12:30.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:30.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:30.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:30.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:30.883 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:30.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:30.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:31.005 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:31.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 13 16 32705 32689 157.126 116.562 0.0040272 0.00635871 2024-09-17T08:12:31.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:31.134 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:31.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:31.265 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:31.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:31.394 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:31.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:31.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:31.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:31.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:31.727 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:31.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:31.794 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:31.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:31.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:31.937 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:31.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:31.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:32.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:32.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:32.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:32.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 14 16 34780 34764 155.166 129.688 0.00525521 0.00643922 2024-09-17T08:12:32.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:32.199 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:32.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:32.301 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:32.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:32.332 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:32.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:32.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:32.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:32.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: pgmap v86: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: pgmap v86: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:12:32.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:32.551 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:32.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:32.605 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:32.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:32.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:32.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: pgmap v86: 148 pgs: 148 active+clean; 1.5 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:32.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:32.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:32.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:32.886 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:32.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:32.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:32.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:33.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:33.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:33.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 15 16 36807 36791 153.267 126.688 0.0124511 0.00651947 2024-09-17T08:12:33.128 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:33.158 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:33.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:33.259 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-17T08:12:27.236930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1646401814,"num_objects":25309,"num_object_clones":0,"num_object_copies":75927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25309,"num_whiteouts":0,"num_read":1066,"num_read_kb":912,"num_write":25722,"num_write_kb":1609691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25738,"ondisk_log_size":25738,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4863220,"kb_used_data":4581556,"kb_used_omap":12,"kb_used_meta":281587,"kb_avail":745097996,"statfs":{"total":767960285184,"available":762980347904,"internally_reserved":0,"allocated":4691513344,"data_stored":4688200538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":288345656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,6,11,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1645805568,"num_objects":25113,"num_object_clones":0,"num_object_copies":75339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25113,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":25113,"num_write_kb":1607232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.881728"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359763+0000","last_change":"2024-09-17T08:11:34.012978+0000","last_active":"2024-09-17T08:12:17.359763+0000","last_peered":"2024-09-17T08:12:17.359763+0000","last_clean":"2024-09-17T08:12:17.359763+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:12:17.359763+0000","last_undegraded":"2024-09-17T08:12:17.359763+0000","last_fullsized":"2024-09-17T08:12:17.359763+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.012877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.012877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.356338+0000","scrub_duration":0.001297653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.078544+0000","last_change":"2024-09-17T08:10:56.845970+0000","last_active":"2024-09-17T08:12:18.078544+0000","last_peered":"2024-09-17T08:12:18.078544+0000","last_clean":"2024-09-17T08:12:18.078544+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:12:18.078544+0000","last_undegraded":"2024-09-17T08:12:18.078544+0000","last_fullsized":"2024-09-17T08:12:18.078544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.845877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.845877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014986139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":77,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.252355+0000","last_change":"2024-09-17T08:12:03.141440+0000","last_active":"2024-09-17T08:12:21.252355+0000","last_peered":"2024-09-17T08:12:21.252355+0000","last_clean":"2024-09-17T08:12:21.252355+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:12:21.252355+0000","last_undegraded":"2024-09-17T08:12:21.252355+0000","last_fullsized":"2024-09-17T08:12:21.252355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:12:03.141349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:03.141349+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:21.954154+0000","scrub_duration":0.0026318309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.241724+0000","last_change":"2024-09-17T08:11:13.631177+0000","last_active":"2024-09-17T08:12:21.241724+0000","last_peered":"2024-09-17T08:12:21.241724+0000","last_clean":"2024-09-17T08:12:21.241724+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:12:21.241724+0000","last_undegraded":"2024-09-17T08:12:21.241724+0000","last_fullsized":"2024-09-17T08:12:21.241724+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.631074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.631074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:37.133314+0000","scrub_duration":0.001610493,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359377+0000","last_change":"2024-09-17T08:09:54.500985+0000","last_active":"2024-09-17T08:12:17.359377+0000","last_peered":"2024-09-17T08:12:17.359377+0000","last_clean":"2024-09-17T08:12:17.359377+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:12:17.359377+0000","last_undegraded":"2024-09-17T08:12:17.359377+0000","last_fullsized":"2024-09-17T08:12:17.359377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:09:53.481832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.241804+0000","last_change":"2024-09-17T08:11:33.629288+0000","last_active":"2024-09-17T08:12:21.241804+0000","last_peered":"2024-09-17T08:12:21.241804+0000","last_clean":"2024-09-17T08:12:21.241804+0000","last_became_active":"2024-09-17T08:11:32.433797+0000","last_became_peered":"2024-09-17T08:11:32.433797+0000","last_unstale":"2024-09-17T08:12:21.241804+0000","last_undegraded":"2024-09-17T08:12:21.241804+0000","last_fullsized":"2024-09-17T08:12:21.241804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.629197+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.320003+0000","scrub_duration":0.003700662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.249912+0000","last_change":"2024-09-17T08:11:34.913426+0000","last_active":"2024-09-17T08:12:21.249912+0000","last_peered":"2024-09-17T08:12:21.249912+0000","last_clean":"2024-09-17T08:12:21.249912+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:12:21.249912+0000","last_undegraded":"2024-09-17T08:12:21.249912+0000","last_fullsized":"2024-09-17T08:12:21.249912+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.913264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.913264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.562937+0000","scrub_duration":0.0017904069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.241797+0000","last_change":"2024-09-17T08:11:04.598021+0000","last_active":"2024-09-17T08:12:21.241797+0000","last_peered":"2024-09-17T08:12:21.241797+0000","last_clean":"2024-09-17T08:12:21.241797+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:12:21.241797+0000","last_undegraded":"2024-09-17T08:12:21.241797+0000","last_fullsized":"2024-09-17T08:12:21.241797+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.597920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.597920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013934189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255079+0000","last_change":"2024-09-17T08:11:22.381594+0000","last_active":"2024-09-17T08:12:21.255079+0000","last_peered":"2024-09-17T08:12:21.255079+0000","last_clean":"2024-09-17T08:12:21.255079+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:12:21.255079+0000","last_undegraded":"2024-09-17T08:12:21.255079+0000","last_fullsized":"2024-09-17T08:12:21.255079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:22.381461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.381461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.013237+0000","scrub_duration":0.0017582229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.241930+0000","last_change":"2024-09-17T08:11:29.578149+0000","last_active":"2024-09-17T08:12:21.241930+0000","last_peered":"2024-09-17T08:12:21.241930+0000","last_clean":"2024-09-17T08:12:21.241930+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:12:21.241930+0000","last_undegraded":"2024-09-17T08:12:21.241930+0000","last_fullsized":"2024-09-17T08:12:21.241930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.578069+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.180514+0000","scrub_duration":0.0025270700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255800+0000","last_change":"2024-09-17T08:11:02.425380+0000","last_active":"2024-09-17T08:12:21.255800+0000","last_peered":"2024-09-17T08:12:21.255800+0000","last_clean":"2024-09-17T08:12:21.255800+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:12:21.255800+0000","last_undegraded":"2024-09-17T08:12:21.255800+0000","last_fullsized":"2024-09-17T08:12:21.255800+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:02.425247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.425247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015898959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075353+0000","last_change":"2024-09-17T08:11:31.086448+0000","last_active":"2024-09-17T08:12:18.075353+0000","last_peered":"2024-09-17T08:12:18.075353+0000","last_clean":"2024-09-17T08:12:18.075353+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:12:18.075353+0000","last_undegraded":"2024-09-17T08:12:18.075353+0000","last_fullsized":"2024-09-17T08:12:18.075353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.086346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.086346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.290177+0000","scrub_duration":0.0013806619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255006+0000","last_change":"2024-09-17T08:11:28.306565+0000","last_active":"2024-09-17T08:12:21.255006+0000","last_peered":"2024-09-17T08:12:21.255006+0000","last_clean":"2024-09-17T08:12:21.255006+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:12:21.255006+0000","last_undegraded":"2024-09-17T08:12:21.255006+0000","last_fullsized":"2024-09-17T08:12:21.255006+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.306480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.306480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:53.967249+0000","scrub_duration":0.001181517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.242125+0000","last_change":"2024-09-17T08:11:03.630715+0000","last_active":"2024-09-17T08:12:21.242125+0000","last_peered":"2024-09-17T08:12:21.242125+0000","last_clean":"2024-09-17T08:12:21.242125+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:12:21.242125+0000","last_undegraded":"2024-09-17T08:12:21.242125+0000","last_fullsized":"2024-09-17T08:12:21.242125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:03.630581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.630581+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021641149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255018+0000","last_change":"2024-09-17T08:11:18.175153+0000","last_active":"2024-09-17T08:12:21.255018+0000","last_peered":"2024-09-17T08:12:21.255018+0000","last_clean":"2024-09-17T08:12:21.255018+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:12:21.255018+0000","last_undegraded":"2024-09-17T08:12:21.255018+0000","last_fullsized":"2024-09-17T08:12:21.255018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.175071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.175071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.354413+0000","scrub_duration":0.001408121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254393+0000","last_change":"2024-09-17T08:11:21.041377+0000","last_active":"2024-09-17T08:12:21.254393+0000","last_peered":"2024-09-17T08:12:21.254393+0000","last_clean":"2024-09-17T08:12:21.254393+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:12:21.254393+0000","last_undegraded":"2024-09-17T08:12:21.254393+0000","last_fullsized":"2024-09-17T08:12:21.254393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.041297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.041297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.672583+0000","scrub_duration":0.0013107559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"97'1575","reported_seq":1588,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.219135+0000","last_change":"2024-09-17T08:12:05.207753+0000","last_active":"2024-09-17T08:12:27.219135+0000","last_peered":"2024-09-17T08:12:27.219135+0000","last_clean":"2024-09-17T08:12:27.219135+0000","last_became_active":"2024-09-17T08:12:05.207481+0000","last_became_peered":"2024-09-17T08:12:05.207481+0000","last_unstale":"2024-09-17T08:12:27.219135+0000","last_undegraded":"2024-09-17T08:12:27.219135+0000","last_fullsized":"2024-09-17T08:12:27.219135+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1575,"log_dups_size":0,"ondisk_log_size":1575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:14.488199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103219200,"num_objects":1575,"num_object_clones":0,"num_object_copies":4725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1575,"num_write_kb":100800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":80,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355421+0000","last_change":"2024-09-17T08:11:02.621987+0000","last_active":"2024-09-17T08:12:17.355421+0000","last_peered":"2024-09-17T08:12:17.355421+0000","last_clean":"2024-09-17T08:12:17.355421+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:12:17.355421+0000","last_undegraded":"2024-09-17T08:12:17.355421+0000","last_fullsized":"2024-09-17T08:12:17.355421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:11:02.621854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.621854+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.526081+0000","scrub_duration":0.0024197989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077134+0000","last_change":"2024-09-17T08:11:18.510464+0000","last_active":"2024-09-17T08:12:18.077134+0000","last_peered":"2024-09-17T08:12:18.077134+0000","last_clean":"2024-09-17T08:12:18.077134+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:12:18.077134+0000","last_undegraded":"2024-09-17T08:12:18.077134+0000","last_fullsized":"2024-09-17T08:12:18.077134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.510308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.510308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.534650+0000","scrub_duration":0.0016130199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358934+0000","last_change":"2024-09-17T08:11:31.017724+0000","last_active":"2024-09-17T08:12:17.358934+0000","last_peered":"2024-09-17T08:12:17.358934+0000","last_clean":"2024-09-17T08:12:17.358934+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:12:17.358934+0000","last_undegraded":"2024-09-17T08:12:17.358934+0000","last_fullsized":"2024-09-17T08:12:17.358934+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.017640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.017640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.994183+0000","scrub_duration":0.0020059460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.252711+0000","last_change":"2024-09-17T08:11:28.097377+0000","last_active":"2024-09-17T08:12:21.252711+0000","last_peered":"2024-09-17T08:12:21.252711+0000","last_clean":"2024-09-17T08:12:21.252711+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:12:21.252711+0000","last_undegraded":"2024-09-17T08:12:21.252711+0000","last_fullsized":"2024-09-17T08:12:21.252711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.097289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.097289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.706372+0000","scrub_duration":0.001520696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"97'1537","reported_seq":1552,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.225863+0000","last_change":"2024-09-17T08:12:05.208204+0000","last_active":"2024-09-17T08:12:27.225863+0000","last_peered":"2024-09-17T08:12:27.225863+0000","last_clean":"2024-09-17T08:12:27.225863+0000","last_became_active":"2024-09-17T08:12:05.207933+0000","last_became_peered":"2024-09-17T08:12:05.207933+0000","last_unstale":"2024-09-17T08:12:27.225863+0000","last_undegraded":"2024-09-17T08:12:27.225863+0000","last_fullsized":"2024-09-17T08:12:27.225863+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1537,"log_dups_size":0,"ondisk_log_size":1537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.365781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100728832,"num_objects":1537,"num_object_clones":0,"num_object_copies":4611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1537,"num_write_kb":98368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355389+0000","last_change":"2024-09-17T08:11:01.605025+0000","last_active":"2024-09-17T08:12:17.355389+0000","last_peered":"2024-09-17T08:12:17.355389+0000","last_clean":"2024-09-17T08:12:17.355389+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:12:17.355389+0000","last_undegraded":"2024-09-17T08:12:17.355389+0000","last_fullsized":"2024-09-17T08:12:17.355389+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:01.604844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.604844+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026767549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358919+0000","last_change":"2024-09-17T08:10:59.270433+0000","last_active":"2024-09-17T08:12:17.358919+0000","last_peered":"2024-09-17T08:12:17.358919+0000","last_clean":"2024-09-17T08:12:17.358919+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:12:17.358919+0000","last_undegraded":"2024-09-17T08:12:17.358919+0000","last_fullsized":"2024-09-17T08:12:17.358919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:59.270354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.270354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.047730+0000","scrub_duration":0.001390388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":89,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.253025+0000","last_change":"2024-09-17T08:11:18.109889+0000","last_active":"2024-09-17T08:12:21.253025+0000","last_peered":"2024-09-17T08:12:21.253025+0000","last_clean":"2024-09-17T08:12:21.253025+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:12:21.253025+0000","last_undegraded":"2024-09-17T08:12:21.253025+0000","last_fullsized":"2024-09-17T08:12:21.253025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.109796+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024437130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355427+0000","last_change":"2024-09-17T08:11:48.545776+0000","last_active":"2024-09-17T08:12:17.355427+0000","last_peered":"2024-09-17T08:12:17.355427+0000","last_clean":"2024-09-17T08:12:17.355427+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:12:17.355427+0000","last_undegraded":"2024-09-17T08:12:17.355427+0000","last_fullsized":"2024-09-17T08:12:17.355427+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:11:48.545686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:15.234576+0000","scrub_duration":0.0015559700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"97'1629","reported_seq":1644,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.206022+0000","last_change":"2024-09-17T08:12:05.206659+0000","last_active":"2024-09-17T08:12:27.206022+0000","last_peered":"2024-09-17T08:12:27.206022+0000","last_clean":"2024-09-17T08:12:27.206022+0000","last_became_active":"2024-09-17T08:12:05.206362+0000","last_became_peered":"2024-09-17T08:12:05.206362+0000","last_unstale":"2024-09-17T08:12:27.206022+0000","last_undegraded":"2024-09-17T08:12:27.206022+0000","last_fullsized":"2024-09-17T08:12:27.206022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:27.687890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106758144,"num_objects":1629,"num_object_clones":0,"num_object_copies":4887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1629,"num_write_kb":104256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"72'15","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359041+0000","last_change":"2024-09-17T08:11:16.970059+0000","last_active":"2024-09-17T08:12:17.359041+0000","last_peered":"2024-09-17T08:12:17.359041+0000","last_clean":"2024-09-17T08:12:17.359041+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:12:17.359041+0000","last_undegraded":"2024-09-17T08:12:17.359041+0000","last_fullsized":"2024-09-17T08:12:17.359041+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:16.969967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.969967+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.592697+0000","scrub_duration":0.0022928860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359258+0000","last_change":"2024-09-17T08:11:43.931670+0000","last_active":"2024-09-17T08:12:17.359258+0000","last_peered":"2024-09-17T08:12:17.359258+0000","last_clean":"2024-09-17T08:12:17.359258+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:12:17.359258+0000","last_undegraded":"2024-09-17T08:12:17.359258+0000","last_fullsized":"2024-09-17T08:12:17.359258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:43.931558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.931558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:00.466096+0000","scrub_duration":0.0017476449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":88,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.242207+0000","last_change":"2024-09-17T08:11:18.627103+0000","last_active":"2024-09-17T08:12:21.242207+0000","last_peered":"2024-09-17T08:12:21.242207+0000","last_clean":"2024-09-17T08:12:21.242207+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:12:21.242207+0000","last_undegraded":"2024-09-17T08:12:21.242207+0000","last_fullsized":"2024-09-17T08:12:21.242207+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:18.626957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.626957+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:41.187018+0000","scrub_duration":0.0016538869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077237+0000","last_change":"2024-09-17T08:11:27.126060+0000","last_active":"2024-09-17T08:12:18.077237+0000","last_peered":"2024-09-17T08:12:18.077237+0000","last_clean":"2024-09-17T08:12:18.077237+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:12:18.077237+0000","last_undegraded":"2024-09-17T08:12:18.077237+0000","last_fullsized":"2024-09-17T08:12:18.077237+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.125964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.125964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:42.326040+0000","scrub_duration":0.0015390530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"97'1553","reported_seq":1568,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.208475+0000","last_change":"2024-09-17T08:12:05.207037+0000","last_active":"2024-09-17T08:12:27.208475+0000","last_peered":"2024-09-17T08:12:27.208475+0000","last_clean":"2024-09-17T08:12:27.208475+0000","last_became_active":"2024-09-17T08:12:05.206426+0000","last_became_peered":"2024-09-17T08:12:05.206426+0000","last_unstale":"2024-09-17T08:12:27.208475+0000","last_undegraded":"2024-09-17T08:12:27.208475+0000","last_fullsized":"2024-09-17T08:12:27.208475+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1553,"log_dups_size":0,"ondisk_log_size":1553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:06.304741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101777408,"num_objects":1553,"num_object_clones":0,"num_object_copies":4659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1553,"num_write_kb":99392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":71,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355549+0000","last_change":"2024-09-17T08:11:25.618404+0000","last_active":"2024-09-17T08:12:17.355549+0000","last_peered":"2024-09-17T08:12:17.355549+0000","last_clean":"2024-09-17T08:12:17.355549+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:12:17.355549+0000","last_undegraded":"2024-09-17T08:12:17.355549+0000","last_fullsized":"2024-09-17T08:12:17.355549+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:25.618290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.618290+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.854332+0000","scrub_duration":0.0022499999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074243+0000","last_change":"2024-09-17T08:11:31.305742+0000","last_active":"2024-09-17T08:12:18.074243+0000","last_peered":"2024-09-17T08:12:18.074243+0000","last_clean":"2024-09-17T08:12:18.074243+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:12:18.074243+0000","last_undegraded":"2024-09-17T08:12:18.074243+0000","last_fullsized":"2024-09-17T08:12:18.074243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.305634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.305634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.476701+0000","scrub_duration":0.001519353,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077669+0000","last_change":"2024-09-17T08:11:00.199362+0000","last_active":"2024-09-17T08:12:18.077669+0000","last_peered":"2024-09-17T08:12:18.077669+0000","last_clean":"2024-09-17T08:12:18.077669+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:12:18.077669+0000","last_undegraded":"2024-09-17T08:12:18.077669+0000","last_fullsized":"2024-09-17T08:12:18.077669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.199236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.199236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.229249+0000","scrub_duration":0.0014479130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359899+0000","last_change":"2024-09-17T08:12:00.142321+0000","last_active":"2024-09-17T08:12:17.359899+0000","last_peered":"2024-09-17T08:12:17.359899+0000","last_clean":"2024-09-17T08:12:17.359899+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:12:17.359899+0000","last_undegraded":"2024-09-17T08:12:17.359899+0000","last_fullsized":"2024-09-17T08:12:17.359899+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:00.142246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:00.142246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:17.245246+0000","scrub_duration":0.001285509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"97'1569","reported_seq":1584,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.226173+0000","last_change":"2024-09-17T08:12:05.209102+0000","last_active":"2024-09-17T08:12:27.226173+0000","last_peered":"2024-09-17T08:12:27.226173+0000","last_clean":"2024-09-17T08:12:27.226173+0000","last_became_active":"2024-09-17T08:12:05.208839+0000","last_became_peered":"2024-09-17T08:12:05.208839+0000","last_unstale":"2024-09-17T08:12:27.226173+0000","last_undegraded":"2024-09-17T08:12:27.226173+0000","last_fullsized":"2024-09-17T08:12:27.226173+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1569,"log_dups_size":0,"ondisk_log_size":1569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:11.452019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102825984,"num_objects":1569,"num_object_clones":0,"num_object_copies":4707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1569,"num_write_kb":100416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":73,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.354870+0000","last_change":"2024-09-17T08:11:07.570044+0000","last_active":"2024-09-17T08:12:17.354870+0000","last_peered":"2024-09-17T08:12:17.354870+0000","last_clean":"2024-09-17T08:12:17.354870+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:12:17.354870+0000","last_undegraded":"2024-09-17T08:12:17.354870+0000","last_fullsized":"2024-09-17T08:12:17.354870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:07.569902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:07.569902+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.667901+0000","scrub_duration":0.0023659520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077109+0000","last_change":"2024-09-17T08:11:06.246988+0000","last_active":"2024-09-17T08:12:18.077109+0000","last_peered":"2024-09-17T08:12:18.077109+0000","last_clean":"2024-09-17T08:12:18.077109+0000","last_became_active":"2024-09-17T08:09:54.506249+0000","last_became_peered":"2024-09-17T08:09:54.506249+0000","last_unstale":"2024-09-17T08:12:18.077109+0000","last_undegraded":"2024-09-17T08:12:18.077109+0000","last_fullsized":"2024-09-17T08:12:18.077109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.246870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.246870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016541940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.250441+0000","last_change":"2024-09-17T08:11:11.035892+0000","last_active":"2024-09-17T08:12:21.250441+0000","last_peered":"2024-09-17T08:12:21.250441+0000","last_clean":"2024-09-17T08:12:21.250441+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:12:21.250441+0000","last_undegraded":"2024-09-17T08:12:21.250441+0000","last_fullsized":"2024-09-17T08:12:21.250441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.035808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.035808+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.708988+0000","scrub_duration":0.001617893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076857+0000","last_change":"2024-09-17T08:11:11.469761+0000","last_active":"2024-09-17T08:12:18.076857+0000","last_peered":"2024-09-17T08:12:18.076857+0000","last_clean":"2024-09-17T08:12:18.076857+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:12:18.076857+0000","last_undegraded":"2024-09-17T08:12:18.076857+0000","last_fullsized":"2024-09-17T08:12:18.076857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.469652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.469652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022773849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"97'1644","reported_seq":1659,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.221531+0000","last_change":"2024-09-17T08:12:05.210025+0000","last_active":"2024-09-17T08:12:27.221531+0000","last_peered":"2024-09-17T08:12:27.221531+0000","last_clean":"2024-09-17T08:12:27.221531+0000","last_became_active":"2024-09-17T08:12:05.209256+0000","last_became_peered":"2024-09-17T08:12:05.209256+0000","last_unstale":"2024-09-17T08:12:27.221531+0000","last_undegraded":"2024-09-17T08:12:27.221531+0000","last_fullsized":"2024-09-17T08:12:27.221531+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1644,"log_dups_size":0,"ondisk_log_size":1644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:16.797108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107741184,"num_objects":1644,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1644,"num_write_kb":105216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":84,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359313+0000","last_change":"2024-09-17T08:11:02.961177+0000","last_active":"2024-09-17T08:12:17.359313+0000","last_peered":"2024-09-17T08:12:17.359313+0000","last_clean":"2024-09-17T08:12:17.359313+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:12:17.359313+0000","last_undegraded":"2024-09-17T08:12:17.359313+0000","last_fullsized":"2024-09-17T08:12:17.359313+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:11:02.961046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:02.961046+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026256159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358943+0000","last_change":"2024-09-17T08:11:33.944624+0000","last_active":"2024-09-17T08:12:17.358943+0000","last_peered":"2024-09-17T08:12:17.358943+0000","last_clean":"2024-09-17T08:12:17.358943+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:12:17.358943+0000","last_undegraded":"2024-09-17T08:12:17.358943+0000","last_fullsized":"2024-09-17T08:12:17.358943+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:33.944526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:33.944526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:03.130795+0000","scrub_duration":0.00148507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.354782+0000","last_change":"2024-09-17T08:11:43.596598+0000","last_active":"2024-09-17T08:12:17.354782+0000","last_peered":"2024-09-17T08:12:17.354782+0000","last_clean":"2024-09-17T08:12:17.354782+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:12:17.354782+0000","last_undegraded":"2024-09-17T08:12:17.354782+0000","last_fullsized":"2024-09-17T08:12:17.354782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.596489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.205254+0000","scrub_duration":0.001427177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355157+0000","last_change":"2024-09-17T08:11:38.545547+0000","last_active":"2024-09-17T08:12:17.355157+0000","last_peered":"2024-09-17T08:12:17.355157+0000","last_clean":"2024-09-17T08:12:17.355157+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:12:17.355157+0000","last_undegraded":"2024-09-17T08:12:17.355157+0000","last_fullsized":"2024-09-17T08:12:17.355157+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:38.545458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:38.545458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.911421+0000","scrub_duration":0.0012391500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"97'1547","reported_seq":1562,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.196595+0000","last_change":"2024-09-17T08:12:05.208958+0000","last_active":"2024-09-17T08:12:27.196595+0000","last_peered":"2024-09-17T08:12:27.196595+0000","last_clean":"2024-09-17T08:12:27.196595+0000","last_became_active":"2024-09-17T08:12:05.208659+0000","last_became_peered":"2024-09-17T08:12:05.208659+0000","last_unstale":"2024-09-17T08:12:27.196595+0000","last_undegraded":"2024-09-17T08:12:27.196595+0000","last_fullsized":"2024-09-17T08:12:27.196595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1547,"log_dups_size":0,"ondisk_log_size":1547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:17.301824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101384192,"num_objects":1547,"num_object_clones":0,"num_object_copies":4641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":99008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":88,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077554+0000","last_change":"2024-09-17T08:11:01.785378+0000","last_active":"2024-09-17T08:12:18.077554+0000","last_peered":"2024-09-17T08:12:18.077554+0000","last_clean":"2024-09-17T08:12:18.077554+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:12:18.077554+0000","last_undegraded":"2024-09-17T08:12:18.077554+0000","last_fullsized":"2024-09-17T08:12:18.077554+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:01.785270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.785270+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029603519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077904+0000","last_change":"2024-09-17T08:10:59.355508+0000","last_active":"2024-09-17T08:12:18.077904+0000","last_peered":"2024-09-17T08:12:18.077904+0000","last_clean":"2024-09-17T08:12:18.077904+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:12:18.077904+0000","last_undegraded":"2024-09-17T08:12:18.077904+0000","last_fullsized":"2024-09-17T08:12:18.077904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:10:59.355400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:59.355400+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.781890+0000","scrub_duration":0.0018629499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076037+0000","last_change":"2024-09-17T08:11:28.433770+0000","last_active":"2024-09-17T08:12:18.076037+0000","last_peered":"2024-09-17T08:12:18.076037+0000","last_clean":"2024-09-17T08:12:18.076037+0000","last_became_active":"2024-09-17T08:09:58.505724+0000","last_became_peered":"2024-09-17T08:09:58.505724+0000","last_unstale":"2024-09-17T08:12:18.076037+0000","last_undegraded":"2024-09-17T08:12:18.076037+0000","last_fullsized":"2024-09-17T08:12:18.076037+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:28.433666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.433666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.684654+0000","scrub_duration":0.001195822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358333+0000","last_change":"2024-09-17T08:11:11.026108+0000","last_active":"2024-09-17T08:12:17.358333+0000","last_peered":"2024-09-17T08:12:17.358333+0000","last_clean":"2024-09-17T08:12:17.358333+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:12:17.358333+0000","last_undegraded":"2024-09-17T08:12:17.358333+0000","last_fullsized":"2024-09-17T08:12:17.358333+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:11.026028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.026028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.088790+0000","scrub_duration":0.0014135770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"97'1580","reported_seq":1595,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.215853+0000","last_change":"2024-09-17T08:12:05.206666+0000","last_active":"2024-09-17T08:12:27.215853+0000","last_peered":"2024-09-17T08:12:27.215853+0000","last_clean":"2024-09-17T08:12:27.215853+0000","last_became_active":"2024-09-17T08:12:05.206361+0000","last_became_peered":"2024-09-17T08:12:05.206361+0000","last_unstale":"2024-09-17T08:12:27.215853+0000","last_undegraded":"2024-09-17T08:12:27.215853+0000","last_fullsized":"2024-09-17T08:12:27.215853+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1580,"log_dups_size":0,"ondisk_log_size":1580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:07.913004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103546880,"num_objects":1580,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1580,"num_write_kb":101120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358801+0000","last_change":"2024-09-17T08:11:17.921084+0000","last_active":"2024-09-17T08:12:17.358801+0000","last_peered":"2024-09-17T08:12:17.358801+0000","last_clean":"2024-09-17T08:12:17.358801+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:12:17.358801+0000","last_undegraded":"2024-09-17T08:12:17.358801+0000","last_fullsized":"2024-09-17T08:12:17.358801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:17.920975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.920975+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.178667+0000","scrub_duration":0.0021654640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358885+0000","last_change":"2024-09-17T08:11:18.904435+0000","last_active":"2024-09-17T08:12:17.358885+0000","last_peered":"2024-09-17T08:12:17.358885+0000","last_clean":"2024-09-17T08:12:17.358885+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:12:17.358885+0000","last_undegraded":"2024-09-17T08:12:17.358885+0000","last_fullsized":"2024-09-17T08:12:17.358885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:18.904342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:18.904342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.402947+0000","scrub_duration":0.0012167429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":88,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.253529+0000","last_change":"2024-09-17T08:11:05.859064+0000","last_active":"2024-09-17T08:12:21.253529+0000","last_peered":"2024-09-17T08:12:21.253529+0000","last_clean":"2024-09-17T08:12:21.253529+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:12:21.253529+0000","last_undegraded":"2024-09-17T08:12:21.253529+0000","last_fullsized":"2024-09-17T08:12:21.253529+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:05.858983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.858983+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.191877+0000","scrub_duration":0.002432306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076265+0000","last_change":"2024-09-17T08:11:32.080416+0000","last_active":"2024-09-17T08:12:18.076265+0000","last_peered":"2024-09-17T08:12:18.076265+0000","last_clean":"2024-09-17T08:12:18.076265+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:12:18.076265+0000","last_undegraded":"2024-09-17T08:12:18.076265+0000","last_fullsized":"2024-09-17T08:12:18.076265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.080269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.080269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.073092+0000","scrub_duration":0.001335506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"97'1569","reported_seq":1584,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.207110+0000","last_change":"2024-09-17T08:12:05.206732+0000","last_active":"2024-09-17T08:12:27.207110+0000","last_peered":"2024-09-17T08:12:27.207110+0000","last_clean":"2024-09-17T08:12:27.207110+0000","last_became_active":"2024-09-17T08:12:05.206481+0000","last_became_peered":"2024-09-17T08:12:05.206481+0000","last_unstale":"2024-09-17T08:12:27.207110+0000","last_undegraded":"2024-09-17T08:12:27.207110+0000","last_fullsized":"2024-09-17T08:12:27.207110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1569,"log_dups_size":0,"ondisk_log_size":1569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:18.169492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102825984,"num_objects":1569,"num_object_clones":0,"num_object_copies":4707,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1569,"num_write_kb":100416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":130,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.253826+0000","last_change":"2024-09-17T08:11:10.072237+0000","last_active":"2024-09-17T08:12:21.253826+0000","last_peered":"2024-09-17T08:12:21.253826+0000","last_clean":"2024-09-17T08:12:21.253826+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:12:21.253826+0000","last_undegraded":"2024-09-17T08:12:21.253826+0000","last_fullsized":"2024-09-17T08:12:21.253826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:11:10.072126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.072126+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.614950+0000","scrub_duration":0.0030770509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075948+0000","last_change":"2024-09-17T08:12:05.967977+0000","last_active":"2024-09-17T08:12:18.075948+0000","last_peered":"2024-09-17T08:12:18.075948+0000","last_clean":"2024-09-17T08:12:18.075948+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:12:18.075948+0000","last_undegraded":"2024-09-17T08:12:18.075948+0000","last_fullsized":"2024-09-17T08:12:18.075948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:05.967899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:05.967899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:11.118936+0000","scrub_duration":0.001398954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076971+0000","last_change":"2024-09-17T08:11:04.845607+0000","last_active":"2024-09-17T08:12:18.076971+0000","last_peered":"2024-09-17T08:12:18.076971+0000","last_clean":"2024-09-17T08:12:18.076971+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:12:18.076971+0000","last_undegraded":"2024-09-17T08:12:18.076971+0000","last_fullsized":"2024-09-17T08:12:18.076971+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.845519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.845519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.671984+0000","scrub_duration":0.0017591849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359157+0000","last_change":"2024-09-17T08:11:10.030674+0000","last_active":"2024-09-17T08:12:17.359157+0000","last_peered":"2024-09-17T08:12:17.359157+0000","last_clean":"2024-09-17T08:12:17.359157+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:12:17.359157+0000","last_undegraded":"2024-09-17T08:12:17.359157+0000","last_fullsized":"2024-09-17T08:12:17.359157+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:10.030576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.030576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014864679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"97'1526","reported_seq":1541,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.201422+0000","last_change":"2024-09-17T08:12:05.206789+0000","last_active":"2024-09-17T08:12:27.201422+0000","last_peered":"2024-09-17T08:12:27.201422+0000","last_clean":"2024-09-17T08:12:27.201422+0000","last_became_active":"2024-09-17T08:12:05.206469+0000","last_became_peered":"2024-09-17T08:12:05.206469+0000","last_unstale":"2024-09-17T08:12:27.201422+0000","last_undegraded":"2024-09-17T08:12:27.201422+0000","last_fullsized":"2024-09-17T08:12:27.201422+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:19.624094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100007936,"num_objects":1526,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1526,"num_write_kb":97664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075316+0000","last_change":"2024-09-17T08:11:34.025771+0000","last_active":"2024-09-17T08:12:18.075316+0000","last_peered":"2024-09-17T08:12:18.075316+0000","last_clean":"2024-09-17T08:12:18.075316+0000","last_became_active":"2024-09-17T08:11:32.435377+0000","last_became_peered":"2024-09-17T08:11:32.435377+0000","last_unstale":"2024-09-17T08:12:18.075316+0000","last_undegraded":"2024-09-17T08:12:18.075316+0000","last_fullsized":"2024-09-17T08:12:18.075316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:11:34.025633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.025633+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:48.265538+0000","scrub_duration":0.0023641140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":207,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:25.270302+0000","last_change":"2024-09-17T08:11:12.988991+0000","last_active":"2024-09-17T08:12:25.270302+0000","last_peered":"2024-09-17T08:12:25.270302+0000","last_clean":"2024-09-17T08:12:25.270302+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:12:25.270302+0000","last_undegraded":"2024-09-17T08:12:25.270302+0000","last_fullsized":"2024-09-17T08:12:25.270302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:11:12.988901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.988901+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.582059+0000","scrub_duration":0.001843676,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":145,"num_read_kb":140,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075029+0000","last_change":"2024-09-17T08:11:23.081976+0000","last_active":"2024-09-17T08:12:18.075029+0000","last_peered":"2024-09-17T08:12:18.075029+0000","last_clean":"2024-09-17T08:12:18.075029+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:12:18.075029+0000","last_undegraded":"2024-09-17T08:12:18.075029+0000","last_fullsized":"2024-09-17T08:12:18.075029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.081888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.170042+0000","scrub_duration":0.0010134619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077934+0000","last_change":"2024-09-17T08:11:27.111013+0000","last_active":"2024-09-17T08:12:18.077934+0000","last_peered":"2024-09-17T08:12:18.077934+0000","last_clean":"2024-09-17T08:12:18.077934+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:12:18.077934+0000","last_undegraded":"2024-09-17T08:12:18.077934+0000","last_fullsized":"2024-09-17T08:12:18.077934+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.110912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.110912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.377143+0000","scrub_duration":0.0013195850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077626+0000","last_change":"2024-09-17T08:11:20.454612+0000","last_active":"2024-09-17T08:12:18.077626+0000","last_peered":"2024-09-17T08:12:18.077626+0000","last_clean":"2024-09-17T08:12:18.077626+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:12:18.077626+0000","last_undegraded":"2024-09-17T08:12:18.077626+0000","last_fullsized":"2024-09-17T08:12:18.077626+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:20.454505+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.454505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:25.882735+0000","scrub_duration":0.0015587559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"97'1578","reported_seq":1593,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.201600+0000","last_change":"2024-09-17T08:12:05.207496+0000","last_active":"2024-09-17T08:12:27.201600+0000","last_peered":"2024-09-17T08:12:27.201600+0000","last_clean":"2024-09-17T08:12:27.201600+0000","last_became_active":"2024-09-17T08:12:05.207217+0000","last_became_peered":"2024-09-17T08:12:05.207217+0000","last_unstale":"2024-09-17T08:12:27.201600+0000","last_undegraded":"2024-09-17T08:12:27.201600+0000","last_fullsized":"2024-09-17T08:12:27.201600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1578,"log_dups_size":0,"ondisk_log_size":1578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:29.380925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103415808,"num_objects":1578,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1578,"num_write_kb":100992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":76,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077630+0000","last_change":"2024-09-17T08:11:11.895771+0000","last_active":"2024-09-17T08:12:18.077630+0000","last_peered":"2024-09-17T08:12:18.077630+0000","last_clean":"2024-09-17T08:12:18.077630+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:12:18.077630+0000","last_undegraded":"2024-09-17T08:12:18.077630+0000","last_fullsized":"2024-09-17T08:12:18.077630+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.895677+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.375972+0000","scrub_duration":0.0046429619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.078163+0000","last_change":"2024-09-17T08:11:28.111674+0000","last_active":"2024-09-17T08:12:18.078163+0000","last_peered":"2024-09-17T08:12:18.078163+0000","last_clean":"2024-09-17T08:12:18.078163+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:12:18.078163+0000","last_undegraded":"2024-09-17T08:12:18.078163+0000","last_fullsized":"2024-09-17T08:12:18.078163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:11:28.111569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:28.111569+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.306471+0000","scrub_duration":0.0016978239999999999,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074627+0000","last_change":"2024-09-17T08:11:34.356211+0000","last_active":"2024-09-17T08:12:18.074627+0000","last_peered":"2024-09-17T08:12:18.074627+0000","last_clean":"2024-09-17T08:12:18.074627+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:12:18.074627+0000","last_undegraded":"2024-09-17T08:12:18.074627+0000","last_fullsized":"2024-09-17T08:12:18.074627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:34.356129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.356129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.828603+0000","scrub_duration":0.001774224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077733+0000","last_change":"2024-09-17T08:11:21.148550+0000","last_active":"2024-09-17T08:12:18.077733+0000","last_peered":"2024-09-17T08:12:18.077733+0000","last_clean":"2024-09-17T08:12:18.077733+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:12:18.077733+0000","last_undegraded":"2024-09-17T08:12:18.077733+0000","last_fullsized":"2024-09-17T08:12:18.077733+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.148461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.148461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.066336+0000","scrub_duration":0.001417183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.358629+0000","last_change":"2024-09-17T08:11:40.881867+0000","last_active":"2024-09-17T08:12:17.358629+0000","last_peered":"2024-09-17T08:12:17.358629+0000","last_clean":"2024-09-17T08:12:17.358629+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:12:17.358629+0000","last_undegraded":"2024-09-17T08:12:17.358629+0000","last_fullsized":"2024-09-17T08:12:17.358629+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:40.881773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.881773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:47.568816+0000","scrub_duration":0.001434877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"97'1582","reported_seq":1597,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.212354+0000","last_change":"2024-09-17T08:12:05.208816+0000","last_active":"2024-09-17T08:12:27.212354+0000","last_peered":"2024-09-17T08:12:27.212354+0000","last_clean":"2024-09-17T08:12:27.212354+0000","last_became_active":"2024-09-17T08:12:05.208529+0000","last_became_peered":"2024-09-17T08:12:05.208529+0000","last_unstale":"2024-09-17T08:12:27.212354+0000","last_undegraded":"2024-09-17T08:12:27.212354+0000","last_fullsized":"2024-09-17T08:12:27.212354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1582,"log_dups_size":0,"ondisk_log_size":1582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:06.324623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103677952,"num_objects":1582,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1582,"num_write_kb":101248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":74,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076894+0000","last_change":"2024-09-17T08:11:11.289842+0000","last_active":"2024-09-17T08:12:18.076894+0000","last_peered":"2024-09-17T08:12:18.076894+0000","last_clean":"2024-09-17T08:12:18.076894+0000","last_became_active":"2024-09-17T08:09:57.042354+0000","last_became_peered":"2024-09-17T08:09:57.042354+0000","last_unstale":"2024-09-17T08:12:18.076894+0000","last_undegraded":"2024-09-17T08:12:18.076894+0000","last_fullsized":"2024-09-17T08:12:18.076894+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:11:11.289667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.289667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003013239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359501+0000","last_change":"2024-09-17T08:12:01.156516+0000","last_active":"2024-09-17T08:12:17.359501+0000","last_peered":"2024-09-17T08:12:17.359501+0000","last_clean":"2024-09-17T08:12:17.359501+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:12:17.359501+0000","last_undegraded":"2024-09-17T08:12:17.359501+0000","last_fullsized":"2024-09-17T08:12:17.359501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:12:01.156410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:01.156410+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:04.907621+0000","scrub_duration":0.0017882880000000001,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.078163+0000","last_change":"2024-09-17T08:11:19.154226+0000","last_active":"2024-09-17T08:12:18.078163+0000","last_peered":"2024-09-17T08:12:18.078163+0000","last_clean":"2024-09-17T08:12:18.078163+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:12:18.078163+0000","last_undegraded":"2024-09-17T08:12:18.078163+0000","last_fullsized":"2024-09-17T08:12:18.078163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.154106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.154106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.719400+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":315,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074778+0000","last_change":"2024-09-17T08:11:40.405063+0000","last_active":"2024-09-17T08:12:18.074778+0000","last_peered":"2024-09-17T08:12:18.074778+0000","last_clean":"2024-09-17T08:12:18.074778+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:12:18.074778+0000","last_undegraded":"2024-09-17T08:12:18.074778+0000","last_fullsized":"2024-09-17T08:12:18.074778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:11:40.404951+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.404951+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:09.296441+0000","scrub_duration":0.001835978,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076714+0000","last_change":"2024-09-17T08:11:37.370612+0000","last_active":"2024-09-17T08:12:18.076714+0000","last_peered":"2024-09-17T08:12:18.076714+0000","last_clean":"2024-09-17T08:12:18.076714+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:12:18.076714+0000","last_undegraded":"2024-09-17T08:12:18.076714+0000","last_fullsized":"2024-09-17T08:12:18.076714+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:11:37.370527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:55.452817+0000","scrub_duration":0.001376932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.078437+0000","last_change":"2024-09-17T08:11:08.227302+0000","last_active":"2024-09-17T08:12:18.078437+0000","last_peered":"2024-09-17T08:12:18.078437+0000","last_clean":"2024-09-17T08:12:18.078437+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:12:18.078437+0000","last_undegraded":"2024-09-17T08:12:18.078437+0000","last_fullsized":"2024-09-17T08:12:18.078437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.227178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.227178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:34.497786+0000","scrub_duration":0.0016917799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"97'1547","reported_seq":1562,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.214847+0000","last_change":"2024-09-17T08:12:05.207359+0000","last_active":"2024-09-17T08:12:27.214847+0000","last_peered":"2024-09-17T08:12:27.214847+0000","last_clean":"2024-09-17T08:12:27.214847+0000","last_became_active":"2024-09-17T08:12:05.207111+0000","last_became_peered":"2024-09-17T08:12:05.207111+0000","last_unstale":"2024-09-17T08:12:27.214847+0000","last_undegraded":"2024-09-17T08:12:27.214847+0000","last_fullsized":"2024-09-17T08:12:27.214847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1547,"log_dups_size":0,"ondisk_log_size":1547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:30.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101384192,"num_objects":1547,"num_object_clones":0,"num_object_copies":4641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":99008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":83,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076169+0000","last_change":"2024-09-17T08:11:27.389593+0000","last_active":"2024-09-17T08:12:18.076169+0000","last_peered":"2024-09-17T08:12:18.076169+0000","last_clean":"2024-09-17T08:12:18.076169+0000","last_became_active":"2024-09-17T08:09:57.040947+0000","last_became_peered":"2024-09-17T08:09:57.040947+0000","last_unstale":"2024-09-17T08:12:18.076169+0000","last_undegraded":"2024-09-17T08:12:18.076169+0000","last_fullsized":"2024-09-17T08:12:18.076169+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:11:27.389490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.389490+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.830431+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075844+0000","last_change":"2024-09-17T08:11:30.055058+0000","last_active":"2024-09-17T08:12:18.075844+0000","last_peered":"2024-09-17T08:12:18.075844+0000","last_clean":"2024-09-17T08:12:18.075844+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:12:18.075844+0000","last_undegraded":"2024-09-17T08:12:18.075844+0000","last_fullsized":"2024-09-17T08:12:18.075844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:30.054968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.054968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.938144+0000","scrub_duration":0.0018411460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074760+0000","last_change":"2024-09-17T08:11:15.148208+0000","last_active":"2024-09-17T08:12:18.074760+0000","last_peered":"2024-09-17T08:12:18.074760+0000","last_clean":"2024-09-17T08:12:18.074760+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:12:18.074760+0000","last_undegraded":"2024-09-17T08:12:18.074760+0000","last_fullsized":"2024-09-17T08:12:18.074760+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.148109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.148109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.448844+0000","scrub_duration":0.001749011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074701+0000","last_change":"2024-09-17T08:11:27.312687+0000","last_active":"2024-09-17T08:12:18.074701+0000","last_peered":"2024-09-17T08:12:18.074701+0000","last_clean":"2024-09-17T08:12:18.074701+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:12:18.074701+0000","last_undegraded":"2024-09-17T08:12:18.074701+0000","last_fullsized":"2024-09-17T08:12:18.074701+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:27.312595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:27.312595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:44.098135+0000","scrub_duration":0.001815178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"97'1523","reported_seq":1538,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.212951+0000","last_change":"2024-09-17T08:12:05.204285+0000","last_active":"2024-09-17T08:12:27.212951+0000","last_peered":"2024-09-17T08:12:27.212951+0000","last_clean":"2024-09-17T08:12:27.212951+0000","last_became_active":"2024-09-17T08:12:05.203964+0000","last_became_peered":"2024-09-17T08:12:05.203964+0000","last_unstale":"2024-09-17T08:12:27.212951+0000","last_undegraded":"2024-09-17T08:12:27.212951+0000","last_fullsized":"2024-09-17T08:12:27.212951+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1523,"log_dups_size":0,"ondisk_log_size":1523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:11.362211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99811328,"num_objects":1523,"num_object_clones":0,"num_object_copies":4569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1523,"num_write_kb":97472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":68,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077484+0000","last_change":"2024-09-17T08:11:30.115032+0000","last_active":"2024-09-17T08:12:18.077484+0000","last_peered":"2024-09-17T08:12:18.077484+0000","last_clean":"2024-09-17T08:12:18.077484+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:12:18.077484+0000","last_undegraded":"2024-09-17T08:12:18.077484+0000","last_fullsized":"2024-09-17T08:12:18.077484+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:30.114921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:30.114921+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.519251+0000","scrub_duration":0.0019919109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359453+0000","last_change":"2024-09-17T08:11:20.897086+0000","last_active":"2024-09-17T08:12:17.359453+0000","last_peered":"2024-09-17T08:12:17.359453+0000","last_clean":"2024-09-17T08:12:17.359453+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:12:17.359453+0000","last_undegraded":"2024-09-17T08:12:17.359453+0000","last_fullsized":"2024-09-17T08:12:17.359453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:20.896996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:20.896996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.372099+0000","scrub_duration":0.001216373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074866+0000","last_change":"2024-09-17T08:11:25.124631+0000","last_active":"2024-09-17T08:12:18.074866+0000","last_peered":"2024-09-17T08:12:18.074866+0000","last_clean":"2024-09-17T08:12:18.074866+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:12:18.074866+0000","last_undegraded":"2024-09-17T08:12:18.074866+0000","last_fullsized":"2024-09-17T08:12:18.074866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.124549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.430574+0000","scrub_duration":0.001281835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076520+0000","last_change":"2024-09-17T08:11:23.073527+0000","last_active":"2024-09-17T08:12:18.076520+0000","last_peered":"2024-09-17T08:12:18.076520+0000","last_clean":"2024-09-17T08:12:18.076520+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:12:18.076520+0000","last_undegraded":"2024-09-17T08:12:18.076520+0000","last_fullsized":"2024-09-17T08:12:18.076520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:23.073448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.073448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.281924+0000","scrub_duration":0.0012514399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"97'1551","reported_seq":1566,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.187414+0000","last_change":"2024-09-17T08:12:05.200413+0000","last_active":"2024-09-17T08:12:27.187414+0000","last_peered":"2024-09-17T08:12:27.187414+0000","last_clean":"2024-09-17T08:12:27.187414+0000","last_became_active":"2024-09-17T08:12:05.200128+0000","last_became_peered":"2024-09-17T08:12:05.200128+0000","last_unstale":"2024-09-17T08:12:27.187414+0000","last_undegraded":"2024-09-17T08:12:27.187414+0000","last_fullsized":"2024-09-17T08:12:27.187414+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:33.281379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101646336,"num_objects":1551,"num_object_clones":0,"num_object_copies":4653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1551,"num_write_kb":99264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075321+0000","last_change":"2024-09-17T08:11:23.079173+0000","last_active":"2024-09-17T08:12:18.075321+0000","last_peered":"2024-09-17T08:12:18.075321+0000","last_clean":"2024-09-17T08:12:18.075321+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:12:18.075321+0000","last_undegraded":"2024-09-17T08:12:18.075321+0000","last_fullsized":"2024-09-17T08:12:18.075321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:11:23.079044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.079044+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.355134+0000","scrub_duration":0.0023640229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.078125+0000","last_change":"2024-09-17T08:10:56.022498+0000","last_active":"2024-09-17T08:12:18.078125+0000","last_peered":"2024-09-17T08:12:18.078125+0000","last_clean":"2024-09-17T08:12:18.078125+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:12:18.078125+0000","last_undegraded":"2024-09-17T08:12:18.078125+0000","last_fullsized":"2024-09-17T08:12:18.078125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:56.022419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:56.022419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014542159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":88,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255599+0000","last_change":"2024-09-17T08:11:36.379074+0000","last_active":"2024-09-17T08:12:21.255599+0000","last_peered":"2024-09-17T08:12:21.255599+0000","last_clean":"2024-09-17T08:12:21.255599+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:12:21.255599+0000","last_undegraded":"2024-09-17T08:12:21.255599+0000","last_fullsized":"2024-09-17T08:12:21.255599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:36.378990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.378990+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:43.391768+0000","scrub_duration":0.001580082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075525+0000","last_change":"2024-09-17T08:11:32.089192+0000","last_active":"2024-09-17T08:12:18.075525+0000","last_peered":"2024-09-17T08:12:18.075525+0000","last_clean":"2024-09-17T08:12:18.075525+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:12:18.075525+0000","last_undegraded":"2024-09-17T08:12:18.075525+0000","last_fullsized":"2024-09-17T08:12:18.075525+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:32.089096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:32.089096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.048086+0000","scrub_duration":0.0013942010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"97'1603","reported_seq":1618,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:27.217132+0000","last_change":"2024-09-17T08:12:05.209733+0000","last_active":"2024-09-17T08:12:27.217132+0000","last_peered":"2024-09-17T08:12:27.217132+0000","last_clean":"2024-09-17T08:12:27.217132+0000","last_became_active":"2024-09-17T08:12:05.209262+0000","last_became_peered":"2024-09-17T08:12:05.209262+0000","last_unstale":"2024-09-17T08:12:27.217132+0000","last_undegraded":"2024-09-17T08:12:27.217132+0000","last_fullsized":"2024-09-17T08:12:27.217132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":1603,"log_dups_size":0,"ondisk_log_size":1603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:12.178178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105054208,"num_objects":1603,"num_object_clones":0,"num_object_copies":4809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1603,"num_write_kb":102592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":75,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.354849+0000","last_change":"2024-09-17T08:11:08.614679+0000","last_active":"2024-09-17T08:12:17.354849+0000","last_peered":"2024-09-17T08:12:17.354849+0000","last_clean":"2024-09-17T08:12:17.354849+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:12:17.354849+0000","last_undegraded":"2024-09-17T08:12:17.354849+0000","last_fullsized":"2024-09-17T08:12:17.354849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:08.614567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.614567+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003334184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355679+0000","last_change":"2024-09-17T08:11:10.663912+0000","last_active":"2024-09-17T08:12:17.355679+0000","last_peered":"2024-09-17T08:12:17.355679+0000","last_clean":"2024-09-17T08:12:17.355679+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:12:17.355679+0000","last_undegraded":"2024-09-17T08:12:17.355679+0000","last_fullsized":"2024-09-17T08:12:17.355679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:11:10.663779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:10.663779+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001800907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.074918+0000","last_change":"2024-09-17T08:11:03.042558+0000","last_active":"2024-09-17T08:12:18.074918+0000","last_peered":"2024-09-17T08:12:18.074918+0000","last_clean":"2024-09-17T08:12:18.074918+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:12:18.074918+0000","last_undegraded":"2024-09-17T08:12:18.074918+0000","last_fullsized":"2024-09-17T08:12:18.074918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:11:03.042476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.822638+0000","scrub_duration":0.0013185009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359837+0000","last_change":"2024-09-17T08:11:06.967842+0000","last_active":"2024-09-17T08:12:17.359837+0000","last_peered":"2024-09-17T08:12:17.359837+0000","last_clean":"2024-09-17T08:12:17.359837+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:12:17.359837+0000","last_undegraded":"2024-09-17T08:12:17.359837+0000","last_fullsized":"2024-09-17T08:12:17.359837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.967748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.967748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018596470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255724+0000","last_change":"2024-09-17T08:11:22.027206+0000","last_active":"2024-09-17T08:12:21.255724+0000","last_peered":"2024-09-17T08:12:21.255724+0000","last_clean":"2024-09-17T08:12:21.255724+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:12:21.255724+0000","last_undegraded":"2024-09-17T08:12:21.255724+0000","last_fullsized":"2024-09-17T08:12:21.255724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:22.027118+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.490136+0000","scrub_duration":0.0032544100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075038+0000","last_change":"2024-09-17T08:10:57.035501+0000","last_active":"2024-09-17T08:12:18.075038+0000","last_peered":"2024-09-17T08:12:18.075038+0000","last_clean":"2024-09-17T08:12:18.075038+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:12:18.075038+0000","last_undegraded":"2024-09-17T08:12:18.075038+0000","last_fullsized":"2024-09-17T08:12:18.075038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.035419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.035419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011499139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251810+0000","last_change":"2024-09-17T08:09:58.518973+0000","last_active":"2024-09-17T08:12:21.251810+0000","last_peered":"2024-09-17T08:12:21.251810+0000","last_clean":"2024-09-17T08:12:21.251810+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:12:21.251810+0000","last_undegraded":"2024-09-17T08:12:21.251810+0000","last_fullsized":"2024-09-17T08:12:21.251810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:09:57.490227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.249494+0000","last_change":"2024-09-17T08:11:14.110687+0000","last_active":"2024-09-17T08:12:21.249494+0000","last_peered":"2024-09-17T08:12:21.249494+0000","last_clean":"2024-09-17T08:12:21.249494+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:12:21.249494+0000","last_undegraded":"2024-09-17T08:12:21.249494+0000","last_fullsized":"2024-09-17T08:12:21.249494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.110608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.110608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.468284+0000","scrub_duration":0.001480608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077242+0000","last_change":"2024-09-17T08:11:21.318902+0000","last_active":"2024-09-17T08:12:18.077242+0000","last_peered":"2024-09-17T08:12:18.077242+0000","last_clean":"2024-09-17T08:12:18.077242+0000","last_became_active":"2024-09-17T08:09:54.498051+0000","last_became_peered":"2024-09-17T08:09:54.498051+0000","last_unstale":"2024-09-17T08:12:18.077242+0000","last_undegraded":"2024-09-17T08:12:18.077242+0000","last_fullsized":"2024-09-17T08:12:18.077242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:21.318789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:21.318789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.653328+0000","scrub_duration":0.0013793810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":61,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.253552+0000","last_change":"2024-09-17T08:11:23.028718+0000","last_active":"2024-09-17T08:12:21.253552+0000","last_peered":"2024-09-17T08:12:21.253552+0000","last_clean":"2024-09-17T08:12:21.253552+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:12:21.253552+0000","last_undegraded":"2024-09-17T08:12:21.253552+0000","last_fullsized":"2024-09-17T08:12:21.253552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:11:23.028619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:23.028619+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.748363+0000","scrub_duration":0.001692315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255664+0000","last_change":"2024-09-17T08:11:12.183764+0000","last_active":"2024-09-17T08:12:21.255664+0000","last_peered":"2024-09-17T08:12:21.255664+0000","last_clean":"2024-09-17T08:12:21.255664+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:12:21.255664+0000","last_undegraded":"2024-09-17T08:12:21.255664+0000","last_fullsized":"2024-09-17T08:12:21.255664+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.183661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.183661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.546179+0000","scrub_duration":0.0018134360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254743+0000","last_change":"2024-09-17T08:11:08.492426+0000","last_active":"2024-09-17T08:12:21.254743+0000","last_peered":"2024-09-17T08:12:21.254743+0000","last_clean":"2024-09-17T08:12:21.254743+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:12:21.254743+0000","last_undegraded":"2024-09-17T08:12:21.254743+0000","last_fullsized":"2024-09-17T08:12:21.254743+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.492197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.492197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:33.701867+0000","scrub_duration":0.001646853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":67,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251175+0000","last_change":"2024-09-17T08:11:29.979585+0000","last_active":"2024-09-17T08:12:21.251175+0000","last_peered":"2024-09-17T08:12:21.251175+0000","last_clean":"2024-09-17T08:12:21.251175+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:12:21.251175+0000","last_undegraded":"2024-09-17T08:12:21.251175+0000","last_fullsized":"2024-09-17T08:12:21.251175+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:29.979461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:29.979461+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.112016+0000","scrub_duration":0.0029734470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":63,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.075788+0000","last_change":"2024-09-17T08:11:32.437732+0000","last_active":"2024-09-17T08:12:18.075788+0000","last_peered":"2024-09-17T08:12:18.075788+0000","last_clean":"2024-09-17T08:12:18.075788+0000","last_became_active":"2024-09-17T08:11:32.437420+0000","last_became_peered":"2024-09-17T08:11:32.437420+0000","last_unstale":"2024-09-17T08:12:18.075788+0000","last_undegraded":"2024-09-17T08:12:18.075788+0000","last_fullsized":"2024-09-17T08:12:18.075788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:54.331799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:54.331799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":88,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:22.224555+0000","last_change":"2024-09-17T08:11:11.217500+0000","last_active":"2024-09-17T08:12:22.224555+0000","last_peered":"2024-09-17T08:12:22.224555+0000","last_clean":"2024-09-17T08:12:22.224555+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:12:22.224555+0000","last_undegraded":"2024-09-17T08:12:22.224555+0000","last_fullsized":"2024-09-17T08:12:22.224555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:11.217367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:11.217367+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.973225+0000","scrub_duration":0.002475348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255378+0000","last_change":"2024-09-17T08:11:24.057092+0000","last_active":"2024-09-17T08:12:21.255378+0000","last_peered":"2024-09-17T08:12:21.255378+0000","last_clean":"2024-09-17T08:12:21.255378+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:12:21.255378+0000","last_undegraded":"2024-09-17T08:12:21.255378+0000","last_fullsized":"2024-09-17T08:12:21.255378+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:24.057006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:24.057006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.156418+0000","scrub_duration":0.0013330760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.255299+0000","last_change":"2024-09-17T08:11:13.214309+0000","last_active":"2024-09-17T08:12:21.255299+0000","last_peered":"2024-09-17T08:12:21.255299+0000","last_clean":"2024-09-17T08:12:21.255299+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:12:21.255299+0000","last_undegraded":"2024-09-17T08:12:21.255299+0000","last_fullsized":"2024-09-17T08:12:21.255299+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:13.214183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.214183+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.720573+0000","scrub_duration":0.0020333399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077668+0000","last_change":"2024-09-17T08:10:57.830030+0000","last_active":"2024-09-17T08:12:18.077668+0000","last_peered":"2024-09-17T08:12:18.077668+0000","last_clean":"2024-09-17T08:12:18.077668+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:12:18.077668+0000","last_undegraded":"2024-09-17T08:12:18.077668+0000","last_fullsized":"2024-09-17T08:12:18.077668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:10:57.829926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:10:57.829926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:27.699350+0000","scrub_duration":0.001948497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":89,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:22.229849+0000","last_change":"2024-09-17T08:11:49.005429+0000","last_active":"2024-09-17T08:12:22.229849+0000","last_peered":"2024-09-17T08:12:22.229849+0000","last_clean":"2024-09-17T08:12:22.229849+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:12:22.229849+0000","last_undegraded":"2024-09-17T08:12:22.229849+0000","last_fullsized":"2024-09-17T08:12:22.229849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:11:49.005295+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:05.519890+0000","scrub_duration":0.0028223409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.242151+0000","last_change":"2024-09-17T08:11:44.613775+0000","last_active":"2024-09-17T08:12:21.242151+0000","last_peered":"2024-09-17T08:12:21.242151+0000","last_clean":"2024-09-17T08:12:21.242151+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:12:21.242151+0000","last_undegraded":"2024-09-17T08:12:21.242151+0000","last_fullsized":"2024-09-17T08:12:21.242151+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:44.613677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:44.613677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:46.055246+0000","scrub_duration":0.001294131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":68,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.248006+0000","last_change":"2024-09-17T08:11:01.275252+0000","last_active":"2024-09-17T08:12:21.248006+0000","last_peered":"2024-09-17T08:12:21.248006+0000","last_clean":"2024-09-17T08:12:21.248006+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:12:21.248006+0000","last_undegraded":"2024-09-17T08:12:21.248006+0000","last_fullsized":"2024-09-17T08:12:21.248006+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:11:01.275154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.275154+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024533549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251301+0000","last_change":"2024-09-17T08:11:12.317266+0000","last_active":"2024-09-17T08:12:21.251301+0000","last_peered":"2024-09-17T08:12:21.251301+0000","last_clean":"2024-09-17T08:12:21.251301+0000","last_became_active":"2024-09-17T08:09:54.502155+0000","last_became_peered":"2024-09-17T08:09:54.502155+0000","last_unstale":"2024-09-17T08:12:21.251301+0000","last_undegraded":"2024-09-17T08:12:21.251301+0000","last_fullsized":"2024-09-17T08:12:21.251301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.317163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.317163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.086566+0000","scrub_duration":0.001540013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254095+0000","last_change":"2024-09-17T08:11:31.019796+0000","last_active":"2024-09-17T08:12:21.254095+0000","last_peered":"2024-09-17T08:12:21.254095+0000","last_clean":"2024-09-17T08:12:21.254095+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:12:21.254095+0000","last_undegraded":"2024-09-17T08:12:21.254095+0000","last_fullsized":"2024-09-17T08:12:21.254095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:31.019695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:31.019695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:40.782145+0000","scrub_duration":0.001088767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254042+0000","last_change":"2024-09-17T08:11:19.995619+0000","last_active":"2024-09-17T08:12:21.254042+0000","last_peered":"2024-09-17T08:12:21.254042+0000","last_clean":"2024-09-17T08:12:21.254042+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:12:21.254042+0000","last_undegraded":"2024-09-17T08:12:21.254042+0000","last_fullsized":"2024-09-17T08:12:21.254042+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.995534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.995534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.902530+0000","scrub_duration":0.001317545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.241937+0000","last_change":"2024-09-17T08:11:00.599752+0000","last_active":"2024-09-17T08:12:21.241937+0000","last_peered":"2024-09-17T08:12:21.241937+0000","last_clean":"2024-09-17T08:12:21.241937+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:12:21.241937+0000","last_undegraded":"2024-09-17T08:12:21.241937+0000","last_fullsized":"2024-09-17T08:12:21.241937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:00.599623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.599623+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.968863+0000","scrub_duration":0.0021287580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359442+0000","last_change":"2024-09-17T08:11:26.933581+0000","last_active":"2024-09-17T08:12:17.359442+0000","last_peered":"2024-09-17T08:12:17.359442+0000","last_clean":"2024-09-17T08:12:17.359442+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:12:17.359442+0000","last_undegraded":"2024-09-17T08:12:17.359442+0000","last_fullsized":"2024-09-17T08:12:17.359442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.933486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.933486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:50.866440+0000","scrub_duration":0.0014923110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.247842+0000","last_change":"2024-09-17T08:11:12.013206+0000","last_active":"2024-09-17T08:12:21.247842+0000","last_peered":"2024-09-17T08:12:21.247842+0000","last_clean":"2024-09-17T08:12:21.247842+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:12:21.247842+0000","last_undegraded":"2024-09-17T08:12:21.247842+0000","last_fullsized":"2024-09-17T08:12:21.247842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:12.013125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:12.013125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001422897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.252091+0000","last_change":"2024-09-17T08:11:14.086825+0000","last_active":"2024-09-17T08:12:21.252091+0000","last_peered":"2024-09-17T08:12:21.252091+0000","last_clean":"2024-09-17T08:12:21.252091+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:12:21.252091+0000","last_undegraded":"2024-09-17T08:12:21.252091+0000","last_fullsized":"2024-09-17T08:12:21.252091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:14.086735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.086735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:23.928536+0000","scrub_duration":0.001779203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359242+0000","last_change":"2024-09-17T08:11:40.948461+0000","last_active":"2024-09-17T08:12:17.359242+0000","last_peered":"2024-09-17T08:12:17.359242+0000","last_clean":"2024-09-17T08:12:17.359242+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:12:17.359242+0000","last_undegraded":"2024-09-17T08:12:17.359242+0000","last_fullsized":"2024-09-17T08:12:17.359242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:11:40.948380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:40.948380+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:05.132541+0000","scrub_duration":0.0015753480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.079287+0000","last_change":"2024-09-17T08:11:05.179529+0000","last_active":"2024-09-17T08:12:18.079287+0000","last_peered":"2024-09-17T08:12:18.079287+0000","last_clean":"2024-09-17T08:12:18.079287+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:12:18.079287+0000","last_undegraded":"2024-09-17T08:12:18.079287+0000","last_fullsized":"2024-09-17T08:12:18.079287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:05.179419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:05.179419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001434273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077024+0000","last_change":"2024-09-17T08:11:08.311127+0000","last_active":"2024-09-17T08:12:18.077024+0000","last_peered":"2024-09-17T08:12:18.077024+0000","last_clean":"2024-09-17T08:12:18.077024+0000","last_became_active":"2024-09-17T08:09:58.513590+0000","last_became_peered":"2024-09-17T08:09:58.513590+0000","last_unstale":"2024-09-17T08:12:18.077024+0000","last_undegraded":"2024-09-17T08:12:18.077024+0000","last_fullsized":"2024-09-17T08:12:18.077024+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.310979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:31.324071+0000","scrub_duration":0.001505475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254879+0000","last_change":"2024-09-17T08:11:04.412186+0000","last_active":"2024-09-17T08:12:21.254879+0000","last_peered":"2024-09-17T08:12:21.254879+0000","last_clean":"2024-09-17T08:12:21.254879+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:12:21.254879+0000","last_undegraded":"2024-09-17T08:12:21.254879+0000","last_fullsized":"2024-09-17T08:12:21.254879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:11:04.412068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:30.005712+0000","scrub_duration":0.0016498649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":71,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359576+0000","last_change":"2024-09-17T08:11:08.985197+0000","last_active":"2024-09-17T08:12:17.359576+0000","last_peered":"2024-09-17T08:12:17.359576+0000","last_clean":"2024-09-17T08:12:17.359576+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:12:17.359576+0000","last_undegraded":"2024-09-17T08:12:17.359576+0000","last_fullsized":"2024-09-17T08:12:17.359576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:08.985086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:08.985086+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:36.941797+0000","scrub_duration":0.002086068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":59,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251666+0000","last_change":"2024-09-17T08:11:19.313452+0000","last_active":"2024-09-17T08:12:21.251666+0000","last_peered":"2024-09-17T08:12:21.251666+0000","last_clean":"2024-09-17T08:12:21.251666+0000","last_became_active":"2024-09-17T08:09:54.505803+0000","last_became_peered":"2024-09-17T08:09:54.505803+0000","last_unstale":"2024-09-17T08:12:21.251666+0000","last_undegraded":"2024-09-17T08:12:21.251666+0000","last_fullsized":"2024-09-17T08:12:21.251666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.313315+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.588077+0000","scrub_duration":0.0027616870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.073962+0000","last_change":"2024-09-17T08:11:13.529422+0000","last_active":"2024-09-17T08:12:18.073962+0000","last_peered":"2024-09-17T08:12:18.073962+0000","last_clean":"2024-09-17T08:12:18.073962+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:12:18.073962+0000","last_undegraded":"2024-09-17T08:12:18.073962+0000","last_fullsized":"2024-09-17T08:12:18.073962+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.529309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.529309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:35.415763+0000","scrub_duration":0.0014475930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":37,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.253956+0000","last_change":"2024-09-17T08:10:00.599152+0000","last_active":"2024-09-17T08:12:21.253956+0000","last_peered":"2024-09-17T08:12:21.253956+0000","last_clean":"2024-09-17T08:12:21.253956+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:12:21.253956+0000","last_undegraded":"2024-09-17T08:12:21.253956+0000","last_fullsized":"2024-09-17T08:12:21.253956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076634+0000","last_change":"2024-09-17T08:11:43.442061+0000","last_active":"2024-09-17T08:12:18.076634+0000","last_peered":"2024-09-17T08:12:18.076634+0000","last_clean":"2024-09-17T08:12:18.076634+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:12:18.076634+0000","last_undegraded":"2024-09-17T08:12:18.076634+0000","last_fullsized":"2024-09-17T08:12:18.076634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:11:43.441954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:43.441954+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:08.765259+0000","scrub_duration":0.0019032389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254091+0000","last_change":"2024-09-17T08:11:16.188158+0000","last_active":"2024-09-17T08:12:21.254091+0000","last_peered":"2024-09-17T08:12:21.254091+0000","last_clean":"2024-09-17T08:12:21.254091+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:12:21.254091+0000","last_undegraded":"2024-09-17T08:12:21.254091+0000","last_fullsized":"2024-09-17T08:12:21.254091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:16.188030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:16.188030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.826968+0000","scrub_duration":0.001467023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077553+0000","last_change":"2024-09-17T08:11:25.077750+0000","last_active":"2024-09-17T08:12:18.077553+0000","last_peered":"2024-09-17T08:12:18.077553+0000","last_clean":"2024-09-17T08:12:18.077553+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:12:18.077553+0000","last_undegraded":"2024-09-17T08:12:18.077553+0000","last_fullsized":"2024-09-17T08:12:18.077553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:11:25.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:25.077639+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:28.720107+0000","scrub_duration":0.001743512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"72'8","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251558+0000","last_change":"2024-09-17T08:11:17.279514+0000","last_active":"2024-09-17T08:12:21.251558+0000","last_peered":"2024-09-17T08:12:21.251558+0000","last_clean":"2024-09-17T08:12:21.251558+0000","last_became_active":"2024-09-17T08:09:58.514441+0000","last_became_peered":"2024-09-17T08:09:58.514441+0000","last_unstale":"2024-09-17T08:12:21.251558+0000","last_undegraded":"2024-09-17T08:12:21.251558+0000","last_fullsized":"2024-09-17T08:12:21.251558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:11:17.279419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:17.279419+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:24.879509+0000","scrub_duration":0.001513896,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359081+0000","last_change":"2024-09-17T08:11:19.931978+0000","last_active":"2024-09-17T08:12:17.359081+0000","last_peered":"2024-09-17T08:12:17.359081+0000","last_clean":"2024-09-17T08:12:17.359081+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:12:17.359081+0000","last_undegraded":"2024-09-17T08:12:17.359081+0000","last_fullsized":"2024-09-17T08:12:17.359081+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:19.931887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:11:19.931887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:26.329845+0000","scrub_duration":0.001357039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.252970+0000","last_change":"2024-09-17T08:11:26.016945+0000","last_active":"2024-09-17T08:12:21.252970+0000","last_peered":"2024-09-17T08:12:21.252970+0000","last_clean":"2024-09-17T08:12:21.252970+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:12:21.252970+0000","last_undegraded":"2024-09-17T08:12:21.252970+0000","last_fullsized":"2024-09-17T08:12:21.252970+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:26.016860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:26.016860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.628074+0000","scrub_duration":0.001281791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":71,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.076482+0000","last_change":"2024-09-17T08:11:14.278901+0000","last_active":"2024-09-17T08:12:18.076482+0000","last_peered":"2024-09-17T08:12:18.076482+0000","last_clean":"2024-09-17T08:12:18.076482+0000","last_became_active":"2024-09-17T08:09:56.503913+0000","last_became_peered":"2024-09-17T08:09:56.503913+0000","last_unstale":"2024-09-17T08:12:18.076482+0000","last_undegraded":"2024-09-17T08:12:18.076482+0000","last_fullsized":"2024-09-17T08:12:18.076482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:11:14.278766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:14.278766+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.358698+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:18.077484+0000","last_change":"2024-09-17T08:11:06.859209+0000","last_active":"2024-09-17T08:12:18.077484+0000","last_peered":"2024-09-17T08:12:18.077484+0000","last_clean":"2024-09-17T08:12:18.077484+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:12:18.077484+0000","last_undegraded":"2024-09-17T08:12:18.077484+0000","last_fullsized":"2024-09-17T08:12:18.077484+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:06.859123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:06.859123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023193100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355476+0000","last_change":"2024-09-17T08:11:36.624844+0000","last_active":"2024-09-17T08:12:17.355476+0000","last_peered":"2024-09-17T08:12:17.355476+0000","last_clean":"2024-09-17T08:12:17.355476+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:12:17.355476+0000","last_undegraded":"2024-09-17T08:12:17.355476+0000","last_fullsized":"2024-09-17T08:12:17.355476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:11:36.624764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:59.550403+0000","scrub_duration":0.00124591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.251136+0000","last_change":"2024-09-17T08:11:13.310069+0000","last_active":"2024-09-17T08:12:21.251136+0000","last_peered":"2024-09-17T08:12:21.251136+0000","last_clean":"2024-09-17T08:12:21.251136+0000","last_became_active":"2024-09-17T08:09:54.498670+0000","last_became_peered":"2024-09-17T08:09:54.498670+0000","last_unstale":"2024-09-17T08:12:21.251136+0000","last_undegraded":"2024-09-17T08:12:21.251136+0000","last_fullsized":"2024-09-17T08:12:21.251136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:13.309969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:13.309969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:32.474219+0000","scrub_duration":0.0014112110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":109,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.252637+0000","last_change":"2024-09-17T08:11:34.992216+0000","last_active":"2024-09-17T08:12:21.252637+0000","last_peered":"2024-09-17T08:12:21.252637+0000","last_clean":"2024-09-17T08:12:21.252637+0000","last_became_active":"2024-09-17T08:11:32.436438+0000","last_became_peered":"2024-09-17T08:11:32.436438+0000","last_unstale":"2024-09-17T08:12:21.252637+0000","last_undegraded":"2024-09-17T08:12:21.252637+0000","last_fullsized":"2024-09-17T08:12:21.252637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:11:34.992118+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:51.249927+0000","scrub_duration":0.0049126200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359297+0000","last_change":"2024-09-17T08:11:38.885523+0000","last_active":"2024-09-17T08:12:17.359297+0000","last_peered":"2024-09-17T08:12:17.359297+0000","last_clean":"2024-09-17T08:12:17.359297+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:12:17.359297+0000","last_undegraded":"2024-09-17T08:12:17.359297+0000","last_fullsized":"2024-09-17T08:12:17.359297+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:38.885436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:38.885436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:01.914843+0000","scrub_duration":0.0016837029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.360152+0000","last_change":"2024-09-17T08:10:00.598056+0000","last_active":"2024-09-17T08:12:17.360152+0000","last_peered":"2024-09-17T08:12:17.360152+0000","last_clean":"2024-09-17T08:12:17.360152+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:12:17.360152+0000","last_undegraded":"2024-09-17T08:12:17.360152+0000","last_fullsized":"2024-09-17T08:12:17.360152+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:09:59.494268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-17T08:12:21.254334+0000","last_change":"2024-09-17T08:11:00.987660+0000","last_active":"2024-09-17T08:12:21.254334+0000","last_peered":"2024-09-17T08:12:21.254334+0000","last_clean":"2024-09-17T08:12:21.254334+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:12:21.254334+0000","last_undegraded":"2024-09-17T08:12:21.254334+0000","last_fullsized":"2024-09-17T08:12:21.254334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:00.987524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:11:00.987524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022245699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":74,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.359703+0000","last_change":"2024-09-17T08:11:01.942994+0000","last_active":"2024-09-17T08:12:17.359703+0000","last_peered":"2024-09-17T08:12:17.359703+0000","last_clean":"2024-09-17T08:12:17.359703+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:12:17.359703+0000","last_undegraded":"2024-09-17T08:12:17.359703+0000","last_fullsized":"2024-09-17T08:12:17.359703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:11:01.942903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:11:01.942903+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:22.878217+0000","scrub_duration":0.0020950360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-17T08:12:17.355300+0000","last_change":"2024-09-17T08:11:15.637755+0000","last_active":"2024-09-17T08:12:17.355300+0000","last_peered":"2024-09-17T08:12:17.355300+0000","last_clean":"2024-09-17T08:12:17.355300+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:12:17.355300+0000","last_undegraded":"2024-09-17T08:12:17.355300+0000","last_fullsized":"2024-09-17T08:12:17.355300+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:11:15.637618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:11:15.637618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:39.324631+0000","scrub_duration":0.0020997780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":1645805568,"num_objects":25113,"num_object_clones":0,"num_object_copies":75339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25113,"num_write_kb":1607232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4939120640,"data_stored":4939120640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25113,"ondisk_log_size":25113,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":140,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070500,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":516408,"kb_used_data":473008,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":93228744,"statfs":{"total":95995035648,"available":95466233856,"internally_reserved":0,"allocated":484360192,"data_stored":483945113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":1.175}]},{"osd":1,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":1.008,"15min":1.008},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":1.095}]},{"osd":2,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":1.8859999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70899999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.556}]},{"osd":4,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55800000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33900000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.503,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41799999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:11:45 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65700000000000003,"5min":0.75,"15min":0.75},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.754}]}]},{"osd":6,"up_from":50,"seq":214748364841,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":506540,"kb_used_data":467236,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93238612,"statfs":{"total":95995035648,"available":95476338688,"internally_reserved":0,"allocated":478449664,"data_stored":478046464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.308,"5min":0.219,"15min":0.219},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63800000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.30299999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.39200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":2.5910000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.53500000000000003},"min":{"1min":0.33600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.92300000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.376,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"min":{"1min":0.38200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.585}]},{"osd":3,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.40300000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.3320000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.40000000000000002,"15min":0.36599999999999999},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.37},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.377},"min":{"1min":0.29499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38300000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.626,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61499999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.31900000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69299999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":41,"seq":176093659182,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":434984,"kb_used_data":395680,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93310168,"statfs":{"total":95995035648,"available":95549612032,"internally_reserved":0,"allocated":405176320,"data_stored":404776127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63900000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.626,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":1.101}]},{"osd":1,"last update":"Tue Sep 17 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.5,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.625,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.27800000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":1.411,"15min":1.411},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60299999999999998,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.349},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.34399999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Tue Sep 17 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50800000000000001,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64000000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65400000000000003,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.373,"15min":0.35599999999999998},"min":{"1min":0.29199999999999998,"5min":0.19,"15min":0.19},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.495},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.28499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.439}]},{"osd":6,"last update":"Tue Sep 17 08:12:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.437,"15min":0.41299999999999998},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.434,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.38700000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66000000000000003,"5min":0.752,"15min":0.752},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56499999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":500472,"kb_used_data":461168,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93244680,"statfs":{"total":95995035648,"available":95482552320,"internally_reserved":0,"allocated":472236032,"data_stored":471820578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.5,"15min":0.5},"min":{"1min":0.35599999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.26200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.52200000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.347},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":1.3919999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":2.415}]},{"osd":3,"last update":"Tue Sep 17 08:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.434},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.32000000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:12:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46899999999999997,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:11:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.498},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39400000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":857412,"kb_used_data":826300,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":92887740,"statfs":{"total":95995035648,"available":95117045760,"internally_reserved":0,"allocated":846131200,"data_stored":845702935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:12:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.214,"5min":0.189,"15min":0.189},"max":{"1min":0.47499999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.35999999999999999,"15min":0.372},"min":{"1min":0.25,"5min":0.219,"15min":0.219},"max":{"1min":0.45000000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.4239999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:12:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39700000000000002,"15min":0.40999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.39500000000000002,"15min":0.41099999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51700000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":1.304}]},{"osd":2,"last update":"Tue Sep 17 08:12:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.38900000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.48699999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.40000000000000002},"min":{"1min":0.251,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52200000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":1.47}]},{"osd":4,"last update":"Tue Sep 17 08:11:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.28699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80200000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":1.667}]},{"osd":6,"last update":"Tue Sep 17 08:12:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.4990000000000001,"5min":3.4990000000000001,"15min":3.4990000000000001},"last":1.321},{"interface":"front","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.8200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":1.748},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.79900000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":633464,"kb_used_data":606448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93111688,"statfs":{"total":95995035648,"available":95346368512,"internally_reserved":0,"allocated":621002752,"data_stored":620587287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.19400000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.48899999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":2.016},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.442,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":1.5589999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48799999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38800000000000001,"15min":0.40500000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.437}]},{"osd":4,"last update":"Tue Sep 17 08:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.502}]},{"osd":5,"last update":"Tue Sep 17 08:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":2.0680000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.403}]},{"osd":6,"last update":"Tue Sep 17 08:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.52000000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":2.1309999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.355}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":596352,"kb_used_data":569336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93148800,"statfs":{"total":95995035648,"available":95384371200,"internally_reserved":0,"allocated":583000064,"data_stored":582576155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:11:29 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.30399999999999999,"15min":0.28199999999999997},"min":{"1min":0.191,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.39400000000000002,"5min":0.625,"15min":0.625},"last":2.1030000000000002},{"interface":"front","average":{"1min":0.312,"5min":0.33300000000000002,"15min":0.30299999999999999},"min":{"1min":0.215,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46100000000000002,"5min":0.625,"15min":0.625},"last":1.1639999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":4.242},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":4.2160000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38500000000000001,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":4.4630000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.39600000000000002,"15min":0.40500000000000003},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Sep 17 08:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.087,"5min":1.329,"15min":1.329},"last":2.0449999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":1.1080000000000001,"5min":1.125,"15min":1.125},"last":1.8380000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77100000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:12:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":2.1240000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82599999999999996,"5min":1.157,"15min":1.157},"last":2.0579999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":2.0790000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":817588,"kb_used_data":782380,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":92927564,"statfs":{"total":95995035648,"available":95157825536,"internally_reserved":0,"allocated":801157120,"data_stored":800745879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33900000000000002,"15min":0.31900000000000001},"min":{"1min":0.20399999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.503},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34999999999999998,"15min":0.33700000000000002},"min":{"1min":0.21199999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.46500000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":1.1599999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36399999999999999,"15min":0.379},"min":{"1min":0.22900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55000000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":2.706},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39100000000000001,"15min":0.40400000000000003},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51200000000000001,"5min":0.626,"15min":0.626},"last":1.004}]},{"osd":3,"last update":"Tue Sep 17 08:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40799999999999997,"15min":0.436},"min":{"1min":0.28100000000000003,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.626,"15min":0.626},"last":2.843},{"interface":"front","average":{"1min":0.371,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.84899999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":0.874},"last":1.234},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":2.7389999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":1.861}]},{"osd":6,"last update":"Tue Sep 17 08:12:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.876},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.9079999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:11:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.755,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":2.7650000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":2.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":817692672,"data_stored":817692672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":609026048,"data_stored":609026048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":622329856,"data_stored":622329856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":934936576,"data_stored":934936576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":512425984,"data_stored":512425984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":411238400,"data_stored":411238400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":516816896,"data_stored":516816896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":514654208,"data_stored":514654208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:12:33.305 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-17T08:12:33.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:33.306 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:12:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:12:33.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:33.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.344 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:33.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:33.480 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:33.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:33.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:33.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:33.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:12:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:33 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:33.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:33 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:12:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:33 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:12:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:33 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:33 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:33.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:33.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:33.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:33.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:33.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:33.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:34.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:34.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:34.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 16 16 38795 38779 151.453 124.25 0.00801209 0.0065974 2024-09-17T08:12:34.141 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:34.171 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:34.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:34.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:34.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.304 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:34.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:34.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:12:34.347 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: pgmap v87: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:12:34.348 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: pgmap v87: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:12:34.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:34 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:12:34.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:34.456 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:34.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:34.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:34.615 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:34.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:34.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: pgmap v87: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:12:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:12:34.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T08:12:34.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:12:34.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:34 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:12:34.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.767 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:34.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:34.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:34.907 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:34.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:34.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:34.913 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:12:34.913 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:12:34.948 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:12:34.949 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:12:34.986 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:12:34.986 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:12:35.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:35.019 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:12:35.019 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:12:35.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:35.053 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:12:35.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:12:35.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 17 16 40596 40580 149.164 112.562 0.00651328 0.00669874 2024-09-17T08:12:35.086 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:12:35.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:12:35.125 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:12:35.125 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:12:35.154 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:35.158 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:12:35.159 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:12:35.182 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:35.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.183 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:35.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:35.308 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:12:35.308 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:12:35.308 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1327216893' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:35.309 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:12:35.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:35.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:35.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:35.459 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:35.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:35.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:35.603 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:35.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:12:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:12:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1327216893' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:35 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:12:35.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:35 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:12:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:35 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:12:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:35 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1327216893' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:35 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:12:35.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:35.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:35.833 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:35.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:35.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:35.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:35.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:36.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:36.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:36.074 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 18 15 42368 42353 147.033 110.812 0.0112756 0.00679608 2024-09-17T08:12:36.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:36.124 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:36.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:36.225 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:36.257 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:36.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:36.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:36.394 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:36.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:36.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:36.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:36.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:12:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: pgmap v89: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:12:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:12:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:12:36.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[40649]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: pgmap v89: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:12:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:36 smithi079 ceph-mon[29565]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T08:12:36.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:36.652 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:36.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:12:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: pgmap v89: 148 pgs: 148 active+clean; 2.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:12:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:12:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:12:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:12:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:12:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:36 smithi173 ceph-mon[33187]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T08:12:36.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:36.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:36.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:36.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:36.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:36.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:36.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:37.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:37.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:37.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:37.073 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 19 16 44015 43999 144.709 102.875 0.00714699 0.00690528 2024-09-17T08:12:37.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:37.183 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:37.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:37.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:37.325 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:37.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:37.427 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:37.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:37.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:37.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:37.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:37.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:12:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:12:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:37 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:12:37.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:37.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:37.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:12:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:37 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:12:37.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:37.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:37.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:37.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:37.993 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:37.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:37.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:38.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:12:38.075545+0000 min lat: 0.00156972 max lat: 0.158314 avg lat: 0.00699909 2024-09-17T08:12:38.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:12:38.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 20 16 45704 45688 142.747 105.562 0.00829868 0.00699909 2024-09-17T08:12:38.096 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:38.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:38.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:38.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:38.263 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:38.305 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:12:38.305 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:12:38.306 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:12:38.306 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-17T08:12:38.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 0 2024-09-17T08:12:38.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:38.391 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:38.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:38.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:38.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.556 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:38.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[29565]: pgmap v91: 148 pgs: 148 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.10k op/s 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[40649]: pgmap v91: 148 pgs: 148 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.10k op/s 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:12:38.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:38 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:12:38.658 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:38.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:38.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:38 smithi173 ceph-mon[33187]: pgmap v91: 148 pgs: 148 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.10k op/s 2024-09-17T08:12:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:38 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:12:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:38 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:12:38.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:38.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:38.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:38.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:38.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:38.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:38.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:39.068 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:39.074 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 21 16 47397 47381 140.988 105.812 0.00893766 0.00708749 2024-09-17T08:12:39.095 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:39.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:39.153 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:39.182 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:39.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:39.205 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:12:39.205 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:12:39.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:39.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:39.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:39.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:39.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:39.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:39.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:39.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:12:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:12:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[29565]: pgmap v92: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:12:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:39 smithi079 ceph-mon[40649]: pgmap v92: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-09-17T08:12:39.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:39.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.664 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:39.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:39 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:12:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:39 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:12:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:39 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:12:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:39 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:12:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:39 smithi173 ceph-mon[33187]: pgmap v92: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.38k op/s 2024-09-17T08:12:39.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:39.803 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:39.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:39.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:39.933 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:39.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:39.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:40.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:40.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 22 16 49054 49038 139.285 103.562 0.00415763 0.00717388 2024-09-17T08:12:40.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:40.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:40.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:40.205 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:40.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:40.346 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:40.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:40.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:12:40.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:40.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:40.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:40.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:40.612 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:12:40.612 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:12:40.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:12:40.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:12:40.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:12:40.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:12:40.615 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:40 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:12:40.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:12:40.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.629 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:40.630 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:12:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:12:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:12:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:12:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:12:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:40 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:12:40.730 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:40.759 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:40.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:40.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:40.915 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:40.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:40.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:41.017 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:41.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:41.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:41.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 23 16 50661 50645 137.596 100.438 0.00648779 0.00726274 2024-09-17T08:12:41.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:41.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:41.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:41.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:41.348 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:41.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:41.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:41.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:41.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:41.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:41.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:41.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[29565]: pgmap v94: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 643 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T08:12:41.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:41 smithi079 ceph-mon[40649]: pgmap v94: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 643 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:12:41.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:12:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:12:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:12:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:12:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T08:12:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:41 smithi173 ceph-mon[33187]: pgmap v94: 148 pgs: 148 active+clean; 2.7 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 643 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:12:41.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:41.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:41.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:41.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:41.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:41.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:41.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:42.010 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:42.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 24 16 52333 52317 136.214 104.5 0.0107579 0.00733604 2024-09-17T08:12:42.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:42.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:42.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:42.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:42.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:42.399 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:42.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:42.535 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:42.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.536 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:12:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:12:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:12:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:42 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:12:42.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:42.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:42.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:42 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:12:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:42 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:12:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:42 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:12:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:42 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:12:42.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:42.794 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:42.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:42.927 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:42.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:42.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:43.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:43.068 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:43.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:43.075 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 25 16 53913 53897 134.715 98.75 0.00963692 0.00741726 2024-09-17T08:12:43.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:43.207 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:43.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:43.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:43.335 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:43.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:43.361 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:12:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:12:43.437 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:43.464 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:43.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:43.566 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:43.596 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:43.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:12:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:12:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:12:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:12:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[29565]: pgmap v95: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:43 smithi079 ceph-mon[40649]: pgmap v95: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-17T08:12:43.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:43 smithi173 ceph-mon[33187]: pgmap v95: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-17T08:12:43.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:43.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:43.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:43.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:43.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:43.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:43.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:44.016 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:44.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:44.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 26 16 55382 55366 133.065 91.8125 0.00733815 0.00750976 2024-09-17T08:12:44.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:44.147 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:44.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:44.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:44.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:44.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:44.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:44.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:44.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:44.520 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:44.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:12:44.557 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:12:44.558 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:12:44.559 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:12:44.559 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:44 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:12:44.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:44.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:44.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:12:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:12:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:12:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:12:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:12:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:44 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:12:44.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:44.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:44.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:44.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:44.945 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:44.950 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:44.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:44.963 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:44.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:45.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:45.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 27 16 56756 56740 131.315 85.875 0.0110218 0.00761002 2024-09-17T08:12:45.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.105 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:45.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:45.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:45.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:45.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:45.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:45.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:45.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:45.401 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-17T08:12:45.403 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:12:45.485 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:45.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:12:45.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1807327993' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4029430630' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[40649]: pgmap v97: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:12:45.507 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:12:45.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1807327993' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4029430630' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:45 smithi079 ceph-mon[29565]: pgmap v97: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:12:45.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.509 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:45.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:45.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:45.631 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:45.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1807327993' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4029430630' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:12:45.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:45 smithi173 ceph-mon[33187]: pgmap v97: 148 pgs: 148 active+clean; 3.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:12:45.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:45.754 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:45.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:45.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:45.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:45.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:45.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:45.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:46.002 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:46.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 28 16 57343 57327 127.936 36.6875 0.00255016 0.00763749 2024-09-17T08:12:46.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:46.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.127 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:46.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:46.266 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:46.368 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:46.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:46.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:46.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:46.540 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[40649]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T08:12:46.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T08:12:46.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:46 smithi079 ceph-mon[29565]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T08:12:46.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:46.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:46.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T08:12:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:46 smithi173 ceph-mon[33187]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T08:12:46.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:46.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:46.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:46.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:46.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:47.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:47.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 29 15 58615 58600 126.267 79.5625 0.0141595 0.00791491 2024-09-17T08:12:47.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:47.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:47.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:47.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:47.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:47.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.365 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:47.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:47.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:47.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:47.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:47.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 6.5 deep-scrub starts 2024-09-17T08:12:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 6.5 deep-scrub ok 2024-09-17T08:12:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 2.1a deep-scrub starts 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: 2.1a deep-scrub ok 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[29565]: pgmap v100: 148 pgs: 148 active+clean; 3.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 6.5 deep-scrub starts 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 6.5 deep-scrub ok 2024-09-17T08:12:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 2.1a deep-scrub starts 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: 2.1a deep-scrub ok 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:47 smithi079 ceph-mon[40649]: pgmap v100: 148 pgs: 148 active+clean; 3.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:12:47.632 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:47.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 6.5 deep-scrub starts 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 6.5 deep-scrub ok 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 2.1a deep-scrub starts 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: 2.1a deep-scrub ok 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:12:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:47 smithi173 ceph-mon[33187]: pgmap v100: 148 pgs: 148 active+clean; 3.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:12:47.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:47.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:47.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:47.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:47.903 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:47.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:47.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:48.005 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:48.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:48.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:48.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 30 16 60122 60106 125.196 94.125 0.0118217 0.00798337 2024-09-17T08:12:48.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:48.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:48.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:48.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:48.306 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:48.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:48.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:48.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:48.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:48.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:48 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:12:48.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:48 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:12:48.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:48 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:12:48.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:48 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:12:48.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:48.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.598 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:48.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:12:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:12:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:12:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:12:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:12:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:12:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:12:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:48 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:12:48.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:48.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:48.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:48.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:48.870 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:48.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:48.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:48.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:49.015 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:49.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:49.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 31 15 61592 61577 124.123 91.9375 0.0101556 0.00805183 2024-09-17T08:12:49.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:49.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:49.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:49.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:49.294 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:49.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:49.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:49.421 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:49.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:49.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:49.549 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:12:49.549 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:12:49.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:49.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:49.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[40649]: pgmap v101: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:12:49.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:49 smithi079 ceph-mon[29565]: pgmap v101: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:49.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:49.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:49.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:49 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:12:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:49 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:12:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:49 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:12:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:49 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:12:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:49 smithi173 ceph-mon[33187]: pgmap v101: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:49.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:49.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:49.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:12:49.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:49.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:49.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:49.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:50.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:50.076 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 32 16 62778 62762 122.558 74.0625 0.0143884 0.00815365 2024-09-17T08:12:50.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:50.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:50.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:50.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:50.323 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:50.348 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:50.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:50.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.474 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:50.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:12:50.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:12:50.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4036802586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:12:50.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4036802586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:50.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:50.600 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:50.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:12:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:12:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:12:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:12:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:12:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:12:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4036802586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:12:50.727 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:50.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:50.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:50.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:50.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:50.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:50.999 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:50.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:51.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:12:51.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:12:51.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:12:51.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 33 16 63825 63809 120.827 65.4375 0.0144645 0.00827094 2024-09-17T08:12:51.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:51.135 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:51.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:51.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:51.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:51.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:51.412 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:51.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:51.514 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:51.541 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:51.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:51.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:51.682 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:51.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:12:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:12:51.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T08:12:51.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:51 smithi173 ceph-mon[33187]: pgmap v103: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T08:12:51.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[29565]: pgmap v103: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T08:12:51.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:51 smithi079 ceph-mon[40649]: pgmap v103: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 341 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:51.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:12:51.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:51.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:51.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:51.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:51.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:51.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:52.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:52.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 34 16 65173 65157 119.751 84.25 0.00889518 0.00834588 2024-09-17T08:12:52.097 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:52.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:52.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:52.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:52.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:52.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:52.384 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:12:52.384 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:12:52.385 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T08:12:52.385 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:12:52.385 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:12:52.385 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-17T08:12:52.414 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:52.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:52.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:52.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:52.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:52.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:52.670 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:12:52.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:52.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:52.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:12:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:12:52.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:52.814 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:52.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:52.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:52.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:52.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:52.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:53.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:53.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 35 16 66474 66458 118.653 81.3125 0.0122937 0.00842196 2024-09-17T08:12:53.092 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:53.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:53.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:53.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:53.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:53.329 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:53.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:12:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:12:53.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:12:53.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 4.f deep-scrub starts 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 4.f deep-scrub ok 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 4.f deep-scrub starts 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 4.f deep-scrub ok 2024-09-17T08:12:53.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:12:53.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:53 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:12:53.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:53.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:12:53.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:12:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 4.f deep-scrub starts 2024-09-17T08:12:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 4.f deep-scrub ok 2024-09-17T08:12:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:12:53.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:53 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:12:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:53.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:53.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:53.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:53.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:53.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:53.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:53.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:53.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:53.886 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:53.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:53.887 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:53.988 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:54.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:54.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 36 16 67653 67637 117.404 73.6875 0.0088128 0.00851265 2024-09-17T08:12:54.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:12:54.137 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:54.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:12:54.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:12:54.239 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:54.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.266 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:54.367 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:54.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.393 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:54.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:54.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:54 smithi173 ceph-mon[33187]: pgmap v104: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 912 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-17T08:12:54.417 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:54 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:12:54.418 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:54 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:12:54.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:54.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:54.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.535 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[29565]: pgmap v104: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 912 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:12:54.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[40649]: pgmap v104: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 912 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-17T08:12:54.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:12:54.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:54 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:12:54.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:54.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:54.706 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:12:54.706 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "" 2024-09-17T08:12:54.706 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:12:54.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:54.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:54.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:54.914 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:54.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:54.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:55.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:55.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:55.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:55.077 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 37 16 68725 68709 116.041 67 0.0121418 0.00861254 2024-09-17T08:12:55.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:55.184 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:55.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:55.212 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:55.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:55.331 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:55.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:12:55.340 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:55 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:12:55.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:55 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:12:55.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:55 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:12:55.379 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:55 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:12:55.379 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:55 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:12:55.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:55.460 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:55.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:55.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:55.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:55.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:55.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:55.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:55.807 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:12:55.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:55.891 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:55.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:55.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:55.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:56.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:56.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 38 16 70028 70012 115.129 81.4375 0.0171256 0.0086804 2024-09-17T08:12:56.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:56.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.150 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:56.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:56.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:56.315 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[29565]: pgmap v105: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 789 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[40649]: pgmap v105: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 789 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:56 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:12:56.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:56.404 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:56 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:12:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:56 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:12:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:56 smithi173 ceph-mon[33187]: pgmap v105: 148 pgs: 148 active+clean; 4.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 789 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:12:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:56 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:12:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:56 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:12:56.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:56.533 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:56.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:56.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:56.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:56.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:56.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:56.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:56.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:56.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:56.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:57.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:57.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:57.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:57.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 39 16 71251 71235 114.137 76.4375 0.00665514 0.00875658 2024-09-17T08:12:57.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:57.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:57.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:57.218 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:12:57.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:57.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:57.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:57.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:57.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:57.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:57.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:12:57.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:12:57.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:12:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:12:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:12:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:12:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/650042782' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:57.589 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:57.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:12:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/650042782' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:12:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:12:57.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/650042782' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:12:57.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:12:57.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:57.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:12:57.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:57.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:57.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:57.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:57.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:57.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:57.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:58.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:12:58.078975+0000 min lat: 0.00156972 max lat: 0.80354 avg lat: 0.00881585 2024-09-17T08:12:58.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:12:58.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 40 16 72579 72563 113.359 83 0.00460686 0.00881585 2024-09-17T08:12:58.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:58.142 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:58.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:58.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:58.284 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:58.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:58.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[29565]: pgmap v107: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T08:12:58.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:12:58 smithi079 ceph-mon[40649]: pgmap v107: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:58.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:58.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:58.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:12:58.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:58.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:58.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:58.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:58.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:58.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:12:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:12:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:12:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:12:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:12:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:12:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T08:12:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:12:58 smithi173 ceph-mon[33187]: pgmap v107: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:12:58.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:12:58.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:58.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:12:58.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:58.963 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:58.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:58.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:59.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:59.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 41 16 73862 73846 112.549 80.1875 0.0242311 0.008879 2024-09-17T08:12:59.114 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:59.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:59.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:59.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:59.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:59.344 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:12:59.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:59.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:12:59.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:12:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:59.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:12:59.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:12:59.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:59.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:12:59.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:12:59.782 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:12:59.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:12:59.884 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:12:59.912 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:12:59.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:12:59.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:00.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:00.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:00.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:00.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 42 16 75067 75051 111.663 75.3125 0.00340414 0.00894966 2024-09-17T08:13:00.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:13:00.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:00.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:00.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:00.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:00.378 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:00.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:00.410 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:13:00.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-17T08:13:00.446 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:00 smithi079 ceph-mon[29565]: pgmap v108: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-17T08:13:00.447 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:00 smithi079 ceph-mon[40649]: pgmap v108: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-17T08:13:00.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:00.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:00.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:00.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:00.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:00.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.663 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:00.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:00 smithi173 ceph-mon[33187]: pgmap v108: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-17T08:13:00.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:00.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:00.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:00.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:00.945 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:00.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:00.946 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:01.046 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:01.075 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:01.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:01.078 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 43 16 76133 76117 110.615 66.625 0.0116175 0.009035 2024-09-17T08:13:01.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:01.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.219 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:01.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:01.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:01.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:01.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:01.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:01.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:01.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:01.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:01.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:01.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:01.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:01.758 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:01.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:01.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:01.899 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:01.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:01.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:02.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:02.040 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:02.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:02.079 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 44 16 77352 77336 109.829 76.1875 0.00532562 0.00910044 2024-09-17T08:13:02.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:02.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:02.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:02.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[40649]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[40649]: pgmap v110: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[29565]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[29565]: pgmap v110: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:02.210 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:02.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:02.295 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:02.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:02.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.14156.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 316.622986056, 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.76600744499999995, 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.14156", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3019822778", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746537+0000", 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746543+0000", 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0100000000000001e-06 2024-09-17T08:13:02.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746551+0000", 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9650000000000008e-06 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746559+0000", 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6389999999999998e-06 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746611+0000", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1743999999999998e-05 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746621+0000", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5219999999999996e-06 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511508+0000", 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.76488780899999997 2024-09-17T08:13:02.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511664+0000", 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000155754 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511784+0000", 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120276 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512293+0000", 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050823700000000003 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512443+0000", 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015041300000000001 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512486+0000", 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2548e-05 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512525+0000", 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9128000000000002e-05 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512545+0000", 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-05 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:30840 7.b 7:dda5b3aa:::benchmark_data_smithi079_83551_object30839:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.330232473999999, 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14325889999999999, 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30840 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039290+0000", 2024-09-17T08:13:02.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039314+0000", 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3815000000000002e-05 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039327+0000", 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3742e-05 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039335+0000", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2930000000000003e-06 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039396+0000", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1291000000000007e-05 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039499+0000", 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102843 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039584+0000", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5100999999999997e-05 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040576+0000", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099186099999999991 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040910+0000", 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033465699999999998 2024-09-17T08:13:02.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182472+0000", 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14156137299999999 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182514+0000", 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2103e-05 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182550+0000", 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6454e-05 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:32612 7.b 7:deb68128:::benchmark_data_smithi079_83551_object32611:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.973440+0000", 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.396083979, 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063835635000000002, 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32612 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973440+0000", 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973440+0000", 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973439+0000", 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973461+0000", 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2093999999999999e-05 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973504+0000", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3302e-05 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.973532+0000", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8388999999999999e-05 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974988+0000", 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014560549999999999 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.975066+0000", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7472999999999995e-05 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.975188+0000", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012242700000000001 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.976693+0000", 2024-09-17T08:13:02.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015052290000000001 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.976698+0000", 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2250000000000002e-06 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977206+0000", 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050870799999999997 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037235+0000", 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060028209999999999 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037275+0000", 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0839e-05 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:32616 7.b 7:d4a90d10:::benchmark_data_smithi079_83551_object32615:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.974081+0000", 2024-09-17T08:13:02.437 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.395443082, 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063912359000000002, 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32616 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974081+0000", 2024-09-17T08:13:02.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974081+0000", 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974080+0000", 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974100+0000", 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0356e-05 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974109+0000", 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0410000000000003e-06 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.974115+0000", 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9490000000000001e-06 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.975913+0000", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017973609999999999 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.976049+0000", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013680800000000001 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.976138+0000", 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8484000000000002e-05 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.976788+0000", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064992200000000004 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037324+0000", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060536118999999999 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037607+0000", 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028347000000000002 2024-09-17T08:13:02.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037952+0000", 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000344378 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037993+0000", 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1257999999999997e-05 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:32612 7.b e98/91 ondisk, result = 0)", 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.977164+0000", 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.392359548999998, 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060124057000000002, 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977164+0000", 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977164+0000", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977164+0000", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977167+0000", 2024-09-17T08:13:02.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7340000000000002e-06 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977177+0000", 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6860000000000001e-06 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977181+0000", 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3030000000000001e-06 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977201+0000", 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9245e-05 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977204+0000", 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2619999999999999e-06 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037288+0000", 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060084409999999998 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:32616 7.b e98/91 ondisk, result = 0)", 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.977315+0000", 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.392208934999999, 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060684912000000001, 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977315+0000", 2024-09-17T08:13:02.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977315+0000", 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977314+0000", 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977318+0000", 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1140000000000002e-06 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037181+0000", 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059863853000000002 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037208+0000", 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6639e-05 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037595+0000", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038665999999999999 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037602+0000", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2490000000000004e-06 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.038000+0000", 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039785200000000003 2024-09-17T08:13:02.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:32616 7.b e98/91 ondisk, result = 0)", 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.977377+0000", 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.392147010999999, 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059950886000000002, 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977377+0000", 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977377+0000", 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977376+0000", 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.977380+0000", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2939999999999999e-06 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037179+0000", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059798565999999997 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037209+0000", 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9811999999999998e-05 2024-09-17T08:13:02.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037309+0000", 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010021799999999999 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037318+0000", 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9670000000000007e-06 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:31.037328+0000", 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8039999999999996e-06 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:65732 7.d 7:b92ab72a:::benchmark_data_smithi079_83551_object65731:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:52.496829+0000", 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.8726952230000009, 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.037214329999999997, 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 65732 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496829+0000", 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496829+0000", 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496827+0000", 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496849+0000", 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1710999999999999e-05 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496861+0000", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1832e-05 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.496868+0000", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9489999999999999e-06 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.499947+0000", 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003079119 2024-09-17T08:13:02.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.500022+0000", 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4549999999999996e-05 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.500098+0000", 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6019000000000005e-05 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.501270+0000", 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011726040000000001 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.501718+0000", 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044775099999999998 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.533713+0000", 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031994957999999997 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.533858+0000", 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014542299999999999 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.534043+0000", 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184651 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:65962 7.d 7:b623dc43:::benchmark_data_smithi079_83551_object65961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:13:02.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:52.661347+0000", 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7081765349999998, 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046205883000000003, 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 65962 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661347+0000", 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661347+0000", 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661346+0000", 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661393+0000", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7615999999999999e-05 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661405+0000", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2112e-05 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661412+0000", 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9299999999999997e-06 2024-09-17T08:13:02.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661461+0000", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8581000000000001e-05 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661575+0000", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113792 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.661699+0000", 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012438400000000001 2024-09-17T08:13:02.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.663734+0000", 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020346520000000001 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.664308+0000", 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00057408800000000005 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.707431+0000", 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043123028000000001 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.707516+0000", 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5239000000000002e-05 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:52.707553+0000", 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6956999999999999e-05 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:67126 7.d 7:bc399fa2:::benchmark_data_smithi079_83551_object67125:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:53.543975+0000", 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.8255488829999997, 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.092908830999999997, 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 67126 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543975+0000", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543975+0000", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543973+0000", 2024-09-17T08:13:02.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543994+0000", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0690999999999999e-05 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.544013+0000", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9304e-05 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.544020+0000", 2024-09-17T08:13:02.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1210000000000003e-06 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.544068+0000", 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8075000000000003e-05 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.544176+0000", 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107948 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.544303+0000", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012709099999999999 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.546899+0000", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002596049 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.547082+0000", 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018301 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.636777+0000", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.089695669000000006 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.636841+0000", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3411000000000007e-05 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.636884+0000", 2024-09-17T08:13:02.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2948999999999998e-05 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:74790 7.d 7:ba1a256d:::benchmark_data_smithi079_83551_object74789:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.776297+0000", 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.593226831, 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059469199, 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74790 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776297+0000", 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776297+0000", 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776295+0000", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776317+0000", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1437000000000002e-05 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776328+0000", 2024-09-17T08:13:02.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1382999999999999e-05 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776335+0000", 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7390000000000002e-06 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776374+0000", 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9764000000000001e-05 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776483+0000", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010828900000000001 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.776634+0000", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000151547 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781044+0000", 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004409306 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781323+0000", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027948399999999998 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.835510+0000", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054187369999999999 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.835676+0000", 2024-09-17T08:13:02.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016545199999999999 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.835766+0000", 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0264000000000002e-05 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:74800 7.d 7:b37cf599:::benchmark_data_smithi079_83551_object74799:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.781759+0000", 2024-09-17T08:13:02.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.5877649099999998, 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.074006231000000006, 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74800 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781759+0000", 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781759+0000", 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781757+0000", 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781786+0000", 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8297e-05 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781798+0000", 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2641e-05 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781805+0000", 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9369999999999998e-06 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781916+0000", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110615 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.782034+0000", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118393 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.782176+0000", 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014174200000000001 2024-09-17T08:13:02.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784340+0000", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021640040000000002 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826141+0000", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041801356999999997 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.855651+0000", 2024-09-17T08:13:02.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029509382000000001 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.855698+0000", 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7451999999999999e-05 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.855765+0000", 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6896999999999993e-05 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:74801 7.b 7:d5442b91:::benchmark_data_smithi079_83551_object74800:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.781919+0000", 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.587605092, 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045699559000000001, 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74801 2024-09-17T08:13:02.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781919+0000", 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781919+0000", 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781918+0000", 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781937+0000", 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9476000000000001e-05 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781947+0000", 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2399999999999996e-06 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781953+0000", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9630000000000003e-06 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.781991+0000", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8439000000000002e-05 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.782102+0000", 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011073699999999999 2024-09-17T08:13:02.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.782279+0000", 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017683600000000001 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826440+0000", 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044160921999999998 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826486+0000", 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6233000000000003e-05 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826973+0000", 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048699799999999999 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.827579+0000", 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00060625399999999997 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.827618+0000", 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9251000000000001e-05 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:74800 7.d e106/103 ondisk, result = 0)", 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.784532+0000", 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.584992009, 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041612557000000001, 2024-09-17T08:13:02.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784532+0000", 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784532+0000", 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784531+0000", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784536+0000", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.57e-06 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784546+0000", 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.91e-06 2024-09-17T08:13:02.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784553+0000", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8299999999999998e-06 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826126+0000", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041572899000000003 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826135+0000", 2024-09-17T08:13:02.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5040000000000002e-06 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826144+0000", 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:74801 7.b e106/91 ondisk, result = 0)", 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.784616+0000", 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.5849074810000001, 2024-09-17T08:13:02.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041872365000000002, 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784616+0000", 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784616+0000", 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784616+0000", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784619+0000", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8430000000000001e-06 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826035+0000", 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041415983000000003 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826095+0000", 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0047000000000003e-05 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826472+0000", 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037737300000000002 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826480+0000", 2024-09-17T08:13:02.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8569999999999995e-06 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826489+0000", 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6959999999999994e-06 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:74801 7.b e106/91 ondisk, result = 0)", 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.784720+0000", 2024-09-17T08:13:02.503 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.5848040220000001, 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042905909999999998, 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784720+0000", 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784720+0000", 2024-09-17T08:13:02.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784719+0000", 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.784722+0000", 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8729999999999999e-06 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826169+0000", 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041446885000000003 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826176+0000", 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6989999999999997e-06 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826959+0000", 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078331899999999999 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.826967+0000", 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6899999999999992e-06 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.827626+0000", 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065880799999999998 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:75003 7.d 7:bd9ea7a1:::benchmark_data_smithi079_83551_object75002:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.987795+0000", 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.381728904, 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.060909208999999999, 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75003 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987795+0000", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987795+0000", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987793+0000", 2024-09-17T08:13:02.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987815+0000", 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1826000000000001e-05 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987827+0000", 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2121e-05 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.987834+0000", 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6250000000000001e-06 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.988451+0000", 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061706800000000002 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.988620+0000", 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000169028 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.988741+0000", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121255 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.994233+0000", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054918659999999998 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.994272+0000", 2024-09-17T08:13:02.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8776999999999998e-05 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.048522+0000", 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054250318999999998 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.048571+0000", 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8562e-05 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.048704+0000", 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:75011 7.d 7:b44a872e:::benchmark_data_smithi079_83551_object75010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:59.990080+0000", 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3794438310000001, 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069332184000000005, 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75011 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990080+0000", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990080+0000", 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990079+0000", 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990100+0000", 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1763000000000002e-05 2024-09-17T08:13:02.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.517 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:02 smithi173 ceph-mon[33187]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T08:13:02.517 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:02 smithi173 ceph-mon[33187]: pgmap v110: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-17T08:13:02.517 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:02.517 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:02.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:02.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990111+0000", 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105999999999999e-05 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.990117+0000", 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.477e-06 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.991474+0000", 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013565020000000001 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.991586+0000", 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011218900000000001 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.991715+0000", 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000129445 2024-09-17T08:13:02.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.994264+0000", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025483929999999999 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:59.994809+0000", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054591100000000005 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.059338+0000", 2024-09-17T08:13:02.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.064528557 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.059380+0000", 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.059412+0000", 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1788999999999997e-05 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:75532 7.d 7:bb00a2c9:::benchmark_data_smithi079_83551_object75531:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:00.521747+0000", 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.8477770790000001, 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040441894999999999, 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75532 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521747+0000", 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521747+0000", 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521745+0000", 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521766+0000", 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0466e-05 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521777+0000", 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1413999999999999e-05 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.521792+0000", 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4419e-05 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.522004+0000", 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000212606 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.522110+0000", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010542100000000001 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.522231+0000", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121563 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.524832+0000", 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002600977 2024-09-17T08:13:02.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.524996+0000", 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016347900000000001 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.562064+0000", 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.037067779000000002 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.562149+0000", 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5364000000000005e-05 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.562189+0000", 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9662000000000001e-05 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:75728 7.d 7:b96e2148:::benchmark_data_smithi079_83551_object75727:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T08:13:02.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:00.694661+0000", 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.6748632400000001, 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042244706, 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75728 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:02.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694661+0000", 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694661+0000", 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694659+0000", 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694679+0000", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0176000000000001e-05 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694688+0000", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1919999999999994e-06 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694694+0000", 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0799999999999996e-06 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694728+0000", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4839999999999998e-05 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694811+0000", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3007e-05 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.694890+0000", 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8895999999999998e-05 2024-09-17T08:13:02.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.696659+0000", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001768773 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.697789+0000", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011299109999999999 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.736843+0000", 2024-09-17T08:13:02.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.039053747 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.736873+0000", 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0732000000000003e-05 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:00.736905+0000", 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1835000000000003e-05 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.536 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:02.536 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:02.536 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:13:02.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:02.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:02.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:02.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:02.783 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:02.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:02.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:02.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:02.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:02.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:03.017 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:03.043 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:03.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:03.080 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 45 16 78472 78456 108.944 70 0.00724851 0.00917392 2024-09-17T08:13:03.139 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:13:03.140 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:13:03.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:13:03.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:13:03.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:13:03.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:13:03.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:13:03.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:03.184 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:03.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:03.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:03.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:03.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.314 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:03.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1981761443' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:03.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1981761443' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:03.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:03.451 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:03.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:03.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1981761443' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:03.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:03.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:03.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:03.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:03.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:03.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:03.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:03.862 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:03.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:03.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:03.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:04.004 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:04.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:04.080 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 46 16 79735 79719 108.292 78.9375 0.0146868 0.00923002 2024-09-17T08:13:04.165 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:04.193 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:04.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:04.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:04.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:04.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:04.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.462 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:04.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:04 smithi173 ceph-mon[33187]: pgmap v111: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:04.564 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:04.589 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:04.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:04 smithi079 ceph-mon[29565]: pgmap v111: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:04 smithi079 ceph-mon[40649]: pgmap v111: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:04.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:04.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:04.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:04.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:04.845 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:04.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:04.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:04.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:04.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:04.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:05.080 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 47 16 80936 80920 107.585 75.0625 0.00950469 0.00928978 2024-09-17T08:13:05.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:05.116 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:05.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:05.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:05.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:05.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:05.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:05.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:05.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:05.485 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:05.511 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:05.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:05.612 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:05.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:05.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:05.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:05.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:05.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:05.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:05.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:05.890 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:05.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:05.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:06.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:06.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:06.080 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 48 16 82073 82057 106.824 71.0625 0.0252213 0.00935598 2024-09-17T08:13:06.131 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:06.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:06.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:06.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:06.293 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:06.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:06.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:06.420 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:06.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:06.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:06.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:06.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:06.576 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:06 smithi079 ceph-mon[29565]: pgmap v112: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:06.576 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:06 smithi079 ceph-mon[40649]: pgmap v112: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:06.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:13:06.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:06.668 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:06.675 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:06.677 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:06.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:06 smithi173 ceph-mon[33187]: pgmap v112: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:13:06.778 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:06.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:06.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:06.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:06.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:06.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:06.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:07.050 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:07.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:13:07.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:07.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:07.087 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 49 16 83152 83136 106.012 67.4375 0.0138989 0.00942745 2024-09-17T08:13:07.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:07.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:07.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:07.302 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:07.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:07.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:07.354 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:07.354 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:07.354 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[40649]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T08:13:07.355 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:07.355 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:07.355 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:07 smithi079 ceph-mon[29565]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T08:13:07.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:07.475 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:07.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:07.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:07.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:07.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:07.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:07 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:07 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:07 smithi173 ceph-mon[33187]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T08:13:07.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:07.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:07.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:07.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:07.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:07.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:08.011 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:08.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:08.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 50 16 84327 84311 105.361 73.4375 0.00755143 0.00948637 2024-09-17T08:13:08.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:08.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:08.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:08.291 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:08.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:08.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:08.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:08.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:08.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:08.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.559 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:08.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:08 smithi079 ceph-mon[29565]: pgmap v114: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 56 MiB/s wr, 895 op/s 2024-09-17T08:13:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:08 smithi079 ceph-mon[40649]: pgmap v114: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 56 MiB/s wr, 895 op/s 2024-09-17T08:13:08.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:08.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:08.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:08 smithi173 ceph-mon[33187]: pgmap v114: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 56 MiB/s wr, 895 op/s 2024-09-17T08:13:08.808 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:08.834 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:08.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:08.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:08.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:08.976 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:08.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:09.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:09.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 51 16 85517 85501 104.753 74.375 0.0104993 0.00954063 2024-09-17T08:13:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:09.134 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:09.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.142 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:09.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:09.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:09.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:09.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:09.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:09.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:09 smithi079 ceph-mon[40649]: pgmap v115: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-17T08:13:09.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:09 smithi079 ceph-mon[29565]: pgmap v115: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-17T08:13:09.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.575 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:09.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:09.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.715 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:09 smithi173 ceph-mon[33187]: pgmap v115: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-17T08:13:09.816 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:09.842 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:09.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:09.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:09.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:09.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:09.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:10.080 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:10.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 52 16 86644 86628 104.093 70.4375 0.0157864 0.00960165 2024-09-17T08:13:10.106 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:10.208 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:10.232 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:10.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:10.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.360 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:10.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:10.483 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:10.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:10.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:10.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:10.738 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:10.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:10.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1486776096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:10.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1486776096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:10 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:10.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:10.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:10.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1486776096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:10 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:10 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:10.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:11.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:11.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:11.024 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:13:11.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 53 16 87671 87655 103.34 64.1875 0.0160197 0.0096721 2024-09-17T08:13:11.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:11.143 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:11.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:11.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:11.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:11.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:11.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:11.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:11.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:11.513 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:11.539 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:11.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:11.604 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T08:13:11.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:11.680 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:11.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:11.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:11.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.815 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:11.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[29565]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T08:13:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/710985577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[29565]: pgmap v117: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 721 op/s 2024-09-17T08:13:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[40649]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T08:13:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/710985577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:11 smithi079 ceph-mon[40649]: pgmap v117: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 721 op/s 2024-09-17T08:13:11.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:11.942 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:11.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:11.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:11 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:11 smithi173 ceph-mon[33187]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T08:13:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:11 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/710985577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:11 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:13:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:11 smithi173 ceph-mon[33187]: pgmap v117: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 721 op/s 2024-09-17T08:13:12.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:12.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 54 16 88856 88840 102.798 74.0625 0.00845945 0.00972287 2024-09-17T08:13:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:12.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:12.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:12.226 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:12.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:12.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:12.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:12.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:12.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:12.479 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:12.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:12.580 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:12.608 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:12.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:12.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:12 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:13:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:12 smithi173 ceph-mon[33187]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T08:13:12.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:12.737 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:12.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:12.839 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:12.863 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:12.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:12.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:12.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:12 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:13:12.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:12 smithi079 ceph-mon[29565]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T08:13:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:12 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:13:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:12 smithi079 ceph-mon[40649]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T08:13:12.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:13.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:13.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:13.084 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 55 16 89940 89924 102.161 67.75 0.00981833 0.00978334 2024-09-17T08:13:13.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:13.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.150 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:13.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:13.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:13.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:13.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:13.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:13.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:13.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:13.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:13.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:13.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:13.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:13.683 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:13.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.684 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:13.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:13.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:13.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:13.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:13.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:13 smithi079 ceph-mon[40649]: pgmap v119: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:13:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:13 smithi079 ceph-mon[29565]: pgmap v119: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:13:13.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:13.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:13.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:13.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:13 smithi173 ceph-mon[33187]: pgmap v119: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:13:14.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:14.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 56 16 91085 91069 101.612 71.5625 0.0162568 0.00983611 2024-09-17T08:13:14.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.092 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:14.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:14.193 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:14.221 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:14.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:14.324 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:14.353 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:14.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:14.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:14.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.493 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:14.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:14.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:14.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:14.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:14.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:14.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.751 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:14.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:14.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:14.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:14.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:14.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:14.991 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:15.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.023 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:15.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 57 16 92231 92215 101.086 71.625 0.00657725 0.00988102 2024-09-17T08:13:15.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:15.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:15.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:15.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:15.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:15.421 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:15.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:15.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:15.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:15.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:15.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:15.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:15.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:15.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:15.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:15.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:15.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:16.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:16.088 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 58 16 93378 93362 100.578 71.6875 0.0058096 0.0099362 2024-09-17T08:13:16.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.090 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:16.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:16.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-17T08:13:16.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:16.216 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:16.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:16.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.343 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:16.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:16 smithi079 ceph-mon[40649]: pgmap v120: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:13:16.372 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:16 smithi079 ceph-mon[40649]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T08:13:16.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:16 smithi079 ceph-mon[29565]: pgmap v120: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:13:16.374 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:16 smithi079 ceph-mon[29565]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T08:13:16.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:13:16.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:16.469 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:16.571 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:16.595 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:16.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:16.723 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:16 smithi173 ceph-mon[33187]: pgmap v120: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:13:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:16 smithi173 ceph-mon[33187]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T08:13:16.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:16.822 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:13:16.822 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:13:16.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:16.847 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:16.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:16.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:16.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:16.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:16.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:17.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 59 16 94426 94410 99.9836 65.5 0.00435994 0.0099971 2024-09-17T08:13:17.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:17.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:17.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:17.261 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:17.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:17.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:17.406 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:17.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:17.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:17.532 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:17.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:17.624 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:17.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:17.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:17.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:17.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:17.810 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:17.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.812 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:17.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:13:17.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:13:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:13:17.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:13:17.958 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:13:17.959 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:13:17.967 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:17.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:17.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:17.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:18.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:18.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:13:18.087608+0000 min lat: 0.00156972 max lat: 0.80354 avg lat: 0.0100546 2024-09-17T08:13:18.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:13:18.087 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 60 16 95467 95451 99.4016 65.0625 0.0189602 0.0100546 2024-09-17T08:13:18.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:18.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:18.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:18.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:18.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:18.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:18.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:18.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:18.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:18.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:18.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:18.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:18 smithi173 ceph-mon[33187]: pgmap v122: 148 pgs: 148 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-17T08:13:18.594 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/591594635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:18.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:18.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:18 smithi079 ceph-mon[29565]: pgmap v122: 148 pgs: 148 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-17T08:13:18.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/591594635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:18.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:18 smithi079 ceph-mon[40649]: pgmap v122: 148 pgs: 148 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-17T08:13:18.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/591594635' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:18.643 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:18.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:18.745 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:18.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:18.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:18.822 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:18.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:18.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:18.929 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:18.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:19.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:19.078 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:19.086 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 61 16 96599 96583 98.9317 70.75 0.0177592 0.0101018 2024-09-17T08:13:19.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:19.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:19.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:19.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:19.370 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:19.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:19.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:19.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:19.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:19.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:19 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:19.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:19 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:19.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:19 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:19.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:19 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:19.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:19.646 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:19.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:19.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:19 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:19.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:19 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:19.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:19.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:19.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:19.809 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:19.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:19.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:19.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:19.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:20.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:20.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:20.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:20.087 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 62 16 97578 97562 98.3229 61.1875 0.0068245 0.0101651 2024-09-17T08:13:20.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:20.192 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:20.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:20.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:20.332 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:20.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:20.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:20.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:20.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:20.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:20 smithi079 ceph-mon[40649]: pgmap v123: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:13:20.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:20 smithi079 ceph-mon[40649]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T08:13:20.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:20 smithi079 ceph-mon[29565]: pgmap v123: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:13:20.533 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:20 smithi079 ceph-mon[29565]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T08:13:20.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:20.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:20.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:20 smithi173 ceph-mon[33187]: pgmap v123: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:13:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:20 smithi173 ceph-mon[33187]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T08:13:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:20.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:20.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:13:20.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:20.882 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:20.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:20.883 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:20.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:21.022 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:21.088 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 63 16 98513 98497 97.6878 58.4375 0.0102249 0.0102316 2024-09-17T08:13:21.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:21.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:21.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:21.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:21.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:21.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:21.600 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:21.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:21.727 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:21.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:21.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:21.868 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:21.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:21.970 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:21.998 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:21.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:21.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:22.088 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 64 16 99644 99628 97.2659 70.6875 0.0179143 0.010276 2024-09-17T08:13:22.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:22.130 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:22.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.131 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:22.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:22.266 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:22.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:22.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:22.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.415 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:22.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:22.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:22 smithi173 ceph-mon[33187]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T08:13:22.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:22 smithi173 ceph-mon[33187]: pgmap v126: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 939 op/s 2024-09-17T08:13:22.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:22.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:22 smithi079 ceph-mon[29565]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T08:13:22.506 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:22 smithi079 ceph-mon[29565]: pgmap v126: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 939 op/s 2024-09-17T08:13:22.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:22 smithi079 ceph-mon[40649]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T08:13:22.506 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:22 smithi079 ceph-mon[40649]: pgmap v126: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 939 op/s 2024-09-17T08:13:22.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:22.540 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:22.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:22.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:22.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:22.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:22.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:22.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:22.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:22.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:22.948 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:22.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:22.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:23.050 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:23.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:23.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 65 16 100711 100695 96.7954 66.6875 0.0174945 0.0103261 2024-09-17T08:13:23.179 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:23.204 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:23.217 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T08:13:23.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:23.341 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:23.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:23.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2390270454' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2390270454' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:23.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:23.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2390270454' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:13:23.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:23.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:23.583 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:23.608 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:23.614 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:13:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:13:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:13:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:13:23.615 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:13:23.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:23.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:23.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:23.867 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:23.969 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:23.995 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:23.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:23.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:24.088 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 66 16 101829 101813 96.3875 69.875 0.00774269 0.01037 2024-09-17T08:13:24.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:24.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.139 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:24.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:24.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:24.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:24.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:24.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:24.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:13:24.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[40649]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T08:13:24.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[40649]: pgmap v128: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-17T08:13:24.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:24.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:13:24.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[29565]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T08:13:24.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:24 smithi079 ceph-mon[29565]: pgmap v128: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-17T08:13:24.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:13:24.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:24.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.415 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:24.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:24.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:24.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:13:24.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:24 smithi173 ceph-mon[33187]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T08:13:24.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:24 smithi173 ceph-mon[33187]: pgmap v128: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-17T08:13:24.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:24.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:24.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:24.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:24.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:24.689 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:24.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:24.829 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:24.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:24.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:24.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:24.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:24.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:25.026 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:25.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-17T08:13:25.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:25.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 67 16 102934 102918 95.9791 69.0625 0.00734588 0.0104141 2024-09-17T08:13:25.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:25.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:25.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:25.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:25.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:25.371 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:25.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:25.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.398 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:25.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:25.571 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:25.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:25.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:25.724 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:25.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:25.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:25.849 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:25.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:25.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:25.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:26.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:26.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:26.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 68 16 103988 103972 95.5362 65.875 0.01906 0.0104621 2024-09-17T08:13:26.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:26.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:26.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:26.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:26.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:26.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:26.309 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:26.316 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[29565]: pgmap v129: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 875 op/s 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[29565]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[40649]: pgmap v129: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 875 op/s 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:13:26.327 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:26 smithi079 ceph-mon[40649]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T08:13:26.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:26.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:26.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:26.520 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:26.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:26.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:26.596 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:26 smithi173 ceph-mon[33187]: pgmap v129: 148 pgs: 148 active+clean; 6.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 875 op/s 2024-09-17T08:13:26.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:26 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:13:26.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:26 smithi173 ceph-mon[33187]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T08:13:26.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:26.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:26.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:26.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:26.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:26.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:26.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:26.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:27.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:27.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:27.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 69 16 105056 105040 95.1189 66.75 0.0101867 0.0105083 2024-09-17T08:13:27.109 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:27.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:27.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:27.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:27.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:27.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:27.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:27.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:27.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:27.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:27.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.551 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:27.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:27.686 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:27.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:27.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:27.824 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:27.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:27.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:27.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:27.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:27.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:28.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:28.083 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:28.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 70 15 106103 106088 94.6958 65.5 0.00545779 0.0105556 2024-09-17T08:13:28.186 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:28.208 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:28.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:28.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:28.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:28 smithi079 ceph-mon[29565]: pgmap v131: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 843 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-17T08:13:28.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:28.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:28.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:28.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:28.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:28.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:28 smithi079 ceph-mon[40649]: pgmap v131: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 843 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-17T08:13:28.665 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:28 smithi173 ceph-mon[33187]: pgmap v131: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 843 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-17T08:13:28.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:28.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:28.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.752 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:28.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:28.881 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:28.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:28.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:28.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:29.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.024 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:29.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:29.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 71 16 107021 107005 94.1693 57.3125 0.0713526 0.0106071 2024-09-17T08:13:29.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:29.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:29.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:29.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:29.304 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:29.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:29.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:29.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:29.545 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:29.669 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:29.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:29.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:29.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:29.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:29.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:13:29.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:13:29.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:29.911 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:13:29.911 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:13:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:29.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:29.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:29.936 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:13:29.937 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:13:29.969 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:13:29.970 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:13:30.002 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:13:30.002 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:13:30.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:30.032 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:13:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:13:30.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.059 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:30.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:30.065 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:13:30.065 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:13:30.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 72 15 108078 108063 93.7798 66.125 0.0181785 0.0106588 2024-09-17T08:13:30.141 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:13:30.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:13:30.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:30.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:30.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:30.316 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:30.418 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:30.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:30.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:30 smithi173 ceph-mon[33187]: pgmap v132: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:13:30.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:30 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2130713671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:30.521 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:30 smithi079 ceph-mon[29565]: pgmap v132: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:13:30.521 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2130713671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:30 smithi079 ceph-mon[40649]: pgmap v132: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-17T08:13:30.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:30 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2130713671' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:30.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:30.573 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:30.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:30.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:30.804 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:30.834 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:30.936 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:30.964 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:30.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:30.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:31.066 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:31.089 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 73 16 109157 109141 93.4181 67.375 0.00939277 0.0106999 2024-09-17T08:13:31.091 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:31.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:31.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:31.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:31.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:31.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:31.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:31.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:31.445 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:31 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:31.445 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:31 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:31.445 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:31 smithi173 ceph-mon[33187]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T08:13:31.445 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:31 smithi173 ceph-mon[33187]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T08:13:31.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:31.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:31.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:31.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[29565]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T08:13:31.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[29565]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T08:13:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[40649]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T08:13:31.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:31 smithi079 ceph-mon[40649]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T08:13:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:31.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:31.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:31.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:31.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:31.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:31.913 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:31.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:31.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:32.015 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:32.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:32.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:32.090 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 74 16 110214 110198 93.0484 66.0625 0.0240522 0.0107421 2024-09-17T08:13:32.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:32.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:32.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:32.274 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:32.298 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:32.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:32.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:32.444 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:32.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:32.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:32.581 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:32.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 900 op/s 2024-09-17T08:13:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2662682629' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 900 op/s 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2662682629' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:32 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:32.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: pgmap v135: 148 pgs: 148 active+clean; 6.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 900 op/s 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2662682629' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:32 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:13:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:32.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:32.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:32.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:32.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:32.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:33.005 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:33.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:33.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:13:33.091 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 75 16 111275 111259 92.6907 66.3125 0.00847436 0.010784 2024-09-17T08:13:33.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:33.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:33.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:33.216 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:13:33.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:33.312 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:33.313 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:33.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:33.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:33.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:33.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:33.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[29565]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:13:33.591 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:33 smithi079 ceph-mon[40649]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T08:13:33.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:33.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:33.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:33.723 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:33 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:13:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:13:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:33 smithi173 ceph-mon[33187]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T08:13:33.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:33.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:33.850 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:33.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:33.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:33.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:33.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:33.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:34.007 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:13:34.007 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:13:34.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:34.091 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 76 16 112256 112240 92.2769 61.3125 0.0123353 0.0108316 2024-09-17T08:13:34.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.119 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:34.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:34.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:34.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:34.358 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T08:13:34.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:34.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:34.537 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: pgmap v137: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-17T08:13:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4126966736' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: pgmap v137: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4126966736' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:34 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.639 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:34.666 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:34.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: pgmap v137: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-17T08:13:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:13:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:13:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4126966736' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:34 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:13:34.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:34.807 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:34.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:34.945 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:34.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:34.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:35.047 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:35.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:35.091 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 77 16 113357 113341 91.9722 68.8125 0.00889747 0.0108682 2024-09-17T08:13:35.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:35.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:35.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:35.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.340 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:35.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:35.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:35.466 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:35.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:35.593 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:35.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:35.720 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T08:13:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:35 smithi173 ceph-mon[33187]: pgmap v139: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:13:35.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:35.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.863 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:35.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:13:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[29565]: pgmap v139: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:13:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:13:35.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T08:13:35.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:35 smithi079 ceph-mon[40649]: pgmap v139: 148 pgs: 148 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:13:35.964 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:35.990 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:35.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:35.992 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:36.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 78 16 114527 114511 91.7305 73.125 0.0145127 0.0108969 2024-09-17T08:13:36.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:36.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:36.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:36.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:36.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:36.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:36.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:36.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:36.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:36.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:36.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:36.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:36.625 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:36.650 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:36.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:36.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:36.779 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:36.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:36.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:36.901 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:36.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:36.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:37.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:37.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:37.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:37.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 79 15 115630 115615 91.4428 69 0.0162401 0.010931 2024-09-17T08:13:37.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:37.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:37.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:37.300 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:37.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:37.427 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:37.530 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:37.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:37.657 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:37.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:37.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:37.717 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[40649]: pgmap v141: 148 pgs: 148 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T08:13:37.718 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:37 smithi079 ceph-mon[29565]: pgmap v141: 148 pgs: 148 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:13:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:13:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:13:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T08:13:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:37 smithi173 ceph-mon[33187]: pgmap v141: 148 pgs: 148 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:13:37.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:13:37.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:37.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.849 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:37.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:37.950 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:37.974 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:37.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:37.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:38.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:38.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:13:38.093145+0000 min lat: 0.00156972 max lat: 0.80354 avg lat: 0.0109722 2024-09-17T08:13:38.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:13:38.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 80 16 116655 116639 91.0997 64 0.0174406 0.0109722 2024-09-17T08:13:38.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:38.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:38.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:38.271 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:38.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:38.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:38.409 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:38.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:38.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-17T08:13:38.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:38.537 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:38.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:38.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:13:38.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:38.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:13:38.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:13:38.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:13:38.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:13:38.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:13:38.656 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:13:38.656 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:13:38.656 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:13:38.657 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:13:38.657 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:13:38.657 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:13:38.657 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:13:38.658 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:13:38.658 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:13:38.658 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:13:38.658 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:38 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:13:38.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:38.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:13:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:13:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:13:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:13:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:13:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:13:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:13:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:38 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:13:38.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:38.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:38.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:38.940 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:38.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:38.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:38.962 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:39.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:39.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:39.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:39.069 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:39.093 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 81 16 117646 117630 90.7396 61.9375 0.014483 0.0110162 2024-09-17T08:13:39.137 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:39.140 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:39.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:39.222 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:39.324 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:39.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:39.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:39.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:39.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:39.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:39.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:39.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.628 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:39.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:39.677 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:13:39.678 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:13:39.678 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[29565]: pgmap v142: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 845 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-17T08:13:39.678 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:13:39.678 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:13:39.678 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:39 smithi079 ceph-mon[40649]: pgmap v142: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 845 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-17T08:13:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:39 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:13:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:39 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:13:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:39 smithi173 ceph-mon[33187]: pgmap v142: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 845 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-17T08:13:39.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:39.763 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:39.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:39.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:39.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:39.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:39.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:40.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:40.040 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:40.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:40.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 82 16 118709 118693 90.4432 66.4375 0.0180356 0.0110517 2024-09-17T08:13:40.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:40.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:40.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:40.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:40.307 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:40.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:40.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:40.448 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:40.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:40.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:40.575 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:40.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:40.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:40.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.718 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:40.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:40 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:13:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:40 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:13:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:40 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:13:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:40 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:13:40.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:40.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:13:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:13:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:40 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:13:40.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:40.995 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:40.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:40.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:41.092 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 83 16 119857 119841 90.218 71.75 0.00857723 0.011079 2024-09-17T08:13:41.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:41.136 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:41.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:41.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:41.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:41.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:41.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:41.450 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:41.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:41.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:41.576 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:41.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[40649]: pgmap v144: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 740 B/s rd, 49 MiB/s wr, 786 op/s 2024-09-17T08:13:41.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:13:41.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:13:41.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:13:41.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:13:41.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T08:13:41.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:41 smithi079 ceph-mon[29565]: pgmap v144: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 740 B/s rd, 49 MiB/s wr, 786 op/s 2024-09-17T08:13:41.678 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:41.704 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:41.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:13:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:13:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:13:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:13:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T08:13:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:41 smithi173 ceph-mon[33187]: pgmap v144: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 740 B/s rd, 49 MiB/s wr, 786 op/s 2024-09-17T08:13:41.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:41.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:41.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:41.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:41.989 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:41.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:41.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:42.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:42.094 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 84 16 120952 120936 89.9584 68.4375 0.0142772 0.0111115 2024-09-17T08:13:42.138 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:42.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:42.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:42.275 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:42.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:42.377 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:42.405 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:42.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:42.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:42.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:42.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:42.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:42.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:42.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:42.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:42.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:42.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:42.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:42.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:42.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:42.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:43.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:43.094 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:43.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 85 16 122058 122042 89.7112 69.125 0.0186943 0.0111418 2024-09-17T08:13:43.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:43.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:43.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:43.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:43.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:13:43.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:13:43.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:13:43.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:13:43.281 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:43.281 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:13:43.281 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:13:43.281 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:13:43.281 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:43 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:13:43.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:43.390 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:43.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:43.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:43 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:13:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:43 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:13:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:43 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:13:43.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:43 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:13:43.492 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:43.518 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:43.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:43.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:43.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:43.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:43.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:43.771 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:43.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:43.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:43.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:43.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:43.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:43.981 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:13:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:13:44.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:44.025 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:13:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:13:44.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:44.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:44.056 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:13:44.056 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:13:44.085 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:13:44.085 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:13:44.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 86 16 123074 123058 89.4064 63.5 0.00981103 0.0111801 2024-09-17T08:13:44.122 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:13:44.122 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:13:44.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:44.154 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:13:44.154 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:13:44.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.173 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:44.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:44.194 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:13:44.194 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:13:44.229 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:13:44.229 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:13:44.240 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T08:13:44.275 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:44.298 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:44.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:44.400 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:44.435 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:44.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.436 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:44.537 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:44.565 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:44.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.566 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: pgmap v145: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:13:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/867439403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:44.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1073010690' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: pgmap v145: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:13:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/867439403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:44.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1073010690' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:44 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:44.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:44.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:13:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:13:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: pgmap v145: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:13:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:13:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:13:44.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/867439403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:13:44.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1073010690' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:44 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:13:44.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:44.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.839 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:44.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:44.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:44.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:44.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:44.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:45.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:45.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 87 15 124078 124063 89.1008 62.8125 0.0110165 0.0112182 2024-09-17T08:13:45.119 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:45.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:45.250 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:45.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.251 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:45.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:45.397 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:45.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:13:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[29565]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T08:13:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:13:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:13:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:13:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[40649]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T08:13:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:13:45.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:45 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:13:45.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:45.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:45 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:13:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:45 smithi173 ceph-mon[33187]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T08:13:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:45 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:13:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:45 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:13:45.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:45.807 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:45.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:45.948 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:45.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:45.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:46.050 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:46.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:46.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:46.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 88 16 125201 125185 88.8851 70.125 0.0117327 0.011246 2024-09-17T08:13:46.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:46.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:46.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:46.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:46.353 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:46.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:46.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:46.491 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:46.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:46.494 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:46 smithi079 ceph-mon[29565]: pgmap v147: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-17T08:13:46.494 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:46 smithi079 ceph-mon[29565]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T08:13:46.494 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:46 smithi079 ceph-mon[40649]: pgmap v147: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-17T08:13:46.494 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:46 smithi079 ceph-mon[40649]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T08:13:46.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:46.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.633 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:46.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:46 smithi173 ceph-mon[33187]: pgmap v147: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-17T08:13:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:46 smithi173 ceph-mon[33187]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T08:13:46.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:46.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:46.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.764 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:46.864 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:46.891 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:46.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:46.892 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:46.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:47.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:47.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:47.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 89 16 126291 126275 88.6518 68.125 0.00653085 0.0112744 2024-09-17T08:13:47.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:47.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:47.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.174 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:47.275 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:47.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:47.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:47.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:13:47.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:47.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:47.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:47.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:47.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:47.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:47.624 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:47 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:13:47.624 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:47 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:13:47.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:47 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:13:47.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:47 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:13:47.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:13:47.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:47.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:47.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:47.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:47.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:47.869 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:47.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:47.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:48.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:48.095 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total time run: 90.0094 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total writes made: 127397 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Write size: 65536 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object size: 65536 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Bandwidth (MB/sec): 88.4609 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Bandwidth: 34.4861 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max bandwidth (MB/sec): 212.938 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min bandwidth (MB/sec): 36.6875 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average IOPS: 1415 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev IOPS: 551.777 2024-09-17T08:13:48.096 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max IOPS: 3407 2024-09-17T08:13:48.097 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min IOPS: 587 2024-09-17T08:13:48.097 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average Latency(s): 0.0113004 2024-09-17T08:13:48.097 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Latency(s): 0.0114446 2024-09-17T08:13:48.097 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max latency(s): 0.80354 2024-09-17T08:13:48.097 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min latency(s): 0.00156972 2024-09-17T08:13:48.102 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T08:13:48.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:48.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:48.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:48.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-17T08:13:48.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:48.454 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:48.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:48.570 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: pgmap v149: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 843 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:13:48.571 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:48 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:13:48.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:48.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:13:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:13:48.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: pgmap v149: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 843 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: pgmap v149: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 843 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:13:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:48 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:13:48.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:48.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:48.825 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:48.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:13:48.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:48.883 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:48.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:48.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:49.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:49.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:49.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:49.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:49.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:49.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:49.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:49.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:49 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:13:49.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:49 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:13:49.433 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:49 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:13:49.433 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:49 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:13:49.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:49.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:49.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:49.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:49 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:13:49.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:49 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:13:49.476 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:13:49.476 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:13:49.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:128801 7.a e123/91)", 2024-09-17T08:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.470496+0000", 2024-09-17T08:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00090735399999999997, 2024-09-17T08:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093199400000000001, 2024-09-17T08:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:13:49.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470496+0000", 2024-09-17T08:13:49.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470496+0000", 2024-09-17T08:13:49.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470495+0000", 2024-09-17T08:13:49.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:49.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470498+0000", 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.847e-06 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470504+0000", 2024-09-17T08:13:49.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2570000000000004e-06 2024-09-17T08:13:49.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.470509+0000", 2024-09-17T08:13:49.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.464e-06 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.484 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:128801 7.a 7:54a8bd64:::benchmark_data_smithi079_83551_object1401:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-17T08:13:49.485 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.446242+0000", 2024-09-17T08:13:49.485 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.025160651999999999, 2024-09-17T08:13:49.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025250354999999999, 2024-09-17T08:13:49.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128801 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.486 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446242+0000", 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446242+0000", 2024-09-17T08:13:49.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446241+0000", 2024-09-17T08:13:49.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:49.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446247+0000", 2024-09-17T08:13:49.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.818e-06 2024-09-17T08:13:49.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446260+0000", 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3325e-05 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446267+0000", 2024-09-17T08:13:49.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3640000000000001e-06 2024-09-17T08:13:49.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:49.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446371+0000", 2024-09-17T08:13:49.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010380800000000001 2024-09-17T08:13:49.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:49.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446490+0000", 2024-09-17T08:13:49.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118715 2024-09-17T08:13:49.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.446613+0000", 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122772 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:128803 7.f e123/91 7:fa255260:::benchmark_data_smithi079_83551_object1403:head v 123'8128, mlcod=123'8126)", 2024-09-17T08:13:49.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.456150+0000", 2024-09-17T08:13:49.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.015252844999999999, 2024-09-17T08:13:49.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015420322, 2024-09-17T08:13:49.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:13:49.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456150+0000", 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456150+0000", 2024-09-17T08:13:49.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456150+0000", 2024-09-17T08:13:49.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:49.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456155+0000", 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456163+0000", 2024-09-17T08:13:49.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9019999999999999e-06 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456169+0000", 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7150000000000003e-06 2024-09-17T08:13:49.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.462995+0000", 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068267079999999999 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.463017+0000", 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1753e-05 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:128805 7.a 7.e5332a7a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-17T08:13:49.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.456221+0000", 2024-09-17T08:13:49.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.015181966, 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01540684, 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:49.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128805 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456221+0000", 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456221+0000", 2024-09-17T08:13:49.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456220+0000", 2024-09-17T08:13:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456224+0000", 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5059999999999997e-06 2024-09-17T08:13:49.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456234+0000", 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000003e-06 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456240+0000", 2024-09-17T08:13:49.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8699999999999997e-06 2024-09-17T08:13:49.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:128809 7.a 7.609db9ca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.456269+0000", 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.015133522999999999, 2024-09-17T08:13:49.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015409938999999999, 2024-09-17T08:13:49.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:13:49.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:13:49.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:13:49.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:13:49.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128809 2024-09-17T08:13:49.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456269+0000", 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456269+0000", 2024-09-17T08:13:49.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456269+0000", 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456271+0000", 2024-09-17T08:13:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4439999999999998e-06 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456277+0000", 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-17T08:13:49.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.456282+0000", 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5390000000000001e-06 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:128801 7.a e123/91)", 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:49.459179+0000", 2024-09-17T08:13:49.521 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.012224137, 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012547628999999999, 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459179+0000", 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:13:49.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459179+0000", 2024-09-17T08:13:49.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:13:49.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:13:49.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459178+0000", 2024-09-17T08:13:49.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:13:49.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459187+0000", 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0000000000000002e-06 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:13:49.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459198+0000", 2024-09-17T08:13:49.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1080000000000001e-05 2024-09-17T08:13:49.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:13:49.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:13:49.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:49.459205+0000", 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7819999999999996e-06 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:13:49.529 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:13:49.529 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 6 2024-09-17T08:13:49.529 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:13:49.561 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:49.589 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:49.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:49.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:49.715 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:49.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.716 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:49.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:49.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:49.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:49.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:49.987 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:49.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:49.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:50.088 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:50.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:50.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:50.233 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:50.258 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:50.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:50.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:50.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:50.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:50.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: pgmap v150: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2680009553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:50 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:50.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:50.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.511 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:50.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: pgmap v150: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:13:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2680009553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: pgmap v150: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2680009553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:50 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:13:50.640 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:13:50.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:50.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.645 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:50.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:50.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:50.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:50.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:50.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:50.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:50.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:51.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:51.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:51.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.025 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:51.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:51.151 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:51.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:51.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:51.291 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:51.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:51.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:51.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:51.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:51.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:51.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:51.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T08:13:51.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[29565]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T08:13:51.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:51 smithi079 ceph-mon[40649]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T08:13:51.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:51.693 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:51.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.695 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:13:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:13:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:13:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:13:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:13:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T08:13:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:51 smithi173 ceph-mon[33187]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T08:13:51.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:51.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:51.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:51.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:51.943 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:51.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:51.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:52.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:52.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:52.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:52.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:52.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:52.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:52.337 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:52.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.366 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:52.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:52.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:52.509 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:52.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:52.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[40649]: pgmap v153: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[29565]: pgmap v153: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:13:52.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:52 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:13:52.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:52.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:13:52.636 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:52.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:52.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:13:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:13:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:13:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-mon[33187]: pgmap v153: 148 pgs: 148 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-17T08:13:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:13:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:52 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:13:52.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:52.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:52.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:52.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:52.919 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:52.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:52.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:53.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:53.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:53.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:53.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:53.193 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:53.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:53.292 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:13:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:13:53.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:53.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:53.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:53.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:53.469 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:53.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:53.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:13:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:13:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[29565]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:13:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:53 smithi079 ceph-mon[40649]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T08:13:53.626 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:53.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:53 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:13:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:53 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:13:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:53 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:13:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:53 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:13:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:53 smithi173 ceph-mon[33187]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T08:13:53.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:53.737 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:53.752 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:53.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:53.753 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:53.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:53.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:53.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:53.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:54.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.020 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:54.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:54.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:54.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:54.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:54.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:54.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:54.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:54.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:54.450 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:54.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:54.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:54.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:13:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: pgmap v155: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 54 MiB/s wr, 1.60k op/s 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: pgmap v155: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 54 MiB/s wr, 1.60k op/s 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:13:54.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:54.716 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:54.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:13:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:13:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:13:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: pgmap v155: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 54 MiB/s wr, 1.60k op/s 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:13:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:13:54.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:54.845 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:54.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:54.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:54.973 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:54.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:54.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:55.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:55.100 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:55.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:55.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:13:55.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:55.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:55.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:55.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:55.368 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:55.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:55.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:55.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:55.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.495 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:55.596 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:55.600 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:13:55.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[29565]: pgmap v156: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 8.5 MiB/s wr, 877 op/s 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:13:55.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:55 smithi079 ceph-mon[40649]: pgmap v156: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 8.5 MiB/s wr, 877 op/s 2024-09-17T08:13:55.623 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:13:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:13:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:13:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:13:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:13:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:13:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:55 smithi173 ceph-mon[33187]: pgmap v156: 148 pgs: 148 active+clean; 7.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 8.5 MiB/s wr, 877 op/s 2024-09-17T08:13:55.725 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:55.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:55.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:55.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:55.893 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:55.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:55.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:55.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:56.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:56.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:56.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:56.170 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:56.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:56.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:56.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:56.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:56.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:56.435 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:56.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:56.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:56.587 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:56.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:56.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:13:56.637 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:13:56.638 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[40649]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T08:13:56.638 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:13:56.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:13:56.639 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:56 smithi079 ceph-mon[29565]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T08:13:56.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:56.700 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:13:56.701 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:13:56.701 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:13:56.701 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:13:56.701 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:13:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:56 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:13:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:56 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:13:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:56 smithi173 ceph-mon[33187]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T08:13:56.725 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:56.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:56.827 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:56.852 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:56.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:56.954 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:56.979 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:56.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:56.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:57.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:57.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:57.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:57.225 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:57.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:57.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:57.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:57.382 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:57.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:57.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[29565]: 2.0 deep-scrub starts 2024-09-17T08:13:57.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[29565]: 2.0 deep-scrub ok 2024-09-17T08:13:57.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:13:57.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:13:57.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[29565]: pgmap v158: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1012 B/s rd, 8.4 MiB/s wr, 1.05k op/s 2024-09-17T08:13:57.415 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[40649]: 2.0 deep-scrub starts 2024-09-17T08:13:57.415 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[40649]: 2.0 deep-scrub ok 2024-09-17T08:13:57.415 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:13:57.415 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:13:57.415 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:57 smithi079 ceph-mon[40649]: pgmap v158: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1012 B/s rd, 8.4 MiB/s wr, 1.05k op/s 2024-09-17T08:13:57.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:57.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:57.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:57.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:57.667 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:57.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:57 smithi173 ceph-mon[33187]: 2.0 deep-scrub starts 2024-09-17T08:13:57.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:57 smithi173 ceph-mon[33187]: 2.0 deep-scrub ok 2024-09-17T08:13:57.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:57 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:13:57.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:57 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:13:57.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:57 smithi173 ceph-mon[33187]: pgmap v158: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1012 B/s rd, 8.4 MiB/s wr, 1.05k op/s 2024-09-17T08:13:57.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:13:57.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:57.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:13:57.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:57.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:57.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:57.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:58.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:58.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:58.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:13:58.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:58.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:58.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:58.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:58.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:58.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.336 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:58.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:13:58.475 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:58.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:13:58.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:13:58.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:58.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:13:58.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:13:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:13:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:13:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:13:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 7.1 scrub starts 2024-09-17T08:13:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:13:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:58 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:13:58.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:58.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:58.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:13:58.868 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:58.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 7.1 scrub starts 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:13:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 7.1 scrub starts 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:13:58.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:58 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:13:58.970 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:13:58.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:58.994 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:58.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:13:59.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:59.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.131 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:59.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:59.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:59.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:59.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:59.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:59.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:13:59.566 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:13:59.572 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:59.600 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:13:59.601 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:13:59.643 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:13:59.644 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:13:59.644 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:13:59.644 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:13:59.644 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[29565]: pgmap v159: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:13:59.645 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:13:59.646 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:13:59 smithi079 ceph-mon[40649]: pgmap v159: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-17T08:13:59.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:13:59.680 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:13:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:13:59.712 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:13:59.712 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:13:59.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:13:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:13:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:13:59 smithi173 ceph-mon[33187]: pgmap v159: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-17T08:13:59.742 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:13:59.742 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:13:59.771 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:13:59.771 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:13:59.803 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:13:59.804 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:13:59.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:13:59.858 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:13:59.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:13:59.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:13:59.985 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:13:59.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:13:59.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:00.087 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:00.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:00.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:00.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:00.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:00.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:00.358 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:00.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:00.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: 7.1 scrub ok 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3875848094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:14:00.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:00 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:14:00.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:00.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:00.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:00.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: 7.1 scrub ok 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3875848094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: 7.1 scrub ok 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:14:00.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:14:00.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3875848094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:00.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:14:00.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:00 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:14:00.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:00.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.661 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:00.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:00.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:00.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:00.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:00.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:00.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:00.922 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:00.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:01.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:01.062 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:01.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:01.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:01.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.207 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:01.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:01.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:01.319 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:01.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:01.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:01.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:14:01.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:01.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:01.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:01.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:01.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:01.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:01.685 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:14:01.685 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:14:01.685 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[29565]: 7.6 scrub starts 2024-09-17T08:14:01.685 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[29565]: pgmap v160: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 572 B/s rd, 3.8 MiB/s wr, 1.05k op/s 2024-09-17T08:14:01.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:14:01.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:14:01.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[40649]: 7.6 scrub starts 2024-09-17T08:14:01.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:01 smithi079 ceph-mon[40649]: pgmap v160: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 572 B/s rd, 3.8 MiB/s wr, 1.05k op/s 2024-09-17T08:14:01.703 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:01 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:14:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:01 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:14:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:01 smithi173 ceph-mon[33187]: 7.6 scrub starts 2024-09-17T08:14:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:01 smithi173 ceph-mon[33187]: pgmap v160: 148 pgs: 148 active+clean; 7.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 572 B/s rd, 3.8 MiB/s wr, 1.05k op/s 2024-09-17T08:14:01.727 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:01.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:01.829 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:01.853 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:01.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:01.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:01.993 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:01.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:01.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:02.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:02.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:02.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:02.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:02.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:02.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:02.383 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:02.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:02.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:02.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:02.526 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:02.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.564 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:02.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:02.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:02.702 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:02.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:02.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:14:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:14:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: 5.1d deep-scrub starts 2024-09-17T08:14:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: 5.1d deep-scrub ok 2024-09-17T08:14:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:02 smithi173 ceph-mon[33187]: 7.4 scrub starts 2024-09-17T08:14:02.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:02.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: 5.1d deep-scrub starts 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: 5.1d deep-scrub ok 2024-09-17T08:14:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[40649]: 7.4 scrub starts 2024-09-17T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: 5.1d deep-scrub starts 2024-09-17T08:14:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: 5.1d deep-scrub ok 2024-09-17T08:14:02.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:02 smithi079 ceph-mon[29565]: 7.4 scrub starts 2024-09-17T08:14:02.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:02.984 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:02.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:02.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:03.086 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:03.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:03.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:14:03.212 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:03.234 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:03.240 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:03.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:03.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:03.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:03.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:03.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:03.625 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:03.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:03.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:03.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:03.867 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:03.867 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:03 smithi079 ceph-mon[29565]: pgmap v161: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:03.867 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:03.868 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:03 smithi079 ceph-mon[40649]: pgmap v161: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:03.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:03.900 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:03.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:03.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:03 smithi173 ceph-mon[33187]: pgmap v161: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:04.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:04.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:04.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:04.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:04.163 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:04.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:04.265 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:04.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.291 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:04.291 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:04.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:04.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:04.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:04.518 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T08:14:04.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:04.556 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:04.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:04.658 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:04.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:04.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:04.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:04.827 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:04.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 2.2 deep-scrub starts 2024-09-17T08:14:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 2.2 deep-scrub ok 2024-09-17T08:14:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 7.6 scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 2.2 deep-scrub starts 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 2.2 deep-scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 7.6 scrub ok 2024-09-17T08:14:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:14:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:14:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:04 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:14:04.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:04.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:04.952 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:04.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 2.2 deep-scrub starts 2024-09-17T08:14:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 2.2 deep-scrub ok 2024-09-17T08:14:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:14:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:14:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 7.6 scrub ok 2024-09-17T08:14:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:14:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:14:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:04 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:14:05.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:05.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:05.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:05.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:05.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:05.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:05.361 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:05.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:05.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:05.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:05.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:05.534 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:14:05.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:05.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:05.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.645 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:05.745 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:05.771 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:05.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:05.852 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 7.4 scrub ok 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 7.8 scrub starts 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3941047547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[40649]: pgmap v163: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 7.4 scrub ok 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 7.8 scrub starts 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:14:05.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:14:05.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3941047547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:05 smithi079 ceph-mon[29565]: pgmap v163: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-17T08:14:05.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:05.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-17T08:14:05.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:05.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:05.899 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 7.4 scrub ok 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2151493033' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 7.8 scrub starts 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3941047547' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:05 smithi173 ceph-mon[33187]: pgmap v163: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-17T08:14:05.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:06.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:06.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:06.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:06.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:06.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:06.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:06.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:06.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:06.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:06.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.424 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:06.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:06.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:06.558 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:06.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:06.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:06.691 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:06.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:06.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:06.817 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:06.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: 7.7 scrub starts 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: 7.0 deep-scrub starts 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[29565]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: 7.7 scrub starts 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: 7.0 deep-scrub starts 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:06 smithi079 ceph-mon[40649]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T08:14:06.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:06.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:06.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:06.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:06.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: 7.7 scrub starts 2024-09-17T08:14:06.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: 7.0 deep-scrub starts 2024-09-17T08:14:06.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:14:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:14:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:06 smithi173 ceph-mon[33187]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T08:14:07.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:07.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:07.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:07.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:07.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:07.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:07.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:07.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:07.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:07.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:07.501 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:07.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:07.606 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:07.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:07.680 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:07 smithi079 ceph-mon[29565]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 795 op/s 2024-09-17T08:14:07.680 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:07 smithi079 ceph-mon[40649]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 795 op/s 2024-09-17T08:14:07.736 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:07.762 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:07.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:07.863 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:07.891 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:07.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:07.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:07.897 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:07 smithi173 ceph-mon[33187]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 795 op/s 2024-09-17T08:14:07.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:08.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:08.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:08.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:08.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:08.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:08.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:08.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:08.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:08.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:08.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:08.595 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:08 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:08.595 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:08 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:08.597 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:08 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:08.597 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:08 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:08.609 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:08.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:08.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:08.754 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:08.755 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:08.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:08.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:08.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.855 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:08.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:08.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:08 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:08.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:08 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:08.992 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:08.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:08.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:09.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:09.119 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:09.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:09.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:09.261 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:09.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:09.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:09.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:09.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:09.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:09.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.526 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:09.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:09.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:09.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.666 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:09.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:09.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:09.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:09.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[29565]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T08:14:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[29565]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[40649]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T08:14:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:09 smithi079 ceph-mon[40649]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:09.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:09.926 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:09.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:09.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:09.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:14:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:09 smithi173 ceph-mon[33187]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T08:14:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:09 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:09 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:09 smithi173 ceph-mon[33187]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:10.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:10.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:10.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:10.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:10.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:10.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:10.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:10.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:10.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:10.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:10.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:10.570 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:10.681 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:10.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:10.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:10.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:10.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:10 smithi079 ceph-mon[29565]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T08:14:10.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:10 smithi079 ceph-mon[40649]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T08:14:10.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:10.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:10.937 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:10.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:10 smithi173 ceph-mon[33187]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T08:14:11.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:11.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:11.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:11.178 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:11.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:11.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:11.303 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:11.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:11.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:11.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:11.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:11.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:11.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:11.600 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:11.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:11.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:11.739 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:11.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:11.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:11.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:11.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:11.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:11.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:12.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:12.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:12.109 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:12.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.137 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:12.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:12.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:12.364 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:12.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:12.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:12.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:12 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:12.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:12 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:12.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:12 smithi173 ceph-mon[33187]: pgmap v169: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:12.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:12 smithi173 ceph-mon[33187]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T08:14:12.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:12.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:12.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:12.612 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[29565]: pgmap v169: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[29565]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T08:14:12.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[40649]: pgmap v169: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s 2024-09-17T08:14:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:12 smithi079 ceph-mon[40649]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T08:14:12.638 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:12.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:12.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:12.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:12.888 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:12.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:12.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:12.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:13.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.027 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:13.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:13.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:13.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:13.302 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:13.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:13.438 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:13.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:13.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.580 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:13.681 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:13.709 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:13.810 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:13.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:13.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:13.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:13.971 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:13.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:13.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:14.073 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:14.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:14.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:14.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:14.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:14.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:14.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:14.365 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:14.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:14.467 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:14.491 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:14.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.492 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:14.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:14.593 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:14 smithi079 ceph-mon[29565]: pgmap v171: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:14:14.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:14.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:14 smithi079 ceph-mon[40649]: pgmap v171: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:14:14.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:14.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:14:14.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:14.620 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:14 smithi173 ceph-mon[33187]: pgmap v171: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:14:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:14.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:14.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:14.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:14.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:14.861 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:14.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:14.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:14.939 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:14.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:15.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:15.072 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:15.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:15.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:15.204 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:15.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:15.306 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:15.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:15.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:15.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:15.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:15.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:15.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:15.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:15.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:15.651 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:15.703 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:14:15.703 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:14:15.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:15.734 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:14:15.735 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:14:15.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:15.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:15.768 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:14:15.768 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:14:15.797 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:14:15.797 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:14:15.830 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:14:15.831 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:14:15.846 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:15.859 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:14:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:14:15.870 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:15.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:15.890 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:14:15.890 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:14:15.921 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:14:15.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:14:15.971 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:15.995 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:15.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:15.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:16.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:16.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:16.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:16.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:16.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.277 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:16.277 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:16.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:16.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:16.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:16.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:16 smithi079 ceph-mon[40649]: pgmap v172: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 762 B/s rd, 0 B/s wr, 689 op/s 2024-09-17T08:14:16.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3785818556' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:16.433 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:16 smithi079 ceph-mon[29565]: pgmap v172: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 762 B/s rd, 0 B/s wr, 689 op/s 2024-09-17T08:14:16.433 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3785818556' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:16 smithi173 ceph-mon[33187]: pgmap v172: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 762 B/s rd, 0 B/s wr, 689 op/s 2024-09-17T08:14:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3785818556' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:16.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:16.566 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:16.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:16.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:16.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:16.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:16.796 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:16.817 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:16.818 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:14:16.818 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:14:16.818 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:16.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:16.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.822 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:16.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:16.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:16.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:16.969 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:17.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:17.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:17.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:17.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:17.226 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:17.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:17.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:17.365 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:17.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:17.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:17.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:17.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:17.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:17.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:17.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:17.737 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:17.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:17.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[40649]: pgmap v173: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 654 op/s 2024-09-17T08:14:17.738 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:17.738 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:17.738 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:17 smithi079 ceph-mon[29565]: pgmap v173: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 654 op/s 2024-09-17T08:14:17.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:17.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:17.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:17.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:17.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:17.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:17.899 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:17 smithi173 ceph-mon[33187]: pgmap v173: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 654 op/s 2024-09-17T08:14:18.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:18.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:18.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:18.135 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:14:18.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:18.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:18.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:18.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:18.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.347 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:18.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:18.412 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:18.447 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:18.478 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:18.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:18.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:18.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.621 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:18.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:18.722 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:18.752 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:18.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:18.807 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T08:14:18.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:18.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:18.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:18.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:18.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:19.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.021 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:19.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:14:19.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:14:19.122 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:19.150 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:19.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:19.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:14:19.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:19.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.285 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:19.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:19.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:19.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:19.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:19.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:14:19.520 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:19.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:19.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:19.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-17T08:14:19.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:19.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:19.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:19.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:19.847 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:19.851 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:14:19.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:19.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:19.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:20.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:20.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:14:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[29565]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T08:14:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/174157027' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[29565]: pgmap v175: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[40649]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/174157027' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:19 smithi079 ceph-mon[40649]: pgmap v175: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:14:20.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3319174439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:14:20.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:19 smithi173 ceph-mon[33187]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T08:14:20.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/174157027' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:19 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:20.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:19 smithi173 ceph-mon[33187]: pgmap v175: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:14:20.144 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:20.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:20.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:20.374 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:20.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.402 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.402 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:20.415 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:20.415 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:14:20.415 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:14:20.415 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:14:20.415 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:20.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:20.543 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:20.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:20.644 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:20.669 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:20.682 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:20.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:20.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:20.844 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:20.844 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:20.844 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[29565]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T08:14:20.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:20.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:20.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:20.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:21.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:21.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:21.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:21.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:21.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:21.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:20 smithi079 ceph-mon[40649]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T08:14:21.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:21.199 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:21.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:21.217 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:20 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:21.218 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:20 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:21.218 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:20 smithi173 ceph-mon[33187]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T08:14:21.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:14:21.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:21.340 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:21.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:21.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:21.476 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:21.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:21.499 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:21.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:21.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:21.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:21.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:21.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.740 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:21.740 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:21.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:21.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:21.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:21.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:21.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:21.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:21.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:22.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:22.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:22.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:22.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:22.244 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:22.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:22.255 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:22 smithi173 ceph-mon[33187]: pgmap v177: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-09-17T08:14:22.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:22.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:22.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:22.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:22.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:22.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:22.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:22 smithi079 ceph-mon[29565]: pgmap v177: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-09-17T08:14:22.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:22 smithi079 ceph-mon[40649]: pgmap v177: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 616 op/s 2024-09-17T08:14:22.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:22.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:14:22.635 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:22.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:22.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:22.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:22.824 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:22.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:22.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:22.952 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:22.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:22.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:23.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:23.094 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:23.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:23.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:23.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:23.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:23.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:23.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:23.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:23.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:23.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:23.490 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:23.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:23.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:23.635 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:23.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:23.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:23.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:23.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:23.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:23.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:23.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:23.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:23.955 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:23.955 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:14:23.955 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:14:23.955 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28708 7.1 7:8a29e2d7:::benchmark_data_smithi079_83551_object28707:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.972520+0000", 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 114.97905705700001, 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.062815255, 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.956 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28708 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972520+0000", 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972520+0000", 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.957 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972518+0000", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972599+0000", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0524000000000004e-05 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972611+0000", 2024-09-17T08:14:23.958 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2262000000000001e-05 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972618+0000", 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7320000000000001e-06 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972667+0000", 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9262999999999999e-05 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.959 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972766+0000", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9462000000000004e-05 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.972890+0000", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012331399999999999 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.973974+0000", 2024-09-17T08:14:23.960 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010844419999999999 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.974353+0000", 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00037911800000000001 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.035118+0000", 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.060765302 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.961 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.035294+0000", 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000175811 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.035335+0000", 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0818000000000001e-05 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.962 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30828 7.1 7:814ad74a:::benchmark_data_smithi079_83551_object30827:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 113.921908448, 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14898619199999999, 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30828 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.964 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029666+0000", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029758+0000", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1730000000000004e-05 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029772+0000", 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3888e-05 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.965 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029779+0000", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0060000000000003e-06 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029822+0000", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2682999999999998e-05 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029949+0000", 2024-09-17T08:14:23.966 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127482 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.030066+0000", 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117027 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.032345+0000", 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022790940000000002 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.967 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035973+0000", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003627791 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178565+0000", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14259147699999999 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2923999999999997e-05 2024-09-17T08:14:23.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178655+0000", 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7166000000000003e-05 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30837 7.1 7:84e0e382:::benchmark_data_smithi079_83551_object30836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 113.914380124, 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14170434200000001, 2024-09-17T08:14:23.969 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30837 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037195+0000", 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:23.971 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037307+0000", 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111345 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037318+0000", 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1612e-05 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037326+0000", 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4959999999999999e-06 2024-09-17T08:14:23.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037356+0000", 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0233e-05 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037437+0000", 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0385999999999999e-05 2024-09-17T08:14:23.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037519+0000", 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2231000000000006e-05 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.038523+0000", 2024-09-17T08:14:23.974 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001004158 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.039311+0000", 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078787300000000002 2024-09-17T08:14:23.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178816+0000", 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13950497000000001 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178860+0000", 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4440000000000001e-05 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.976 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178901+0000", 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0961999999999998e-05 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:32611 7.1 7:8f98a80c:::benchmark_data_smithi079_83551_object32610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.973193+0000", 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 112.97838351999999, 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.066614121999999998, 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.977 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 32611 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973193+0000", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:23.978 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973193+0000", 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973192+0000", 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973270+0000", 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7751999999999997e-05 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.979 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973283+0000", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3458000000000001e-05 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973290+0000", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9879999999999998e-06 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973331+0000", 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0979999999999997e-05 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.980 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973460+0000", 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012912199999999999 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973582+0000", 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121395 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:23.981 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.974880+0000", 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012987350000000001 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.975087+0000", 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000206443 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.982 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.039731+0000", 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.064643987 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.039768+0000", 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7558000000000002e-05 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.039808+0000", 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9295999999999998e-05 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.983 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:32618 7.1 7:89a79323:::benchmark_data_smithi079_83551_object32617:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.976000+0000", 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 112.975576687, 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.068773713, 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.984 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 32618 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976000+0000", 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976000+0000", 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.985 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.986 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:23.986 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.975999+0000", 2024-09-17T08:14:23.986 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:14:23.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976081+0000", 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2207999999999996e-05 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.988 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976094+0000", 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2741e-05 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.989 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976099+0000", 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0069999999999997e-06 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976132+0000", 2024-09-17T08:14:23.990 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3769999999999997e-05 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976273+0000", 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000140252 2024-09-17T08:14:23.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.976417+0000", 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014447900000000001 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.992 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.977600+0000", 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011832279999999999 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.977768+0000", 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016745799999999999 2024-09-17T08:14:23.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.044641+0000", 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.066873574000000005 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.994 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.044689+0000", 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8006000000000001e-05 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.044774+0000", 2024-09-17T08:14:23.995 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4670000000000001e-05 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67130 7.1 7:868286e5:::benchmark_data_smithi079_83551_object67129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:23.997 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:14:23.997 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 90.404920427999997, 2024-09-17T08:14:23.997 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099432551999999993, 2024-09-17T08:14:23.997 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:23.997 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:23.998 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:23.998 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:23.998 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:23.998 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67130 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:23.999 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546655+0000", 2024-09-17T08:14:24.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546716+0000", 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1740000000000002e-05 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546732+0000", 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6005999999999999e-05 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546741+0000", 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0229999999999992e-06 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546792+0000", 2024-09-17T08:14:24.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1205000000000002e-05 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546929+0000", 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137167 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.547027+0000", 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8185999999999998e-05 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.548900+0000", 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018727170000000001 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.549110+0000", 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021050600000000001 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.645997+0000", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096886776999999993 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.646051+0000", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3665000000000002e-05 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.646089+0000", 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8324000000000001e-05 2024-09-17T08:14:24.006 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67361 7.1 7:8095d638:::benchmark_data_smithi079_83551_object67360:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 90.155590098000005, 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.084037299999999995, 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.007 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67361 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795986+0000", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796036+0000", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0630999999999997e-05 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796046+0000", 2024-09-17T08:14:24.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0249e-05 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796052+0000", 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2689999999999999e-06 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796097+0000", 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5562999999999997e-05 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796237+0000", 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000139307 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796396+0000", 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000159145 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800418+0000", 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040222030000000002 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800445+0000", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7274e-05 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.879940+0000", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.079494422999999995 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.879985+0000", 2024-09-17T08:14:24.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5290999999999997e-05 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880024+0000", 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9240999999999999e-05 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67364 7.1 7:8e7f7c18:::benchmark_data_smithi079_83551_object67363:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:14:24.013 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 90.155279610999997, 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.083829862000000005, 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67364 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:14:24.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796366+0000", 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9509000000000001e-05 2024-09-17T08:14:24.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796372+0000", 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0480000000000004e-06 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796376+0000", 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1250000000000003e-06 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798382+0000", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002005976 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798502+0000", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119765 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798645+0000", 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014252899999999999 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800474+0000", 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018297389999999999 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800512+0000", 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7613000000000001e-05 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880060+0000", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.079547962 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880098+0000", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7691999999999998e-05 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880127+0000", 2024-09-17T08:14:24.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9541999999999999e-05 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68437 7.1 7:8794a2e1:::benchmark_data_smithi079_83551_object68436:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 89.189554111000007, 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11749947199999999, 2024-09-17T08:14:24.021 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68437 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762022+0000", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762153+0000", 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131729 2024-09-17T08:14:24.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762168+0000", 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4851999999999999e-05 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762174+0000", 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6439999999999997e-06 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762209+0000", 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5030999999999997e-05 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762299+0000", 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0492e-05 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762395+0000", 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5126999999999999e-05 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764253+0000", 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018581089999999999 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764600+0000", 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034766899999999999 2024-09-17T08:14:24.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879354+0000", 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11475331499999999 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879444+0000", 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0832e-05 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879522+0000", 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7851999999999999e-05 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68447 7.1 7:81e8cead:::benchmark_data_smithi079_83551_object68446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 89.183394157999999, 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.116018159, 2024-09-17T08:14:24.028 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68447 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768181+0000", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768276+0000", 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5210999999999994e-05 2024-09-17T08:14:24.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768289+0000", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2994e-05 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768296+0000", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8510000000000001e-06 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768336+0000", 2024-09-17T08:14:24.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9638999999999998e-05 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768469+0000", 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132881 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768636+0000", 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016691399999999999 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771127+0000", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002491643 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771774+0000", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064712199999999998 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884106+0000", 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11233185 2024-09-17T08:14:24.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884160+0000", 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3644999999999999e-05 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884201+0000", 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1152000000000002e-05 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68453 7.1 7:847278dd:::benchmark_data_smithi079_83551_object68452:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 89.179645492000006, 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.115527932, 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68453 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771930+0000", 2024-09-17T08:14:24.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772008+0000", 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7849999999999995e-05 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772019+0000", 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1488000000000001e-05 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772025+0000", 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1270000000000002e-06 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772067+0000", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2552000000000002e-05 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772178+0000", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110838 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772272+0000", 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3548999999999995e-05 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774206+0000", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001934727 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774634+0000", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00042775400000000002 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887387+0000", 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.112753373 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887423+0000", 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5494999999999998e-05 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887459+0000", 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6498999999999997e-05 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:105993 7.1 7:82850e8f:::benchmark_data_smithi079_83551_object105992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:27.957339+0000", 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 55.994237888999997, 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.064103973999999994, 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 105993 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957339+0000", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957339+0000", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957338+0000", 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957380+0000", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1965999999999998e-05 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957391+0000", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1168e-05 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957396+0000", 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4399999999999996e-06 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957429+0000", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2638000000000002e-05 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957541+0000", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011198 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.957654+0000", 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112608 2024-09-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.963224+0000", 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055703030000000004 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.964056+0000", 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00083171099999999995 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021353+0000", 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057297019999999997 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021403+0000", 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0543e-05 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021443+0000", 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9854000000000002e-05 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:105995 7.1 7:82a82945:::benchmark_data_smithi079_83551_object105994:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:27.960151+0000", 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 55.991425503000002, 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.06142852, 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 105995 2024-09-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960151+0000", 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960151+0000", 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960150+0000", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960206+0000", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6399e-05 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960219+0000", 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2692e-05 2024-09-17T08:14:24.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960227+0000", 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0239999999999997e-06 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960281+0000", 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4382000000000002e-05 2024-09-17T08:14:24.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960373+0000", 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1452999999999993e-05 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.960515+0000", 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014249400000000001 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.964299+0000", 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037839470000000002 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:27.964560+0000", 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026078199999999998 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021498+0000", 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.056937660000000001 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021544+0000", 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5750000000000001e-05 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.021580+0000", 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6426e-05 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.057 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:106513 7.1 7:8d652158:::benchmark_data_smithi079_83551_object106512:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-17T08:14:24.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:24.059 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:28.525339+0000", 2024-09-17T08:14:24.059 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 55.426237725999997, 2024-09-17T08:14:24.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.061647304, 2024-09-17T08:14:24.059 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.059 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106513 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525339+0000", 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525339+0000", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525338+0000", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525378+0000", 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9878e-05 2024-09-17T08:14:24.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525392+0000", 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4277000000000001e-05 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525399+0000", 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.401e-06 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525435+0000", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5797000000000003e-05 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525542+0000", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000107073 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.525655+0000", 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011274599999999999 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.528624+0000", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029688900000000001 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.529274+0000", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00065046799999999997 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.586890+0000", 2024-09-17T08:14:24.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057615619999999999 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.586946+0000", 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6365e-05 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:28.586987+0000", 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0339999999999997e-05 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:107873 7.1 7:80bb83e2:::benchmark_data_smithi079_83551_object107872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:29.882849+0000", 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 54.068728129999997, 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.045909718000000002, 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 107873 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.066 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882849+0000", 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882849+0000", 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882848+0000", 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882901+0000", 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3529999999999997e-05 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882912+0000", 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0642000000000001e-05 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882917+0000", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8080000000000004e-06 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.882946+0000", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9357999999999999e-05 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.883035+0000", 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8751000000000005e-05 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.914961+0000", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.031926726000000002 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.921464+0000", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0065021849999999997 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.927854+0000", 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063905890000000003 2024-09-17T08:14:24.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.928637+0000", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078255700000000002 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.928718+0000", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1086000000000003e-05 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:29.928759+0000", 2024-09-17T08:14:24.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0664999999999998e-05 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:129742 7.1 7:8ce1dc21:::benchmark_data_smithi079_83551_object2342:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:50.440285+0000", 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.511291743999998, 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.066683869000000007, 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.072 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 129742 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440285+0000", 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440285+0000", 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440284+0000", 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440291+0000", 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1169999999999998e-06 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440305+0000", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4721e-05 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440316+0000", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0341e-05 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440358+0000", 2024-09-17T08:14:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2205e-05 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440491+0000", 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132726 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.440607+0000", 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116338 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.442794+0000", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021865510000000001 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.443148+0000", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035432399999999999 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.506912+0000", 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.063763696999999994 2024-09-17T08:14:24.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.506942+0000", 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0658999999999999e-05 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:50.506969+0000", 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6716999999999998e-05 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:131327 7.1 7:800692b8:::benchmark_data_smithi079_83551_object3927:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:51.901605+0000", 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.049971523000004, 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.056981927000000002, 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 131327 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.079 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901605+0000", 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901605+0000", 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901604+0000", 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901610+0000", 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2820000000000002e-06 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901624+0000", 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3232e-05 2024-09-17T08:14:24.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.901630+0000", 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8070000000000002e-06 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.902641+0000", 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001010941 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.902734+0000", 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2596000000000006e-05 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.902820+0000", 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5978999999999998e-05 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.904747+0000", 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001927488 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.905074+0000", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032672199999999999 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958495+0000", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.053420678999999999 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958546+0000", 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1557000000000001e-05 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958587+0000", 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0955999999999999e-05 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:131334 7.1 7:8a881ac9:::benchmark_data_smithi079_83551_object3934:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:51.906244+0000", 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.045333196000001, 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.052439939999999997, 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 131334 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906244+0000", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906244+0000", 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906242+0000", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906248+0000", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7740000000000001e-06 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906317+0000", 2024-09-17T08:14:24.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8833999999999999e-05 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906324+0000", 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6329999999999999e-06 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906394+0000", 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0131999999999999e-05 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906513+0000", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119397 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.906641+0000", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000128212 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909294+0000", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026527819999999998 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909401+0000", 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010676200000000001 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958634+0000", 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.049232947999999999 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958663+0000", 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8895e-05 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958684+0000", 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1002e-05 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:131342 7.1 7:80a62c4c:::benchmark_data_smithi079_83551_object3942:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:51.909517+0000", 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.042059492999996, 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.049283345999999999, 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.091 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 131342 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909517+0000", 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909517+0000", 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909517+0000", 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909521+0000", 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3100000000000002e-06 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909529+0000", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3210000000000005e-06 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909534+0000", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9019999999999998e-06 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909565+0000", 2024-09-17T08:14:24.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0926999999999997e-05 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909671+0000", 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010634 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.909784+0000", 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011240500000000001 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.912190+0000", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024060420000000002 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.912418+0000", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000227865 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958724+0000", 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.046306121999999998 2024-09-17T08:14:24.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958763+0000", 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9634e-05 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:51.958801+0000", 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7321999999999999e-05 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:138557 7.1 7:8b1ebd74:::benchmark_data_smithi079_83551_object11157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 25.757637016, 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.074466521999999993, 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 138557 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:24.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193938+0000", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193946+0000", 2024-09-17T08:14:24.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8720000000000002e-06 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193959+0000", 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2288e-05 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193966+0000", 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6899999999999992e-06 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:24.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198079+0000", 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041130380000000003 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198163+0000", 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4127000000000003e-05 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198250+0000", 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6661000000000003e-05 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.200929+0000", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026791829999999999 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.201219+0000", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028933099999999998 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268334+0000", 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067115887999999999 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268372+0000", 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7580999999999998e-05 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268406+0000", 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4388999999999999e-05 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:24.103 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:24.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.108 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:24.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:24.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:24.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.251 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:24.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:24.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:24.375 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:24.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:24.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:24.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:24.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:24 smithi079 ceph-mon[40649]: pgmap v178: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:14:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:24 smithi079 ceph-mon[29565]: pgmap v178: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:14:24.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:24.660 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:24.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:24 smithi173 ceph-mon[33187]: pgmap v178: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:14:24.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:24.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:24.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.793 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:24.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:24.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:24.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:24.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:24.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:14:25.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:25.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:25.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:25.122 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:14:25.122 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:14:25.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:25.160 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:14:25.160 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:14:25.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:25.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:25.194 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:14:25.195 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:14:25.221 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:14:25.221 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:14:25.236 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:25.254 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:14:25.254 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:14:25.283 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:14:25.284 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:14:25.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:25.318 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:14:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:14:25.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:25.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:25.353 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:14:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:14:25.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:25.462 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:25.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:25.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:25.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:25.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:25.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3954195710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:25.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3954195710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:25.653 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3954195710' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:25.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:25.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:25.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:25.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:25.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:25.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:25.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:25.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:26.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:26.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:26.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:26.142 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:26.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:26.244 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:26.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:26.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:26.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:26 smithi079 ceph-mon[40649]: pgmap v179: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:14:26.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:14:26.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:26 smithi079 ceph-mon[29565]: pgmap v179: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:14:26.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:14:26.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:26.374 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T08:14:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:26.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:26.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:26 smithi173 ceph-mon[33187]: pgmap v179: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:14:26.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:14:26.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:26.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:26.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:26.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:26.708 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:26.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:26.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:26.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.852 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:26.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:26.953 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:26.979 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:26.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:26.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:27.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:27.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:27.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:27.166 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:27.166 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:27.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:27.263 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:27.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:27.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:27.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-17T08:14:27.389 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:27.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:27.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:27 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:14:27.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:27 smithi173 ceph-mon[33187]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T08:14:27.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:27.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.531 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:27.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:27.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:27 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:14:27.609 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:27 smithi079 ceph-mon[29565]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T08:14:27.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:27 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2422135018' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:14:27.610 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:27 smithi079 ceph-mon[40649]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T08:14:27.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:27.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:27.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:27.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:27.812 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:27.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:27.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:14:27.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:27.951 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:27.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:27.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:28.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:28.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:28.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:28.127 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:28.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:14:28.187 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:28.211 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:28.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:28.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:28.337 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:28.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:28.438 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:28.468 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:28.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:28.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:28.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:28.610 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:28.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:28 smithi173 ceph-mon[33187]: pgmap v181: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:28.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:28.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:28.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:28.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:28.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:28 smithi079 ceph-mon[29565]: pgmap v181: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:28.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:28 smithi079 ceph-mon[40649]: pgmap v181: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:28.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:28.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:28.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:28.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:29.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:29.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:29.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:29.134 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:29.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:29.200 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:29.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:29.266 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:29.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.267 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:29.368 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:29.394 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:29.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:29.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:29.531 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:29.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.532 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:29.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:29 smithi079 ceph-mon[29565]: pgmap v182: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:29.569 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:29 smithi079 ceph-mon[40649]: pgmap v182: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:29.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:29.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:29.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.663 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:29 smithi173 ceph-mon[33187]: pgmap v182: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:14:29.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:29.800 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:29.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:29.902 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:29.926 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:29.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:29.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:30.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:30.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:30.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:30.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:30.187 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:30.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:30.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:30.321 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:30.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:30.389 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:30.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:30.452 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:30.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:30.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:30.576 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:30.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:30.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:30.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:30.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:30.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:30.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:30.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:30.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:30.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:30.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:31.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:31.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:31.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:31.151 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:31.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:31.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:14:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:14:31.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:31.292 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:31.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:31.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:31 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3945916936' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:31 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.324 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:31 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3945916936' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.324 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:31 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:31.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:31.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:31.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:31 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3945916936' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:31 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:14:31.500 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:31.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:31.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:31.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:31.704 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:31.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:31.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:31.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:31.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:31.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:31.981 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:31.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:31.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:32.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:32.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:32.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:32.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:32.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:32.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:32.358 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: 7.0 scrub starts 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: pgmap v184: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[29565]: 7.7 scrub starts 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:14:32.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T08:14:32.361 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: 7.0 scrub starts 2024-09-17T08:14:32.361 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: pgmap v184: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:32.361 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:32.361 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:32 smithi079 ceph-mon[40649]: 7.7 scrub starts 2024-09-17T08:14:32.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:32.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:32.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:32.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:32.529 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:32.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:32.656 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:32.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:32.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:14:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T08:14:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: 7.0 scrub starts 2024-09-17T08:14:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: pgmap v184: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 706 op/s 2024-09-17T08:14:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:32 smithi173 ceph-mon[33187]: 7.7 scrub starts 2024-09-17T08:14:32.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:32.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:32.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:32.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:32.920 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:32.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:32.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:33.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:33.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.058 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:33.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:33.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:33.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:33.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:33.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:33.348 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:33.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:33.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[29565]: 7.0 scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:14:33.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:33.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:33.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:33 smithi079 ceph-mon[40649]: 7.0 scrub ok 2024-09-17T08:14:33.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:33.476 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:33.477 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:33.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:33.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:33.487 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:14:33.487 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:14:33.487 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:14:33.487 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:14:33.487 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:33.488 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:33.488 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:33 smithi173 ceph-mon[33187]: 7.0 scrub ok 2024-09-17T08:14:33.579 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:33.601 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:33.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.602 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:33.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:33.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.729 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:33.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:33.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:33.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.864 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:33.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:33.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:33.991 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:33.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:33.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:34.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:34.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:34.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:14:34.239 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:34.262 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:34.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:34.269 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T08:14:34.364 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:34.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:34.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:34.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:34.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: pgmap v186: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:14:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:34 smithi173 ceph-mon[33187]: 7.f scrub starts 2024-09-17T08:14:34.490 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:34.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.516 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:34.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:34.610 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:14:34.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: pgmap v186: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[29565]: 7.f scrub starts 2024-09-17T08:14:34.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: pgmap v186: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:14:34.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:34 smithi079 ceph-mon[40649]: 7.f scrub starts 2024-09-17T08:14:34.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:34.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:34.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:34.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:34.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:34.855 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:34.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:34.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:34.910 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:34.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:35.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:35.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.044 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:35.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:35.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:35.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.188 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:35.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:35.289 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:35.320 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:35.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:35.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:35.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:35.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:35.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.591 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:35.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 7.e scrub starts 2024-09-17T08:14:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 7.7 scrub ok 2024-09-17T08:14:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:14:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:14:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 7.d scrub starts 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[40649]: 7.9 scrub starts 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 7.e scrub starts 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 7.7 scrub ok 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T08:14:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 7.d scrub starts 2024-09-17T08:14:35.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:35 smithi079 ceph-mon[29565]: 7.9 scrub starts 2024-09-17T08:14:35.692 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:35.715 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:35.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 7.e scrub starts 2024-09-17T08:14:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 7.7 scrub ok 2024-09-17T08:14:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:14:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:14:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3667154362' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:14:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T08:14:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 7.d scrub starts 2024-09-17T08:14:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:35 smithi173 ceph-mon[33187]: 7.9 scrub starts 2024-09-17T08:14:35.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:35.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.854 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:35.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:35.955 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:35.980 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:35.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:35.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:36.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:36.131 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:36.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:36.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:36.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:36.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 7.8 scrub starts 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 7.5 scrub starts 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: pgmap v188: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 921 op/s 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:14:36.431 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:36 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:14:36.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:36.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:36.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:36.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:36.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:36.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 7.8 scrub starts 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 7.5 scrub starts 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: pgmap v188: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 921 op/s 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:14:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 7.8 scrub starts 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 7.5 scrub starts 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: pgmap v188: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 921 op/s 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:14:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:36 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:14:36.673 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T08:14:36.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:14:36.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:36.712 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:36.712 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:36.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.760 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:36.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:36.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:36.885 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:36.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:36.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:36.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:37.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:37.101 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:37.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:37.143 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:37.182 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:37.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:37.338 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:37.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:37.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:37 smithi173 ceph-mon[33187]: 7.e scrub ok 2024-09-17T08:14:37.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:37 smithi173 ceph-mon[33187]: 7.f scrub ok 2024-09-17T08:14:37.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:37 smithi173 ceph-mon[33187]: 7.9 scrub ok 2024-09-17T08:14:37.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:37 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:14:37.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:37 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:14:37.476 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[40649]: 7.e scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[40649]: 7.f scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[40649]: 7.9 scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[29565]: 7.e scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[29565]: 7.f scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[29565]: 7.9 scrub ok 2024-09-17T08:14:37.514 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:14:37.515 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:37 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:14:37.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:14:37.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:37.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:37.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:37.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:37.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:37.845 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:37.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:37.882 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:37.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:37.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:37.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:38.008 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:38.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:38.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:38.136 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:38.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:38.170 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:14:38.171 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:14:38.203 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:14:38.203 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:14:38.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:38.239 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:14:38.239 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:14:38.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:38.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:38.279 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:14:38.279 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:14:38.314 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:14:38.314 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:14:38.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:14:38.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:14:38.350 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[40649]: 7.8 scrub ok 2024-09-17T08:14:38.350 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:14:38.350 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:14:38.351 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[40649]: pgmap v189: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 844 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:38.351 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2359533266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:38.351 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[29565]: 7.8 scrub ok 2024-09-17T08:14:38.351 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:14:38.351 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:14:38.352 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[29565]: pgmap v189: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 844 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:38.352 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:38 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2359533266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:38.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:38.386 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:14:38.386 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:14:38.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:38.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:38.419 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:14:38.419 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:14:38.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:38.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:38.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:38.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:38 smithi173 ceph-mon[33187]: 7.8 scrub ok 2024-09-17T08:14:38.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:38 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:14:38.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:38 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:14:38.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:38 smithi173 ceph-mon[33187]: pgmap v189: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 844 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:38.592 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:38 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2359533266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:38.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:38.665 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:38.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:38.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:38.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:38.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:38.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:38.928 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:38.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:38.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:39.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:39.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:39.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:39.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:39.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:39.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:39.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:39.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:39.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:39.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:39.458 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:39.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:39.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:39.597 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:39.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:14:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:14:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:14:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:39 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:14:39.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:14:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:14:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:14:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:14:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:14:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:39 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:14:39.727 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:39.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:39.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:39.874 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:39.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:39.875 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:39.976 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:40.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:40.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:40.132 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:40.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:40.275 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:40.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:40.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:40.405 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:40.470 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: pgmap v190: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:14:40.471 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:40 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:14:40.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:40.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: pgmap v190: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:14:40.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: pgmap v190: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:14:40.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:40 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:14:40.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:40.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:40.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:40.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:40.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:40.900 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:40.900 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:14:40.900 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:14:40.900 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:14:40.900 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:40.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:40.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:40.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:14:40.956 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:14:41.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:41.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:41.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:41.187 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:41.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:41.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:41.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:41.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:41.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:41.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:41.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:41.590 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.591 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:41.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:14:41.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:14:41.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[40649]: 7.2 scrub starts 2024-09-17T08:14:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:14:41.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:14:41.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:41 smithi079 ceph-mon[29565]: 7.2 scrub starts 2024-09-17T08:14:41.644 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:41 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:14:41.644 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:41 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:14:41.645 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:41 smithi173 ceph-mon[33187]: 7.2 scrub starts 2024-09-17T08:14:41.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:14:41.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:41.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:41.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:41.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:41.891 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:41.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:41.992 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:41.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:41.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:42.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:14:42.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:42.119 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:42.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:42.248 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:42.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:42.350 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:42.371 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:42.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:42.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:42.501 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:42.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:42.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:42.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[29565]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[29565]: pgmap v191: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 591 op/s 2024-09-17T08:14:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[29565]: 7.a scrub starts 2024-09-17T08:14:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[40649]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[40649]: pgmap v191: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 591 op/s 2024-09-17T08:14:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:42 smithi079 ceph-mon[40649]: 7.a scrub starts 2024-09-17T08:14:42.642 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:42.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:42.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:42 smithi173 ceph-mon[33187]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:42 smithi173 ceph-mon[33187]: pgmap v191: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 591 op/s 2024-09-17T08:14:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:42 smithi173 ceph-mon[33187]: 7.a scrub starts 2024-09-17T08:14:42.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:42.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.794 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:42.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:42.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:42.921 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:42.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:42.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:43.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:43.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:43.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:43.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:43.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:43.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:43.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:43.287 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:43.330 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-17T08:14:39.292409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5018032430,"num_objects":76757,"num_object_clones":0,"num_object_copies":230271,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76757,"num_whiteouts":0,"num_read":1194,"num_read_kb":1040,"num_write":178844,"num_write_kb":8155868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86523,"ondisk_log_size":86523,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15640656,"kb_used_data":14790992,"kb_used_omap":12,"kb_used_meta":849267,"kb_avail":734320560,"statfs":{"total":767960285184,"available":751944253440,"internally_reserved":0,"allocated":15145975808,"data_stored":15143447954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":869649976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-674037760,"num_objects":-10285,"num_object_clones":0,"num_object_copies":-30855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10285,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10285,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002771"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254777+0000","last_change":"2024-09-17T08:14:03.327276+0000","last_active":"2024-09-17T08:14:33.254777+0000","last_peered":"2024-09-17T08:14:33.254777+0000","last_clean":"2024-09-17T08:14:33.254777+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:14:33.254777+0000","last_undegraded":"2024-09-17T08:14:33.254777+0000","last_fullsized":"2024-09-17T08:14:33.254777+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.327169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.327169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:26.263265+0000","scrub_duration":0.013886001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":118,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:36.791664+0000","last_change":"2024-09-17T08:14:36.791881+0000","last_active":"2024-09-17T08:14:36.791664+0000","last_peered":"2024-09-17T08:14:36.791664+0000","last_clean":"2024-09-17T08:14:36.791664+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:14:36.791664+0000","last_undegraded":"2024-09-17T08:14:36.791664+0000","last_fullsized":"2024-09-17T08:14:36.791664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:36.791645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.791645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:39.635176+0000","scrub_duration":0.013350902,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":118,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.255201+0000","last_change":"2024-09-17T08:12:03.141440+0000","last_active":"2024-09-17T08:14:33.255201+0000","last_peered":"2024-09-17T08:14:33.255201+0000","last_clean":"2024-09-17T08:14:33.255201+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:14:33.255201+0000","last_undegraded":"2024-09-17T08:14:33.255201+0000","last_fullsized":"2024-09-17T08:14:33.255201+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:12:03.141349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:03.141349+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026318309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257416+0000","last_change":"2024-09-17T08:13:37.294653+0000","last_active":"2024-09-17T08:14:33.257416+0000","last_peered":"2024-09-17T08:14:33.257416+0000","last_clean":"2024-09-17T08:14:33.257416+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:14:33.257416+0000","last_undegraded":"2024-09-17T08:14:33.257416+0000","last_fullsized":"2024-09-17T08:14:33.257416+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.294551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.294551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:45.707563+0000","scrub_duration":0.0012183000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":96,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264741+0000","last_change":"2024-09-17T08:12:33.271632+0000","last_active":"2024-09-17T08:14:33.264741+0000","last_peered":"2024-09-17T08:14:33.264741+0000","last_clean":"2024-09-17T08:14:33.264741+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:14:33.264741+0000","last_undegraded":"2024-09-17T08:14:33.264741+0000","last_fullsized":"2024-09-17T08:14:33.264741+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:33.271466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:33.271466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057842520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":143,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257511+0000","last_change":"2024-09-17T08:13:50.305904+0000","last_active":"2024-09-17T08:14:33.257511+0000","last_peered":"2024-09-17T08:14:33.257511+0000","last_clean":"2024-09-17T08:14:33.257511+0000","last_became_active":"2024-09-17T08:11:32.433797+0000","last_became_peered":"2024-09-17T08:11:32.433797+0000","last_unstale":"2024-09-17T08:14:33.257511+0000","last_undegraded":"2024-09-17T08:14:33.257511+0000","last_fullsized":"2024-09-17T08:14:33.257511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:13:50.305732+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:13:50.305732+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:04.124060+0000","scrub_duration":0.020356910999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257620+0000","last_change":"2024-09-17T08:13:47.758152+0000","last_active":"2024-09-17T08:14:33.257620+0000","last_peered":"2024-09-17T08:14:33.257620+0000","last_clean":"2024-09-17T08:14:33.257620+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:14:33.257620+0000","last_undegraded":"2024-09-17T08:14:33.257620+0000","last_fullsized":"2024-09-17T08:14:33.257620+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.757894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.757894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:51.494786+0000","scrub_duration":0.00167778,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":111,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257613+0000","last_change":"2024-09-17T08:14:32.183940+0000","last_active":"2024-09-17T08:14:33.257613+0000","last_peered":"2024-09-17T08:14:33.257613+0000","last_clean":"2024-09-17T08:14:33.257613+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:14:33.257613+0000","last_undegraded":"2024-09-17T08:14:33.257613+0000","last_fullsized":"2024-09-17T08:14:33.257613+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:32.183821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.183821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.373034+0000","scrub_duration":0.001688852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270785+0000","last_change":"2024-09-17T08:13:47.009995+0000","last_active":"2024-09-17T08:14:33.270785+0000","last_peered":"2024-09-17T08:14:33.270785+0000","last_clean":"2024-09-17T08:14:33.270785+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:14:33.270785+0000","last_undegraded":"2024-09-17T08:14:33.270785+0000","last_fullsized":"2024-09-17T08:14:33.270785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.009857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.009857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.755878+0000","scrub_duration":0.002516679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":116,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257656+0000","last_change":"2024-09-17T08:13:40.204062+0000","last_active":"2024-09-17T08:14:33.257656+0000","last_peered":"2024-09-17T08:14:33.257656+0000","last_clean":"2024-09-17T08:14:33.257656+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:14:33.257656+0000","last_undegraded":"2024-09-17T08:14:33.257656+0000","last_fullsized":"2024-09-17T08:14:33.257656+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:13:40.203920+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:13:40.203920+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:47.482974+0000","scrub_duration":0.0058130819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270202+0000","last_change":"2024-09-17T08:12:40.610530+0000","last_active":"2024-09-17T08:14:33.270202+0000","last_peered":"2024-09-17T08:14:33.270202+0000","last_clean":"2024-09-17T08:14:33.270202+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:14:33.270202+0000","last_undegraded":"2024-09-17T08:14:33.270202+0000","last_fullsized":"2024-09-17T08:14:33.270202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:40.610413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:40.610413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054815920000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.247689+0000","last_change":"2024-09-17T08:13:53.151626+0000","last_active":"2024-09-17T08:14:33.247689+0000","last_peered":"2024-09-17T08:14:33.247689+0000","last_clean":"2024-09-17T08:14:33.247689+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:14:33.247689+0000","last_undegraded":"2024-09-17T08:14:33.247689+0000","last_fullsized":"2024-09-17T08:14:33.247689+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.151500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.151500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.803456+0000","scrub_duration":0.0089419549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.271260+0000","last_change":"2024-09-17T08:13:57.066226+0000","last_active":"2024-09-17T08:14:33.271260+0000","last_peered":"2024-09-17T08:14:33.271260+0000","last_clean":"2024-09-17T08:14:33.271260+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:14:33.271260+0000","last_undegraded":"2024-09-17T08:14:33.271260+0000","last_fullsized":"2024-09-17T08:14:33.271260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:57.066069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.066069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:59.438287+0000","scrub_duration":0.0018716270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":131,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.212267+0000","last_change":"2024-09-17T08:14:34.212267+0000","last_active":"2024-09-17T08:14:34.212267+0000","last_peered":"2024-09-17T08:14:34.212267+0000","last_clean":"2024-09-17T08:14:34.212267+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:14:34.212267+0000","last_undegraded":"2024-09-17T08:14:34.212267+0000","last_fullsized":"2024-09-17T08:14:34.212267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:14:34.212100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:34.212100+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:35.674727+0000","scrub_duration":0.0066844529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248180+0000","last_change":"2024-09-17T08:13:54.189394+0000","last_active":"2024-09-17T08:14:33.248180+0000","last_peered":"2024-09-17T08:14:33.248180+0000","last_clean":"2024-09-17T08:14:33.248180+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:14:33.248180+0000","last_undegraded":"2024-09-17T08:14:33.248180+0000","last_fullsized":"2024-09-17T08:14:33.248180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:54.189279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.189279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.336622+0000","scrub_duration":0.002245637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263471+0000","last_change":"2024-09-17T08:13:40.588846+0000","last_active":"2024-09-17T08:14:33.263471+0000","last_peered":"2024-09-17T08:14:33.263471+0000","last_clean":"2024-09-17T08:14:33.263471+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:14:33.263471+0000","last_undegraded":"2024-09-17T08:14:33.263471+0000","last_fullsized":"2024-09-17T08:14:33.263471+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:40.588715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:40.588715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:08.948015+0000","scrub_duration":0.0021501789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"138'11318","reported_seq":11488,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.417307+0000","last_change":"2024-09-17T08:14:36.071289+0000","last_active":"2024-09-17T08:14:37.417307+0000","last_peered":"2024-09-17T08:14:37.417307+0000","last_clean":"2024-09-17T08:14:37.417307+0000","last_became_active":"2024-09-17T08:12:05.207481+0000","last_became_peered":"2024-09-17T08:12:05.207481+0000","last_unstale":"2024-09-17T08:14:37.417307+0000","last_undegraded":"2024-09-17T08:14:37.417307+0000","last_fullsized":"2024-09-17T08:14:37.417307+0000","mapping_epoch":91,"log_start":"106'4700","ondisk_log_start":"106'4700","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11226","last_scrub_stamp":"2024-09-17T08:14:36.068645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.068645+0000","objects_scrubbed":5035,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:05.867385+0000","scrub_duration":2.2070316010000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320077824,"num_objects":4884,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":518464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":133,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258221+0000","last_change":"2024-09-17T08:13:33.288306+0000","last_active":"2024-09-17T08:14:33.258221+0000","last_peered":"2024-09-17T08:14:33.258221+0000","last_clean":"2024-09-17T08:14:33.258221+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:14:33.258221+0000","last_undegraded":"2024-09-17T08:14:33.258221+0000","last_fullsized":"2024-09-17T08:14:33.258221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:13:33.288160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:33.288160+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:58.878404+0000","scrub_duration":0.0089040130000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270542+0000","last_change":"2024-09-17T08:13:46.010229+0000","last_active":"2024-09-17T08:14:33.270542+0000","last_peered":"2024-09-17T08:14:33.270542+0000","last_clean":"2024-09-17T08:14:33.270542+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:14:33.270542+0000","last_undegraded":"2024-09-17T08:14:33.270542+0000","last_fullsized":"2024-09-17T08:14:33.270542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:46.010103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:46.010103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:52.201701+0000","scrub_duration":0.0016522850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.256968+0000","last_change":"2024-09-17T08:12:48.882990+0000","last_active":"2024-09-17T08:14:33.256968+0000","last_peered":"2024-09-17T08:14:33.256968+0000","last_clean":"2024-09-17T08:14:33.256968+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:14:33.256968+0000","last_undegraded":"2024-09-17T08:14:33.256968+0000","last_fullsized":"2024-09-17T08:14:33.256968+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:48.881035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:48.881035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014848926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.277632+0000","last_change":"2024-09-17T08:12:43.301341+0000","last_active":"2024-09-17T08:14:34.277632+0000","last_peered":"2024-09-17T08:14:34.277632+0000","last_clean":"2024-09-17T08:14:34.277632+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:14:34.277632+0000","last_undegraded":"2024-09-17T08:14:34.277632+0000","last_fullsized":"2024-09-17T08:14:34.277632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.301210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.301210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005881786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"138'11234","reported_seq":11402,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.611350+0000","last_change":"2024-09-17T08:14:36.164221+0000","last_active":"2024-09-17T08:14:37.611350+0000","last_peered":"2024-09-17T08:14:37.611350+0000","last_clean":"2024-09-17T08:14:37.611350+0000","last_became_active":"2024-09-17T08:12:05.207933+0000","last_became_peered":"2024-09-17T08:12:05.207933+0000","last_unstale":"2024-09-17T08:14:37.611350+0000","last_undegraded":"2024-09-17T08:14:37.611350+0000","last_fullsized":"2024-09-17T08:14:37.611350+0000","mapping_epoch":91,"log_start":"110'5700","ondisk_log_start":"110'5700","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11122","last_scrub_stamp":"2024-09-17T08:14:36.163963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.163963+0000","objects_scrubbed":5049,"log_size":5534,"log_dups_size":0,"ondisk_log_size":5534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:39.578774+0000","scrub_duration":2.3961925420000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319291392,"num_objects":4872,"num_object_clones":0,"num_object_copies":14616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11234,"num_write_kb":515392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.198979+0000","last_change":"2024-09-17T08:14:37.199095+0000","last_active":"2024-09-17T08:14:37.198979+0000","last_peered":"2024-09-17T08:14:37.198979+0000","last_clean":"2024-09-17T08:14:37.198979+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:14:37.198979+0000","last_undegraded":"2024-09-17T08:14:37.198979+0000","last_fullsized":"2024-09-17T08:14:37.198979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:14:37.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:37.198955+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:46.937610+0000","scrub_duration":0.0087823410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257400+0000","last_change":"2024-09-17T08:12:39.889514+0000","last_active":"2024-09-17T08:14:33.257400+0000","last_peered":"2024-09-17T08:14:33.257400+0000","last_clean":"2024-09-17T08:14:33.257400+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:14:33.257400+0000","last_undegraded":"2024-09-17T08:14:33.257400+0000","last_fullsized":"2024-09-17T08:14:33.257400+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:39.889220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:39.889220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033635470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":168,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263021+0000","last_change":"2024-09-17T08:12:38.173535+0000","last_active":"2024-09-17T08:14:33.263021+0000","last_peered":"2024-09-17T08:14:33.263021+0000","last_clean":"2024-09-17T08:14:33.263021+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:14:33.263021+0000","last_undegraded":"2024-09-17T08:14:33.263021+0000","last_fullsized":"2024-09-17T08:14:33.263021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:12:38.173398+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:11:18.109796+0000","last_clean_scrub_stamp":"2024-09-17T08:12:38.173398+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035078980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257982+0000","last_change":"2024-09-17T08:13:59.358631+0000","last_active":"2024-09-17T08:14:33.257982+0000","last_peered":"2024-09-17T08:14:33.257982+0000","last_clean":"2024-09-17T08:14:33.257982+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:14:33.257982+0000","last_undegraded":"2024-09-17T08:14:33.257982+0000","last_fullsized":"2024-09-17T08:14:33.257982+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:59.358484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:13:59.358484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:09.991516+0000","scrub_duration":0.0089018259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"138'11457","reported_seq":11520,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.499366+0000","last_change":"2024-09-17T08:12:46.191819+0000","last_active":"2024-09-17T08:14:37.499366+0000","last_peered":"2024-09-17T08:14:37.499366+0000","last_clean":"2024-09-17T08:14:37.499366+0000","last_became_active":"2024-09-17T08:12:46.185461+0000","last_became_peered":"2024-09-17T08:12:46.185461+0000","last_unstale":"2024-09-17T08:14:37.499366+0000","last_undegraded":"2024-09-17T08:14:37.499366+0000","last_fullsized":"2024-09-17T08:14:37.499366+0000","mapping_epoch":103,"log_start":"115'6700","ondisk_log_start":"115'6700","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4757,"log_dups_size":0,"ondisk_log_size":4757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319619072,"num_objects":4877,"num_object_clones":0,"num_object_copies":14631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11457,"num_write_kb":522688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":137,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257001+0000","last_change":"2024-09-17T08:12:51.952863+0000","last_active":"2024-09-17T08:14:33.257001+0000","last_peered":"2024-09-17T08:14:33.257001+0000","last_clean":"2024-09-17T08:14:33.257001+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:14:33.257001+0000","last_undegraded":"2024-09-17T08:14:33.257001+0000","last_fullsized":"2024-09-17T08:14:33.257001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:12:51.952706+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021231191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257799+0000","last_change":"2024-09-17T08:13:58.754546+0000","last_active":"2024-09-17T08:14:33.257799+0000","last_peered":"2024-09-17T08:14:33.257799+0000","last_clean":"2024-09-17T08:14:33.257799+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:14:33.257799+0000","last_undegraded":"2024-09-17T08:14:33.257799+0000","last_fullsized":"2024-09-17T08:14:33.257799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:58.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:25.006130+0000","scrub_duration":0.050368468,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":167,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258327+0000","last_change":"2024-09-17T08:13:43.244011+0000","last_active":"2024-09-17T08:14:33.258327+0000","last_peered":"2024-09-17T08:14:33.258327+0000","last_clean":"2024-09-17T08:14:33.258327+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:14:33.258327+0000","last_undegraded":"2024-09-17T08:14:33.258327+0000","last_fullsized":"2024-09-17T08:14:33.258327+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:43.243896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:43.243896+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:52.639156+0000","scrub_duration":0.0044011149999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248143+0000","last_change":"2024-09-17T08:13:57.216118+0000","last_active":"2024-09-17T08:14:33.248143+0000","last_peered":"2024-09-17T08:14:33.248143+0000","last_clean":"2024-09-17T08:14:33.248143+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:14:33.248143+0000","last_undegraded":"2024-09-17T08:14:33.248143+0000","last_fullsized":"2024-09-17T08:14:33.248143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:57.216011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.216011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.931644+0000","scrub_duration":0.0045278590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"138'11019","reported_seq":11085,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.326863+0000","last_change":"2024-09-17T08:14:34.806303+0000","last_active":"2024-09-17T08:14:37.326863+0000","last_peered":"2024-09-17T08:14:37.326863+0000","last_clean":"2024-09-17T08:14:37.326863+0000","last_became_active":"2024-09-17T08:12:46.183640+0000","last_became_peered":"2024-09-17T08:12:46.183640+0000","last_unstale":"2024-09-17T08:14:37.326863+0000","last_undegraded":"2024-09-17T08:14:37.326863+0000","last_fullsized":"2024-09-17T08:14:37.326863+0000","mapping_epoch":103,"log_start":"111'5900","ondisk_log_start":"111'5900","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":5119,"log_dups_size":0,"ondisk_log_size":5119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309461016,"num_objects":4723,"num_object_clones":0,"num_object_copies":14169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11019,"num_write_kb":503681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":124,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258222+0000","last_change":"2024-09-17T08:13:51.259561+0000","last_active":"2024-09-17T08:14:33.258222+0000","last_peered":"2024-09-17T08:14:33.258222+0000","last_clean":"2024-09-17T08:14:33.258222+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:14:33.258222+0000","last_undegraded":"2024-09-17T08:14:33.258222+0000","last_fullsized":"2024-09-17T08:14:33.258222+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:13:51.259464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:51.259464+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.498439+0000","scrub_duration":0.0023729620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270325+0000","last_change":"2024-09-17T08:13:49.008226+0000","last_active":"2024-09-17T08:14:33.270325+0000","last_peered":"2024-09-17T08:14:33.270325+0000","last_clean":"2024-09-17T08:14:33.270325+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:14:33.270325+0000","last_undegraded":"2024-09-17T08:14:33.270325+0000","last_fullsized":"2024-09-17T08:14:33.270325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.008096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.008096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.255380+0000","scrub_duration":0.010394446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248382+0000","last_change":"2024-09-17T08:12:42.124247+0000","last_active":"2024-09-17T08:14:33.248382+0000","last_peered":"2024-09-17T08:14:33.248382+0000","last_clean":"2024-09-17T08:14:33.248382+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:14:33.248382+0000","last_undegraded":"2024-09-17T08:14:33.248382+0000","last_fullsized":"2024-09-17T08:14:33.248382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.124120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.124120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001609774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":90,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254608+0000","last_change":"2024-09-17T08:12:00.142321+0000","last_active":"2024-09-17T08:14:33.254608+0000","last_peered":"2024-09-17T08:14:33.254608+0000","last_clean":"2024-09-17T08:14:33.254608+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:14:33.254608+0000","last_undegraded":"2024-09-17T08:14:33.254608+0000","last_fullsized":"2024-09-17T08:14:33.254608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:00.142246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:00.142246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001285509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"138'10985","reported_seq":11040,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.489926+0000","last_change":"2024-09-17T08:12:05.209102+0000","last_active":"2024-09-17T08:14:37.489926+0000","last_peered":"2024-09-17T08:14:37.489926+0000","last_clean":"2024-09-17T08:14:37.489926+0000","last_became_active":"2024-09-17T08:12:05.208839+0000","last_became_peered":"2024-09-17T08:12:05.208839+0000","last_unstale":"2024-09-17T08:14:37.489926+0000","last_undegraded":"2024-09-17T08:14:37.489926+0000","last_fullsized":"2024-09-17T08:14:37.489926+0000","mapping_epoch":91,"log_start":"114'6200","ondisk_log_start":"114'6200","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303235072,"num_objects":4627,"num_object_clones":0,"num_object_copies":13881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10985,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":126,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257622+0000","last_change":"2024-09-17T08:13:34.301444+0000","last_active":"2024-09-17T08:14:33.257622+0000","last_peered":"2024-09-17T08:14:33.257622+0000","last_clean":"2024-09-17T08:14:33.257622+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:14:33.257622+0000","last_undegraded":"2024-09-17T08:14:33.257622+0000","last_fullsized":"2024-09-17T08:14:33.257622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:13:34.301346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.301346+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.346147+0000","scrub_duration":0.0022927440000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270824+0000","last_change":"2024-09-17T08:12:46.185743+0000","last_active":"2024-09-17T08:14:33.270824+0000","last_peered":"2024-09-17T08:14:33.270824+0000","last_clean":"2024-09-17T08:14:33.270824+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:14:33.270824+0000","last_undegraded":"2024-09-17T08:14:33.270824+0000","last_fullsized":"2024-09-17T08:14:33.270824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:35.429117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:35.429117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":170,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254653+0000","last_change":"2024-09-17T08:13:55.128984+0000","last_active":"2024-09-17T08:14:33.254653+0000","last_peered":"2024-09-17T08:14:33.254653+0000","last_clean":"2024-09-17T08:14:33.254653+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:14:33.254653+0000","last_undegraded":"2024-09-17T08:14:33.254653+0000","last_fullsized":"2024-09-17T08:14:33.254653+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:55.128587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:55.128587+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:03.969779+0000","scrub_duration":0.0087543629999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270945+0000","last_change":"2024-09-17T08:12:42.563391+0000","last_active":"2024-09-17T08:14:33.270945+0000","last_peered":"2024-09-17T08:14:33.270945+0000","last_clean":"2024-09-17T08:14:33.270945+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:14:33.270945+0000","last_undegraded":"2024-09-17T08:14:33.270945+0000","last_fullsized":"2024-09-17T08:14:33.270945+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.563306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036476260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"138'11249","reported_seq":11306,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.488214+0000","last_change":"2024-09-17T08:12:05.210025+0000","last_active":"2024-09-17T08:14:37.488214+0000","last_peered":"2024-09-17T08:14:37.488214+0000","last_clean":"2024-09-17T08:14:37.488214+0000","last_became_active":"2024-09-17T08:12:05.209256+0000","last_became_peered":"2024-09-17T08:12:05.209256+0000","last_unstale":"2024-09-17T08:14:37.488214+0000","last_undegraded":"2024-09-17T08:14:37.488214+0000","last_fullsized":"2024-09-17T08:14:37.488214+0000","mapping_epoch":91,"log_start":"114'6500","ondisk_log_start":"114'6500","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312672256,"num_objects":4771,"num_object_clones":0,"num_object_copies":14313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11249,"num_write_kb":512640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":137,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254856+0000","last_change":"2024-09-17T08:12:56.366089+0000","last_active":"2024-09-17T08:14:33.254856+0000","last_peered":"2024-09-17T08:14:33.254856+0000","last_clean":"2024-09-17T08:14:33.254856+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:14:33.254856+0000","last_undegraded":"2024-09-17T08:14:33.254856+0000","last_fullsized":"2024-09-17T08:14:33.254856+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:12:56.365721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.365721+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012144524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.256945+0000","last_change":"2024-09-17T08:14:03.639239+0000","last_active":"2024-09-17T08:14:33.256945+0000","last_peered":"2024-09-17T08:14:33.256945+0000","last_clean":"2024-09-17T08:14:33.256945+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:14:33.256945+0000","last_undegraded":"2024-09-17T08:14:33.256945+0000","last_fullsized":"2024-09-17T08:14:33.256945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.639130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.639130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.754192+0000","scrub_duration":0.0025313649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257523+0000","last_change":"2024-09-17T08:13:52.280824+0000","last_active":"2024-09-17T08:14:33.257523+0000","last_peered":"2024-09-17T08:14:33.257523+0000","last_clean":"2024-09-17T08:14:33.257523+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:14:33.257523+0000","last_undegraded":"2024-09-17T08:14:33.257523+0000","last_fullsized":"2024-09-17T08:14:33.257523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:52.280710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:13:52.280710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:54.376108+0000","scrub_duration":0.0055970719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257692+0000","last_change":"2024-09-17T08:13:46.308939+0000","last_active":"2024-09-17T08:14:33.257692+0000","last_peered":"2024-09-17T08:14:33.257692+0000","last_clean":"2024-09-17T08:14:33.257692+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:14:33.257692+0000","last_undegraded":"2024-09-17T08:14:33.257692+0000","last_fullsized":"2024-09-17T08:14:33.257692+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:46.308829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:46.308829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:48.009682+0000","scrub_duration":0.0085156869999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"138'13085","reported_seq":13440,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.301796+0000","last_change":"2024-09-17T08:14:32.435613+0000","last_active":"2024-09-17T08:14:37.301796+0000","last_peered":"2024-09-17T08:14:37.301796+0000","last_clean":"2024-09-17T08:14:37.301796+0000","last_became_active":"2024-09-17T08:12:05.208659+0000","last_became_peered":"2024-09-17T08:12:05.208659+0000","last_unstale":"2024-09-17T08:14:37.301796+0000","last_undegraded":"2024-09-17T08:14:37.301796+0000","last_fullsized":"2024-09-17T08:14:37.301796+0000","mapping_epoch":91,"log_start":"121'7401","ondisk_log_start":"121'7401","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'12787","last_scrub_stamp":"2024-09-17T08:14:32.435450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.435450+0000","objects_scrubbed":4928,"log_size":5684,"log_dups_size":0,"ondisk_log_size":5684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:52.936186+0000","scrub_duration":1.1622392939999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301137920,"num_objects":4595,"num_object_clones":0,"num_object_copies":13785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10929,"num_write_kb":496768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":141,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264094+0000","last_change":"2024-09-17T08:12:44.798160+0000","last_active":"2024-09-17T08:14:33.264094+0000","last_peered":"2024-09-17T08:14:33.264094+0000","last_clean":"2024-09-17T08:14:33.264094+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:14:33.264094+0000","last_undegraded":"2024-09-17T08:14:33.264094+0000","last_fullsized":"2024-09-17T08:14:33.264094+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:12:44.798020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:44.798020+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050403740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":112,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270866+0000","last_change":"2024-09-17T08:12:43.593151+0000","last_active":"2024-09-17T08:14:33.270866+0000","last_peered":"2024-09-17T08:14:33.270866+0000","last_clean":"2024-09-17T08:14:33.270866+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:14:33.270866+0000","last_undegraded":"2024-09-17T08:14:33.270866+0000","last_fullsized":"2024-09-17T08:14:33.270866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:12:43.592847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.592847+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024350980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":58,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264264+0000","last_change":"2024-09-17T08:13:59.677721+0000","last_active":"2024-09-17T08:14:33.264264+0000","last_peered":"2024-09-17T08:14:33.264264+0000","last_clean":"2024-09-17T08:14:33.264264+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:14:33.264264+0000","last_undegraded":"2024-09-17T08:14:33.264264+0000","last_fullsized":"2024-09-17T08:14:33.264264+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:59.677603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:59.677603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.145100+0000","scrub_duration":0.003242782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257553+0000","last_change":"2024-09-17T08:12:43.884405+0000","last_active":"2024-09-17T08:14:33.257553+0000","last_peered":"2024-09-17T08:14:33.257553+0000","last_clean":"2024-09-17T08:14:33.257553+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:14:33.257553+0000","last_undegraded":"2024-09-17T08:14:33.257553+0000","last_fullsized":"2024-09-17T08:14:33.257553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.884245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.884245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044087800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"138'11066","reported_seq":11280,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.236651+0000","last_change":"2024-09-17T08:13:58.900421+0000","last_active":"2024-09-17T08:14:37.236651+0000","last_peered":"2024-09-17T08:14:37.236651+0000","last_clean":"2024-09-17T08:14:37.236651+0000","last_became_active":"2024-09-17T08:12:05.206361+0000","last_became_peered":"2024-09-17T08:12:05.206361+0000","last_unstale":"2024-09-17T08:14:37.236651+0000","last_undegraded":"2024-09-17T08:14:37.236651+0000","last_fullsized":"2024-09-17T08:14:37.236651+0000","mapping_epoch":91,"log_start":"110'5800","ondisk_log_start":"110'5800","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8703","last_scrub_stamp":"2024-09-17T08:13:58.900297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.900297+0000","objects_scrubbed":7257,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:59.838343+0000","scrub_duration":1.5784367939999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317849600,"num_objects":4850,"num_object_clones":0,"num_object_copies":14550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11066,"num_write_kb":509312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":120,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257324+0000","last_change":"2024-09-17T08:12:53.896268+0000","last_active":"2024-09-17T08:14:33.257324+0000","last_peered":"2024-09-17T08:14:33.257324+0000","last_clean":"2024-09-17T08:14:33.257324+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:14:33.257324+0000","last_undegraded":"2024-09-17T08:14:33.257324+0000","last_fullsized":"2024-09-17T08:14:33.257324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:12:53.894036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:53.894036+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022172299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257227+0000","last_change":"2024-09-17T08:12:32.839208+0000","last_active":"2024-09-17T08:14:33.257227+0000","last_peered":"2024-09-17T08:14:33.257227+0000","last_clean":"2024-09-17T08:14:33.257227+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:14:33.257227+0000","last_undegraded":"2024-09-17T08:14:33.257227+0000","last_fullsized":"2024-09-17T08:14:33.257227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:32.838985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:32.838985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088459530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":167,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263956+0000","last_change":"2024-09-17T08:12:51.719764+0000","last_active":"2024-09-17T08:14:33.263956+0000","last_peered":"2024-09-17T08:14:33.263956+0000","last_clean":"2024-09-17T08:14:33.263956+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:14:33.263956+0000","last_undegraded":"2024-09-17T08:14:33.263956+0000","last_fullsized":"2024-09-17T08:14:33.263956+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:12:51.719648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:51.719648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007910116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263362+0000","last_change":"2024-09-17T08:13:53.683916+0000","last_active":"2024-09-17T08:14:33.263362+0000","last_peered":"2024-09-17T08:14:33.263362+0000","last_clean":"2024-09-17T08:14:33.263362+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:14:33.263362+0000","last_undegraded":"2024-09-17T08:14:33.263362+0000","last_fullsized":"2024-09-17T08:14:33.263362+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.683777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.683777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.587530+0000","scrub_duration":0.007378796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"138'11073","reported_seq":11293,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.600348+0000","last_change":"2024-09-17T08:14:34.008301+0000","last_active":"2024-09-17T08:14:37.600348+0000","last_peered":"2024-09-17T08:14:37.600348+0000","last_clean":"2024-09-17T08:14:37.600348+0000","last_became_active":"2024-09-17T08:12:46.185093+0000","last_became_peered":"2024-09-17T08:12:46.185093+0000","last_unstale":"2024-09-17T08:14:37.600348+0000","last_undegraded":"2024-09-17T08:14:37.600348+0000","last_fullsized":"2024-09-17T08:14:37.600348+0000","mapping_epoch":103,"log_start":"110'5600","ondisk_log_start":"110'5600","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'10882","last_scrub_stamp":"2024-09-17T08:14:34.007989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:34.007989+0000","objects_scrubbed":4991,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:44.112073+0000","scrub_duration":2.270366321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309264384,"num_objects":4719,"num_object_clones":0,"num_object_copies":14157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11073,"num_write_kb":505344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":209,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257155+0000","last_change":"2024-09-17T08:12:49.895324+0000","last_active":"2024-09-17T08:14:33.257155+0000","last_peered":"2024-09-17T08:14:33.257155+0000","last_clean":"2024-09-17T08:14:33.257155+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:14:33.257155+0000","last_undegraded":"2024-09-17T08:14:33.257155+0000","last_fullsized":"2024-09-17T08:14:33.257155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:12:49.895130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.895130+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076594480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":118,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.249766+0000","last_change":"2024-09-17T08:14:03.251865+0000","last_active":"2024-09-17T08:14:33.249766+0000","last_peered":"2024-09-17T08:14:33.249766+0000","last_clean":"2024-09-17T08:14:33.249766+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:14:33.249766+0000","last_undegraded":"2024-09-17T08:14:33.249766+0000","last_fullsized":"2024-09-17T08:14:33.249766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.251783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:03.251783+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.251783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.513602+0000","scrub_duration":0.002663585,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265097+0000","last_change":"2024-09-17T08:13:49.610395+0000","last_active":"2024-09-17T08:14:33.265097+0000","last_peered":"2024-09-17T08:14:33.265097+0000","last_clean":"2024-09-17T08:14:33.265097+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:14:33.265097+0000","last_undegraded":"2024-09-17T08:14:33.265097+0000","last_fullsized":"2024-09-17T08:14:33.265097+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.610090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.610090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:50.329505+0000","scrub_duration":0.0081973259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264502+0000","last_change":"2024-09-17T08:12:52.393950+0000","last_active":"2024-09-17T08:14:33.264502+0000","last_peered":"2024-09-17T08:14:33.264502+0000","last_clean":"2024-09-17T08:14:33.264502+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:14:33.264502+0000","last_undegraded":"2024-09-17T08:14:33.264502+0000","last_fullsized":"2024-09-17T08:14:33.264502+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:52.393815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:52.393815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010050285000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"138'11055","reported_seq":11263,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.576843+0000","last_change":"2024-09-17T08:14:04.288383+0000","last_active":"2024-09-17T08:14:37.576843+0000","last_peered":"2024-09-17T08:14:37.576843+0000","last_clean":"2024-09-17T08:14:37.576843+0000","last_became_active":"2024-09-17T08:12:05.206469+0000","last_became_peered":"2024-09-17T08:12:05.206469+0000","last_unstale":"2024-09-17T08:14:37.576843+0000","last_undegraded":"2024-09-17T08:14:37.576843+0000","last_fullsized":"2024-09-17T08:14:37.576843+0000","mapping_epoch":91,"log_start":"110'5600","ondisk_log_start":"110'5600","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8997","last_scrub_stamp":"2024-09-17T08:14:04.288198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.288198+0000","objects_scrubbed":6921,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:32.382198+0000","scrub_duration":2.5911683210000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313720832,"num_objects":4787,"num_object_clones":0,"num_object_copies":14361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11055,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":97,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.261797+0000","last_change":"2024-09-17T08:13:39.910053+0000","last_active":"2024-09-17T08:14:33.261797+0000","last_peered":"2024-09-17T08:14:33.261797+0000","last_clean":"2024-09-17T08:14:33.261797+0000","last_became_active":"2024-09-17T08:11:32.435377+0000","last_became_peered":"2024-09-17T08:11:32.435377+0000","last_unstale":"2024-09-17T08:14:33.261797+0000","last_undegraded":"2024-09-17T08:14:33.261797+0000","last_fullsized":"2024-09-17T08:14:33.261797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:13:39.909755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:39.909755+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:05.391461+0000","scrub_duration":0.0076279349999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":390,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:35.951562+0000","last_change":"2024-09-17T08:12:47.905087+0000","last_active":"2024-09-17T08:14:35.951562+0000","last_peered":"2024-09-17T08:14:35.951562+0000","last_clean":"2024-09-17T08:14:35.951562+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:14:35.951562+0000","last_undegraded":"2024-09-17T08:14:35.951562+0000","last_fullsized":"2024-09-17T08:14:35.951562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:12:47.904961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:47.904961+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015034595,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":275,"num_read_kb":270,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263204+0000","last_change":"2024-09-17T08:12:42.266996+0000","last_active":"2024-09-17T08:14:33.263204+0000","last_peered":"2024-09-17T08:14:33.263204+0000","last_clean":"2024-09-17T08:14:33.263204+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:14:33.263204+0000","last_undegraded":"2024-09-17T08:14:33.263204+0000","last_fullsized":"2024-09-17T08:14:33.263204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.266792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.266792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085145599999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263282+0000","last_change":"2024-09-17T08:12:55.327814+0000","last_active":"2024-09-17T08:14:33.263282+0000","last_peered":"2024-09-17T08:14:33.263282+0000","last_clean":"2024-09-17T08:14:33.263282+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:14:33.263282+0000","last_undegraded":"2024-09-17T08:14:33.263282+0000","last_fullsized":"2024-09-17T08:14:33.263282+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:55.327282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.327282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066521990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270561+0000","last_change":"2024-09-17T08:12:45.574193+0000","last_active":"2024-09-17T08:14:33.270561+0000","last_peered":"2024-09-17T08:14:33.270561+0000","last_clean":"2024-09-17T08:14:33.270561+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:14:33.270561+0000","last_undegraded":"2024-09-17T08:14:33.270561+0000","last_fullsized":"2024-09-17T08:14:33.270561+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:12:45.574097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00138104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"138'11270","reported_seq":11328,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.197534+0000","last_change":"2024-09-17T08:14:35.242009+0000","last_active":"2024-09-17T08:14:37.197534+0000","last_peered":"2024-09-17T08:14:37.197534+0000","last_clean":"2024-09-17T08:14:37.197534+0000","last_became_active":"2024-09-17T08:12:05.207217+0000","last_became_peered":"2024-09-17T08:12:05.207217+0000","last_unstale":"2024-09-17T08:14:37.197534+0000","last_undegraded":"2024-09-17T08:14:37.197534+0000","last_fullsized":"2024-09-17T08:14:37.197534+0000","mapping_epoch":91,"log_start":"113'6300","ondisk_log_start":"113'6300","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314441728,"num_objects":4798,"num_object_clones":0,"num_object_copies":14394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":514176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":129,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265920+0000","last_change":"2024-09-17T08:13:43.584287+0000","last_active":"2024-09-17T08:14:33.265920+0000","last_peered":"2024-09-17T08:14:33.265920+0000","last_clean":"2024-09-17T08:14:33.265920+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:14:33.265920+0000","last_undegraded":"2024-09-17T08:14:33.265920+0000","last_fullsized":"2024-09-17T08:14:33.265920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:13:43.584153+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:13:43.584153+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.644811+0000","scrub_duration":0.017778021000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":113,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.268501+0000","last_change":"2024-09-17T08:12:55.802841+0000","last_active":"2024-09-17T08:14:33.268501+0000","last_peered":"2024-09-17T08:14:33.268501+0000","last_clean":"2024-09-17T08:14:33.268501+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:14:33.268501+0000","last_undegraded":"2024-09-17T08:14:33.268501+0000","last_fullsized":"2024-09-17T08:14:33.268501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:12:55.802644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.802644+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010970435000000001,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270343+0000","last_change":"2024-09-17T08:13:56.115539+0000","last_active":"2024-09-17T08:14:33.270343+0000","last_peered":"2024-09-17T08:14:33.270343+0000","last_clean":"2024-09-17T08:14:33.270343+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:14:33.270343+0000","last_undegraded":"2024-09-17T08:14:33.270343+0000","last_fullsized":"2024-09-17T08:14:33.270343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:13:56.115447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:21.526055+0000","scrub_duration":0.0045415680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248448+0000","last_change":"2024-09-17T08:12:43.075970+0000","last_active":"2024-09-17T08:14:33.248448+0000","last_peered":"2024-09-17T08:14:33.248448+0000","last_clean":"2024-09-17T08:14:33.248448+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:14:33.248448+0000","last_undegraded":"2024-09-17T08:14:33.248448+0000","last_fullsized":"2024-09-17T08:14:33.248448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.075863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.075863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013502640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.256897+0000","last_change":"2024-09-17T08:13:44.744020+0000","last_active":"2024-09-17T08:14:33.256897+0000","last_peered":"2024-09-17T08:14:33.256897+0000","last_clean":"2024-09-17T08:14:33.256897+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:14:33.256897+0000","last_undegraded":"2024-09-17T08:14:33.256897+0000","last_fullsized":"2024-09-17T08:14:33.256897+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:44.743795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:44.743795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:03.716597+0000","scrub_duration":0.0038700259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"138'11089","reported_seq":11301,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.603857+0000","last_change":"2024-09-17T08:14:03.362812+0000","last_active":"2024-09-17T08:14:37.603857+0000","last_peered":"2024-09-17T08:14:37.603857+0000","last_clean":"2024-09-17T08:14:37.603857+0000","last_became_active":"2024-09-17T08:12:05.208529+0000","last_became_peered":"2024-09-17T08:12:05.208529+0000","last_unstale":"2024-09-17T08:14:37.603857+0000","last_undegraded":"2024-09-17T08:14:37.603857+0000","last_fullsized":"2024-09-17T08:14:37.603857+0000","mapping_epoch":91,"log_start":"110'5600","ondisk_log_start":"110'5600","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8967","last_scrub_stamp":"2024-09-17T08:14:03.362611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.362611+0000","objects_scrubbed":6991,"log_size":5489,"log_dups_size":0,"ondisk_log_size":5489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:19.667692+0000","scrub_duration":2.7054181179999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313851904,"num_objects":4789,"num_object_clones":0,"num_object_copies":14367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11089,"num_write_kb":508096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"72'14","reported_seq":81,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258096+0000","last_change":"2024-09-17T08:12:46.186990+0000","last_active":"2024-09-17T08:14:33.258096+0000","last_peered":"2024-09-17T08:14:33.258096+0000","last_clean":"2024-09-17T08:14:33.258096+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:14:33.258096+0000","last_undegraded":"2024-09-17T08:14:33.258096+0000","last_fullsized":"2024-09-17T08:14:33.258096+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:12:36.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:36.472360+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":114,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.367441+0000","last_change":"2024-09-17T08:14:33.367441+0000","last_active":"2024-09-17T08:14:33.367441+0000","last_peered":"2024-09-17T08:14:33.367441+0000","last_clean":"2024-09-17T08:14:33.367441+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:14:33.367441+0000","last_undegraded":"2024-09-17T08:14:33.367441+0000","last_fullsized":"2024-09-17T08:14:33.367441+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:14:33.367113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:33.367113+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:43.951220+0000","scrub_duration":0.0052554730000000001,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248393+0000","last_change":"2024-09-17T08:12:38.112848+0000","last_active":"2024-09-17T08:14:33.248393+0000","last_peered":"2024-09-17T08:14:33.248393+0000","last_clean":"2024-09-17T08:14:33.248393+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:14:33.248393+0000","last_undegraded":"2024-09-17T08:14:33.248393+0000","last_fullsized":"2024-09-17T08:14:33.248393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:38.112582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:38.112582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027848069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":368,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270478+0000","last_change":"2024-09-17T08:14:32.196268+0000","last_active":"2024-09-17T08:14:33.270478+0000","last_peered":"2024-09-17T08:14:33.270478+0000","last_clean":"2024-09-17T08:14:33.270478+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:14:33.270478+0000","last_undegraded":"2024-09-17T08:14:33.270478+0000","last_fullsized":"2024-09-17T08:14:33.270478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:14:32.196101+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.196101+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:32.599164+0000","scrub_duration":0.0029103029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.271184+0000","last_change":"2024-09-17T08:14:04.052308+0000","last_active":"2024-09-17T08:14:33.271184+0000","last_peered":"2024-09-17T08:14:33.271184+0000","last_clean":"2024-09-17T08:14:33.271184+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:14:33.271184+0000","last_undegraded":"2024-09-17T08:14:33.271184+0000","last_fullsized":"2024-09-17T08:14:33.271184+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:04.052233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.052233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:07.637042+0000","scrub_duration":0.0041087229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248518+0000","last_change":"2024-09-17T08:13:34.022691+0000","last_active":"2024-09-17T08:14:33.248518+0000","last_peered":"2024-09-17T08:14:33.248518+0000","last_clean":"2024-09-17T08:14:33.248518+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:14:33.248518+0000","last_undegraded":"2024-09-17T08:14:33.248518+0000","last_fullsized":"2024-09-17T08:14:33.248518+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:34.022556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.022556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:36.433149+0000","scrub_duration":0.0065771129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"138'11134","reported_seq":11302,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.589835+0000","last_change":"2024-09-17T08:14:36.586011+0000","last_active":"2024-09-17T08:14:37.589835+0000","last_peered":"2024-09-17T08:14:37.589835+0000","last_clean":"2024-09-17T08:14:37.589835+0000","last_became_active":"2024-09-17T08:12:05.207111+0000","last_became_peered":"2024-09-17T08:12:05.207111+0000","last_unstale":"2024-09-17T08:14:37.589835+0000","last_undegraded":"2024-09-17T08:14:37.589835+0000","last_fullsized":"2024-09-17T08:14:37.589835+0000","mapping_epoch":91,"log_start":"110'5600","ondisk_log_start":"110'5600","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11070","last_scrub_stamp":"2024-09-17T08:14:36.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.585116+0000","objects_scrubbed":4966,"log_size":5534,"log_dups_size":0,"ondisk_log_size":5534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:49.382672+0000","scrub_duration":1.821818428,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317587456,"num_objects":4846,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11134,"num_write_kb":511360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257246+0000","last_change":"2024-09-17T08:12:56.917131+0000","last_active":"2024-09-17T08:14:33.257246+0000","last_peered":"2024-09-17T08:14:33.257246+0000","last_clean":"2024-09-17T08:14:33.257246+0000","last_became_active":"2024-09-17T08:12:46.183692+0000","last_became_peered":"2024-09-17T08:12:46.183692+0000","last_unstale":"2024-09-17T08:14:33.257246+0000","last_undegraded":"2024-09-17T08:14:33.257246+0000","last_fullsized":"2024-09-17T08:14:33.257246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:12:56.916985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.916985+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011500321000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.261848+0000","last_change":"2024-09-17T08:12:49.325243+0000","last_active":"2024-09-17T08:14:33.261848+0000","last_peered":"2024-09-17T08:14:33.261848+0000","last_clean":"2024-09-17T08:14:33.261848+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:14:33.261848+0000","last_undegraded":"2024-09-17T08:14:33.261848+0000","last_fullsized":"2024-09-17T08:14:33.261848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.325122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.325122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003561631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.262734+0000","last_change":"2024-09-17T08:12:50.355594+0000","last_active":"2024-09-17T08:14:33.262734+0000","last_peered":"2024-09-17T08:14:33.262734+0000","last_clean":"2024-09-17T08:14:33.262734+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:14:33.262734+0000","last_undegraded":"2024-09-17T08:14:33.262734+0000","last_fullsized":"2024-09-17T08:14:33.262734+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:50.355473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:50.355473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094222349999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270408+0000","last_change":"2024-09-17T08:13:41.903624+0000","last_active":"2024-09-17T08:14:33.270408+0000","last_peered":"2024-09-17T08:14:33.270408+0000","last_clean":"2024-09-17T08:14:33.270408+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:14:33.270408+0000","last_undegraded":"2024-09-17T08:14:33.270408+0000","last_fullsized":"2024-09-17T08:14:33.270408+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:41.903070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:41.903070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:51.229180+0000","scrub_duration":0.0063924560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"138'11074","reported_seq":11286,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.436917+0000","last_change":"2024-09-17T08:14:36.549130+0000","last_active":"2024-09-17T08:14:37.436917+0000","last_peered":"2024-09-17T08:14:37.436917+0000","last_clean":"2024-09-17T08:14:37.436917+0000","last_became_active":"2024-09-17T08:12:05.203964+0000","last_became_peered":"2024-09-17T08:12:05.203964+0000","last_unstale":"2024-09-17T08:14:37.436917+0000","last_undegraded":"2024-09-17T08:14:37.436917+0000","last_fullsized":"2024-09-17T08:14:37.436917+0000","mapping_epoch":91,"log_start":"106'4500","ondisk_log_start":"106'4500","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11018","last_scrub_stamp":"2024-09-17T08:14:36.548737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.548737+0000","objects_scrubbed":4930,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:51.619556+0000","scrub_duration":1.6783069149999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315883520,"num_objects":4820,"num_object_clones":0,"num_object_copies":14460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11074,"num_write_kb":508608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":121,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264026+0000","last_change":"2024-09-17T08:13:54.660609+0000","last_active":"2024-09-17T08:14:33.264026+0000","last_peered":"2024-09-17T08:14:33.264026+0000","last_clean":"2024-09-17T08:14:33.264026+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:14:33.264026+0000","last_undegraded":"2024-09-17T08:14:33.264026+0000","last_fullsized":"2024-09-17T08:14:33.264026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:13:54.660479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.660479+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.788584+0000","scrub_duration":0.015027664,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257503+0000","last_change":"2024-09-17T08:12:37.930254+0000","last_active":"2024-09-17T08:14:33.257503+0000","last_peered":"2024-09-17T08:14:33.257503+0000","last_clean":"2024-09-17T08:14:33.257503+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:14:33.257503+0000","last_undegraded":"2024-09-17T08:14:33.257503+0000","last_fullsized":"2024-09-17T08:14:33.257503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:37.929953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:37.929953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031686209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.264929+0000","last_change":"2024-09-17T08:13:54.967053+0000","last_active":"2024-09-17T08:14:33.264929+0000","last_peered":"2024-09-17T08:14:33.264929+0000","last_clean":"2024-09-17T08:14:33.264929+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:14:33.264929+0000","last_undegraded":"2024-09-17T08:14:33.264929+0000","last_fullsized":"2024-09-17T08:14:33.264929+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:54.966885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.966885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:09.852302+0000","scrub_duration":0.014432926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.268489+0000","last_change":"2024-09-17T08:12:49.737151+0000","last_active":"2024-09-17T08:14:33.268489+0000","last_peered":"2024-09-17T08:14:33.268489+0000","last_clean":"2024-09-17T08:14:33.268489+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:14:33.268489+0000","last_undegraded":"2024-09-17T08:14:33.268489+0000","last_fullsized":"2024-09-17T08:14:33.268489+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.736793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.736793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004027793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"138'10927","reported_seq":10984,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.308383+0000","last_change":"2024-09-17T08:12:05.200413+0000","last_active":"2024-09-17T08:14:37.308383+0000","last_peered":"2024-09-17T08:14:37.308383+0000","last_clean":"2024-09-17T08:14:37.308383+0000","last_became_active":"2024-09-17T08:12:05.200128+0000","last_became_peered":"2024-09-17T08:12:05.200128+0000","last_unstale":"2024-09-17T08:14:37.308383+0000","last_undegraded":"2024-09-17T08:14:37.308383+0000","last_fullsized":"2024-09-17T08:14:37.308383+0000","mapping_epoch":91,"log_start":"111'5800","ondisk_log_start":"111'5800","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":5127,"log_dups_size":0,"ondisk_log_size":5127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308871168,"num_objects":4713,"num_object_clones":0,"num_object_copies":14139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10927,"num_write_kb":500480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":130,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.247625+0000","last_change":"2024-09-17T08:12:45.062371+0000","last_active":"2024-09-17T08:14:33.247625+0000","last_peered":"2024-09-17T08:14:33.247625+0000","last_clean":"2024-09-17T08:14:33.247625+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:14:33.247625+0000","last_undegraded":"2024-09-17T08:14:33.247625+0000","last_fullsized":"2024-09-17T08:14:33.247625+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:12:45.062287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:45.062287+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020883730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":107,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263449+0000","last_change":"2024-09-17T08:12:37.141169+0000","last_active":"2024-09-17T08:14:33.263449+0000","last_peered":"2024-09-17T08:14:33.263449+0000","last_clean":"2024-09-17T08:14:33.263449+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:14:33.263449+0000","last_undegraded":"2024-09-17T08:14:33.263449+0000","last_fullsized":"2024-09-17T08:14:33.263449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:37.141052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:37.141052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020501897000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":167,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270209+0000","last_change":"2024-09-17T08:13:36.798097+0000","last_active":"2024-09-17T08:14:33.270209+0000","last_peered":"2024-09-17T08:14:33.270209+0000","last_clean":"2024-09-17T08:14:33.270209+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:14:33.270209+0000","last_undegraded":"2024-09-17T08:14:33.270209+0000","last_fullsized":"2024-09-17T08:14:33.270209+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:36.797958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:36.797958+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.004868+0000","scrub_duration":0.0023544389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.247648+0000","last_change":"2024-09-17T08:14:00.164468+0000","last_active":"2024-09-17T08:14:33.247648+0000","last_peered":"2024-09-17T08:14:33.247648+0000","last_clean":"2024-09-17T08:14:33.247648+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:14:33.247648+0000","last_undegraded":"2024-09-17T08:14:33.247648+0000","last_fullsized":"2024-09-17T08:14:33.247648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:00.164369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:00.164369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:07.156293+0000","scrub_duration":0.001471577,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"138'11356","reported_seq":11413,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.461386+0000","last_change":"2024-09-17T08:12:05.209733+0000","last_active":"2024-09-17T08:14:37.461386+0000","last_peered":"2024-09-17T08:14:37.461386+0000","last_clean":"2024-09-17T08:14:37.461386+0000","last_became_active":"2024-09-17T08:12:05.209262+0000","last_became_peered":"2024-09-17T08:12:05.209262+0000","last_unstale":"2024-09-17T08:14:37.461386+0000","last_undegraded":"2024-09-17T08:14:37.461386+0000","last_fullsized":"2024-09-17T08:14:37.461386+0000","mapping_epoch":91,"log_start":"114'6600","ondisk_log_start":"114'6600","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320471040,"num_objects":4890,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11356,"num_write_kb":519872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":128,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257587+0000","last_change":"2024-09-17T08:12:33.348079+0000","last_active":"2024-09-17T08:14:33.257587+0000","last_peered":"2024-09-17T08:14:33.257587+0000","last_clean":"2024-09-17T08:14:33.257587+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:14:33.257587+0000","last_undegraded":"2024-09-17T08:14:33.257587+0000","last_fullsized":"2024-09-17T08:14:33.257587+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:12:33.347961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:33.347961+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006521977,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":125,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258427+0000","last_change":"2024-09-17T08:14:05.379019+0000","last_active":"2024-09-17T08:14:33.258427+0000","last_peered":"2024-09-17T08:14:33.258427+0000","last_clean":"2024-09-17T08:14:33.258427+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:14:33.258427+0000","last_undegraded":"2024-09-17T08:14:33.258427+0000","last_fullsized":"2024-09-17T08:14:33.258427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:14:05.378907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:05.378907+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:08.195877+0000","scrub_duration":0.0051126039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263356+0000","last_change":"2024-09-17T08:12:40.211029+0000","last_active":"2024-09-17T08:14:33.263356+0000","last_peered":"2024-09-17T08:14:33.263356+0000","last_clean":"2024-09-17T08:14:33.263356+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:14:33.263356+0000","last_undegraded":"2024-09-17T08:14:33.263356+0000","last_fullsized":"2024-09-17T08:14:33.263356+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:40.210735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:12:40.210735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073656709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254923+0000","last_change":"2024-09-17T08:12:46.344419+0000","last_active":"2024-09-17T08:14:33.254923+0000","last_peered":"2024-09-17T08:14:33.254923+0000","last_clean":"2024-09-17T08:14:33.254923+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:14:33.254923+0000","last_undegraded":"2024-09-17T08:14:33.254923+0000","last_fullsized":"2024-09-17T08:14:33.254923+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:46.344243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:46.344243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010636784999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":124,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265853+0000","last_change":"2024-09-17T08:13:57.680834+0000","last_active":"2024-09-17T08:14:33.265853+0000","last_peered":"2024-09-17T08:14:33.265853+0000","last_clean":"2024-09-17T08:14:33.265853+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:14:33.265853+0000","last_undegraded":"2024-09-17T08:14:33.265853+0000","last_fullsized":"2024-09-17T08:14:33.265853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:13:57.680700+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.680700+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.392665+0000","scrub_duration":0.031491026999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.280185+0000","last_change":"2024-09-17T08:12:35.149784+0000","last_active":"2024-09-17T08:14:34.280185+0000","last_peered":"2024-09-17T08:14:34.280185+0000","last_clean":"2024-09-17T08:14:34.280185+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:14:34.280185+0000","last_undegraded":"2024-09-17T08:14:34.280185+0000","last_fullsized":"2024-09-17T08:14:34.280185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:35.149490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:35.149490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013490703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":91,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.259099+0000","last_change":"2024-09-17T08:12:35.295415+0000","last_active":"2024-09-17T08:14:33.259099+0000","last_peered":"2024-09-17T08:14:33.259099+0000","last_clean":"2024-09-17T08:14:33.259099+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:14:33.259099+0000","last_undegraded":"2024-09-17T08:14:33.259099+0000","last_fullsized":"2024-09-17T08:14:33.259099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:35.295240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:35.295240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075020399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.261266+0000","last_change":"2024-09-17T08:12:48.286558+0000","last_active":"2024-09-17T08:14:33.261266+0000","last_peered":"2024-09-17T08:14:33.261266+0000","last_clean":"2024-09-17T08:14:33.261266+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:14:33.261266+0000","last_undegraded":"2024-09-17T08:14:33.261266+0000","last_fullsized":"2024-09-17T08:14:33.261266+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:48.286452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:48.286452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028224719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":57,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257710+0000","last_change":"2024-09-17T08:13:36.335503+0000","last_active":"2024-09-17T08:14:33.257710+0000","last_peered":"2024-09-17T08:14:33.257710+0000","last_clean":"2024-09-17T08:14:33.257710+0000","last_became_active":"2024-09-17T08:12:46.186606+0000","last_became_peered":"2024-09-17T08:12:46.186606+0000","last_unstale":"2024-09-17T08:14:33.257710+0000","last_undegraded":"2024-09-17T08:14:33.257710+0000","last_fullsized":"2024-09-17T08:14:33.257710+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:36.335413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:36.335413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:54.653797+0000","scrub_duration":0.0030218530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":113,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.259224+0000","last_change":"2024-09-17T08:13:42.209088+0000","last_active":"2024-09-17T08:14:33.259224+0000","last_peered":"2024-09-17T08:14:33.259224+0000","last_clean":"2024-09-17T08:14:33.259224+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:14:33.259224+0000","last_undegraded":"2024-09-17T08:14:33.259224+0000","last_fullsized":"2024-09-17T08:14:33.259224+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:13:42.208919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:42.208919+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:42.578643+0000","scrub_duration":0.0099883109999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248402+0000","last_change":"2024-09-17T08:12:50.030274+0000","last_active":"2024-09-17T08:14:33.248402+0000","last_peered":"2024-09-17T08:14:33.248402+0000","last_clean":"2024-09-17T08:14:33.248402+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:14:33.248402+0000","last_undegraded":"2024-09-17T08:14:33.248402+0000","last_fullsized":"2024-09-17T08:14:33.248402+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:50.030147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:50.030147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063418609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270493+0000","last_change":"2024-09-17T08:12:56.560628+0000","last_active":"2024-09-17T08:14:33.270493+0000","last_peered":"2024-09-17T08:14:33.270493+0000","last_clean":"2024-09-17T08:14:33.270493+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:14:33.270493+0000","last_undegraded":"2024-09-17T08:14:33.270493+0000","last_fullsized":"2024-09-17T08:14:33.270493+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:56.560534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.560534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001610879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":119,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.259011+0000","last_change":"2024-09-17T08:14:01.307387+0000","last_active":"2024-09-17T08:14:33.259011+0000","last_peered":"2024-09-17T08:14:33.259011+0000","last_clean":"2024-09-17T08:14:33.259011+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:14:33.259011+0000","last_undegraded":"2024-09-17T08:14:33.259011+0000","last_fullsized":"2024-09-17T08:14:33.259011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:14:01.307065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:01.307065+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.149862+0000","scrub_duration":0.01517745,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":116,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248446+0000","last_change":"2024-09-17T08:12:31.043113+0000","last_active":"2024-09-17T08:14:33.248446+0000","last_peered":"2024-09-17T08:14:33.248446+0000","last_clean":"2024-09-17T08:14:33.248446+0000","last_became_active":"2024-09-17T08:11:32.437420+0000","last_became_peered":"2024-09-17T08:11:32.437420+0000","last_unstale":"2024-09-17T08:14:33.248446+0000","last_undegraded":"2024-09-17T08:14:33.248446+0000","last_fullsized":"2024-09-17T08:14:33.248446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:31.043016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:31.043016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021661863999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":166,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248229+0000","last_change":"2024-09-17T08:13:38.125058+0000","last_active":"2024-09-17T08:14:33.248229+0000","last_peered":"2024-09-17T08:14:33.248229+0000","last_clean":"2024-09-17T08:14:33.248229+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:14:33.248229+0000","last_undegraded":"2024-09-17T08:14:33.248229+0000","last_fullsized":"2024-09-17T08:14:33.248229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:38.124922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:38.124922+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:42.669681+0000","scrub_duration":0.012576178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265517+0000","last_change":"2024-09-17T08:13:51.666223+0000","last_active":"2024-09-17T08:14:33.265517+0000","last_peered":"2024-09-17T08:14:33.265517+0000","last_clean":"2024-09-17T08:14:33.265517+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:14:33.265517+0000","last_undegraded":"2024-09-17T08:14:33.265517+0000","last_fullsized":"2024-09-17T08:14:33.265517+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:51.665999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:51.665999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:15.188087+0000","scrub_duration":0.01465203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":120,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248312+0000","last_change":"2024-09-17T08:12:36.171494+0000","last_active":"2024-09-17T08:14:33.248312+0000","last_peered":"2024-09-17T08:14:33.248312+0000","last_clean":"2024-09-17T08:14:33.248312+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:14:33.248312+0000","last_undegraded":"2024-09-17T08:14:33.248312+0000","last_fullsized":"2024-09-17T08:14:33.248312+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:12:36.171403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:36.171403+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.060433385999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.288397+0000","last_change":"2024-09-17T08:12:53.737283+0000","last_active":"2024-09-17T08:14:34.288397+0000","last_peered":"2024-09-17T08:14:34.288397+0000","last_clean":"2024-09-17T08:14:34.288397+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:14:34.288397+0000","last_undegraded":"2024-09-17T08:14:34.288397+0000","last_fullsized":"2024-09-17T08:14:34.288397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:53.737169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:53.737169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099853470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":156,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258644+0000","last_change":"2024-09-17T08:11:49.005429+0000","last_active":"2024-09-17T08:14:33.258644+0000","last_peered":"2024-09-17T08:14:33.258644+0000","last_clean":"2024-09-17T08:14:33.258644+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:14:33.258644+0000","last_undegraded":"2024-09-17T08:14:33.258644+0000","last_fullsized":"2024-09-17T08:14:33.258644+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:11:49.005295+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028223409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258221+0000","last_change":"2024-09-17T08:13:49.324546+0000","last_active":"2024-09-17T08:14:33.258221+0000","last_peered":"2024-09-17T08:14:33.258221+0000","last_clean":"2024-09-17T08:14:33.258221+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:14:33.258221+0000","last_undegraded":"2024-09-17T08:14:33.258221+0000","last_fullsized":"2024-09-17T08:14:33.258221+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.324431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.324431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.346128+0000","scrub_duration":0.0098997129999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:35.753834+0000","last_change":"2024-09-17T08:14:35.754010+0000","last_active":"2024-09-17T08:14:35.753834+0000","last_peered":"2024-09-17T08:14:35.753834+0000","last_clean":"2024-09-17T08:14:35.753834+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:14:35.753834+0000","last_undegraded":"2024-09-17T08:14:35.753834+0000","last_fullsized":"2024-09-17T08:14:35.753834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:14:35.753807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.753807+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:54.769940+0000","scrub_duration":0.028091901999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":57,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.259081+0000","last_change":"2024-09-17T08:12:53.362491+0000","last_active":"2024-09-17T08:14:33.259081+0000","last_peered":"2024-09-17T08:14:33.259081+0000","last_clean":"2024-09-17T08:14:33.259081+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:14:33.259081+0000","last_undegraded":"2024-09-17T08:14:33.259081+0000","last_fullsized":"2024-09-17T08:14:33.259081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:53.362370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:53.362370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011603571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254945+0000","last_change":"2024-09-17T08:13:58.267374+0000","last_active":"2024-09-17T08:14:33.254945+0000","last_peered":"2024-09-17T08:14:33.254945+0000","last_clean":"2024-09-17T08:14:33.254945+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:14:33.254945+0000","last_undegraded":"2024-09-17T08:14:33.254945+0000","last_fullsized":"2024-09-17T08:14:33.254945+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:58.267204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.267204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.437180+0000","scrub_duration":0.0072680419999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.263295+0000","last_change":"2024-09-17T08:13:47.616836+0000","last_active":"2024-09-17T08:14:33.263295+0000","last_peered":"2024-09-17T08:14:33.263295+0000","last_clean":"2024-09-17T08:14:33.263295+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:14:33.263295+0000","last_undegraded":"2024-09-17T08:14:33.263295+0000","last_fullsized":"2024-09-17T08:14:33.263295+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.616708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.616708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.493191+0000","scrub_duration":0.0071559839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":123,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257745+0000","last_change":"2024-09-17T08:12:47.358670+0000","last_active":"2024-09-17T08:14:33.257745+0000","last_peered":"2024-09-17T08:14:33.257745+0000","last_clean":"2024-09-17T08:14:33.257745+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:14:33.257745+0000","last_undegraded":"2024-09-17T08:14:33.257745+0000","last_fullsized":"2024-09-17T08:14:33.257745+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:12:47.358546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:47.358546+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022561080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257435+0000","last_change":"2024-09-17T08:13:56.658599+0000","last_active":"2024-09-17T08:14:33.257435+0000","last_peered":"2024-09-17T08:14:33.257435+0000","last_clean":"2024-09-17T08:14:33.257435+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:14:33.257435+0000","last_undegraded":"2024-09-17T08:14:33.257435+0000","last_fullsized":"2024-09-17T08:14:33.257435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:56.658473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:56.658473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:25.037189+0000","scrub_duration":0.0071637649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257166+0000","last_change":"2024-09-17T08:12:29.772511+0000","last_active":"2024-09-17T08:14:33.257166+0000","last_peered":"2024-09-17T08:14:33.257166+0000","last_clean":"2024-09-17T08:14:33.257166+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:14:33.257166+0000","last_undegraded":"2024-09-17T08:14:33.257166+0000","last_fullsized":"2024-09-17T08:14:33.257166+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:29.772397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:29.772397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0092546309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254969+0000","last_change":"2024-09-17T08:13:37.215381+0000","last_active":"2024-09-17T08:14:33.254969+0000","last_peered":"2024-09-17T08:14:33.254969+0000","last_clean":"2024-09-17T08:14:33.254969+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:14:33.254969+0000","last_undegraded":"2024-09-17T08:14:33.254969+0000","last_fullsized":"2024-09-17T08:14:33.254969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.215249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.215249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:57.556524+0000","scrub_duration":0.035524337000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:35.434487+0000","last_change":"2024-09-17T08:14:35.434487+0000","last_active":"2024-09-17T08:14:35.434487+0000","last_peered":"2024-09-17T08:14:35.434487+0000","last_clean":"2024-09-17T08:14:35.434487+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:14:35.434487+0000","last_undegraded":"2024-09-17T08:14:35.434487+0000","last_fullsized":"2024-09-17T08:14:35.434487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:14:35.433619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.433619+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.671533+0000","scrub_duration":0.0079958110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248467+0000","last_change":"2024-09-17T08:12:33.093326+0000","last_active":"2024-09-17T08:14:33.248467+0000","last_peered":"2024-09-17T08:14:33.248467+0000","last_clean":"2024-09-17T08:14:33.248467+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:14:33.248467+0000","last_undegraded":"2024-09-17T08:14:33.248467+0000","last_fullsized":"2024-09-17T08:14:33.248467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:33.093246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:33.093246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030934140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":58,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257461+0000","last_change":"2024-09-17T08:12:55.327121+0000","last_active":"2024-09-17T08:14:33.257461+0000","last_peered":"2024-09-17T08:14:33.257461+0000","last_clean":"2024-09-17T08:14:33.257461+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:14:33.257461+0000","last_undegraded":"2024-09-17T08:14:33.257461+0000","last_fullsized":"2024-09-17T08:14:33.257461+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:55.327041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.327041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013685990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.270191+0000","last_change":"2024-09-17T08:12:49.448378+0000","last_active":"2024-09-17T08:14:33.270191+0000","last_peered":"2024-09-17T08:14:33.270191+0000","last_clean":"2024-09-17T08:14:33.270191+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:14:33.270191+0000","last_undegraded":"2024-09-17T08:14:33.270191+0000","last_fullsized":"2024-09-17T08:14:33.270191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.448287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.448287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026602499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":124,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294566+0000","last_change":"2024-09-17T08:13:48.179594+0000","last_active":"2024-09-17T08:14:34.294566+0000","last_peered":"2024-09-17T08:14:34.294566+0000","last_clean":"2024-09-17T08:14:34.294566+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:14:34.294566+0000","last_undegraded":"2024-09-17T08:14:34.294566+0000","last_fullsized":"2024-09-17T08:14:34.294566+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:13:48.179449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:48.179449+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.230628+0000","scrub_duration":0.0059153540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":84,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248708+0000","last_change":"2024-09-17T08:12:46.184677+0000","last_active":"2024-09-17T08:14:33.248708+0000","last_peered":"2024-09-17T08:14:33.248708+0000","last_clean":"2024-09-17T08:14:33.248708+0000","last_became_active":"2024-09-17T08:12:46.183935+0000","last_became_peered":"2024-09-17T08:12:46.183935+0000","last_unstale":"2024-09-17T08:14:33.248708+0000","last_undegraded":"2024-09-17T08:14:33.248708+0000","last_fullsized":"2024-09-17T08:14:33.248708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:12:39.460413+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:12:39.460413+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281326+0000","last_change":"2024-09-17T08:13:33.776436+0000","last_active":"2024-09-17T08:14:34.281326+0000","last_peered":"2024-09-17T08:14:34.281326+0000","last_clean":"2024-09-17T08:14:34.281326+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:14:34.281326+0000","last_undegraded":"2024-09-17T08:14:34.281326+0000","last_fullsized":"2024-09-17T08:14:34.281326+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:33.776333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:33.776333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.865312+0000","scrub_duration":0.0016169979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":89,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.254856+0000","last_change":"2024-09-17T08:12:40.337179+0000","last_active":"2024-09-17T08:14:33.254856+0000","last_peered":"2024-09-17T08:14:33.254856+0000","last_clean":"2024-09-17T08:14:33.254856+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:14:33.254856+0000","last_undegraded":"2024-09-17T08:14:33.254856+0000","last_fullsized":"2024-09-17T08:14:33.254856+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:40.336921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:40.336921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014625262999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.271146+0000","last_change":"2024-09-17T08:14:05.076766+0000","last_active":"2024-09-17T08:14:33.271146+0000","last_peered":"2024-09-17T08:14:33.271146+0000","last_clean":"2024-09-17T08:14:33.271146+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:14:33.271146+0000","last_undegraded":"2024-09-17T08:14:33.271146+0000","last_fullsized":"2024-09-17T08:14:33.271146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:14:05.076681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:05.076681+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.231207+0000","scrub_duration":0.0076773190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.247669+0000","last_change":"2024-09-17T08:13:37.089663+0000","last_active":"2024-09-17T08:14:33.247669+0000","last_peered":"2024-09-17T08:14:33.247669+0000","last_clean":"2024-09-17T08:14:33.247669+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:14:33.247669+0000","last_undegraded":"2024-09-17T08:14:33.247669+0000","last_fullsized":"2024-09-17T08:14:33.247669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.089515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.089515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:45.679241+0000","scrub_duration":0.0016432529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":120,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265766+0000","last_change":"2024-09-17T08:12:54.884143+0000","last_active":"2024-09-17T08:14:33.265766+0000","last_peered":"2024-09-17T08:14:33.265766+0000","last_clean":"2024-09-17T08:14:33.265766+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:14:33.265766+0000","last_undegraded":"2024-09-17T08:14:33.265766+0000","last_fullsized":"2024-09-17T08:14:33.265766+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:12:54.883622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:54.883622+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.125166219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258414+0000","last_change":"2024-09-17T08:14:33.194618+0000","last_active":"2024-09-17T08:14:33.258414+0000","last_peered":"2024-09-17T08:14:33.258414+0000","last_clean":"2024-09-17T08:14:33.258414+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:14:33.258414+0000","last_undegraded":"2024-09-17T08:14:33.258414+0000","last_fullsized":"2024-09-17T08:14:33.258414+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:14:33.194496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:33.194496+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:45.510334+0000","scrub_duration":0.0014499820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":100,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.256599+0000","last_change":"2024-09-17T08:12:42.853739+0000","last_active":"2024-09-17T08:14:33.256599+0000","last_peered":"2024-09-17T08:14:33.256599+0000","last_clean":"2024-09-17T08:14:33.256599+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:14:33.256599+0000","last_undegraded":"2024-09-17T08:14:33.256599+0000","last_fullsized":"2024-09-17T08:14:33.256599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.853611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.853611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046570830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.255392+0000","last_change":"2024-09-17T08:14:04.365914+0000","last_active":"2024-09-17T08:14:33.255392+0000","last_peered":"2024-09-17T08:14:33.255392+0000","last_clean":"2024-09-17T08:14:33.255392+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:14:33.255392+0000","last_undegraded":"2024-09-17T08:14:33.255392+0000","last_fullsized":"2024-09-17T08:14:33.255392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:04.365774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.365774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.392202+0000","scrub_duration":0.0083787500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":89,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.248586+0000","last_change":"2024-09-17T08:12:47.057999+0000","last_active":"2024-09-17T08:14:33.248586+0000","last_peered":"2024-09-17T08:14:33.248586+0000","last_clean":"2024-09-17T08:14:33.248586+0000","last_became_active":"2024-09-17T08:12:46.184351+0000","last_became_peered":"2024-09-17T08:12:46.184351+0000","last_unstale":"2024-09-17T08:14:33.248586+0000","last_undegraded":"2024-09-17T08:14:33.248586+0000","last_fullsized":"2024-09-17T08:14:33.248586+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:12:47.057855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:47.057855+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066983809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":103,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.265701+0000","last_change":"2024-09-17T08:12:45.754294+0000","last_active":"2024-09-17T08:14:33.265701+0000","last_peered":"2024-09-17T08:14:33.265701+0000","last_clean":"2024-09-17T08:14:33.265701+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:14:33.265701+0000","last_undegraded":"2024-09-17T08:14:33.265701+0000","last_fullsized":"2024-09-17T08:14:33.265701+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:45.754216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:45.754216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001170784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":101,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258036+0000","last_change":"2024-09-17T08:13:53.296955+0000","last_active":"2024-09-17T08:14:33.258036+0000","last_peered":"2024-09-17T08:14:33.258036+0000","last_clean":"2024-09-17T08:14:33.258036+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:14:33.258036+0000","last_undegraded":"2024-09-17T08:14:33.258036+0000","last_fullsized":"2024-09-17T08:14:33.258036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.296823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.296823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.838694+0000","scrub_duration":0.0020070050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":57,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257761+0000","last_change":"2024-09-17T08:12:46.386848+0000","last_active":"2024-09-17T08:14:33.257761+0000","last_peered":"2024-09-17T08:14:33.257761+0000","last_clean":"2024-09-17T08:14:33.257761+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:14:33.257761+0000","last_undegraded":"2024-09-17T08:14:33.257761+0000","last_fullsized":"2024-09-17T08:14:33.257761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:12:46.386759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014658869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":161,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.261686+0000","last_change":"2024-09-17T08:13:58.963656+0000","last_active":"2024-09-17T08:14:33.261686+0000","last_peered":"2024-09-17T08:14:33.261686+0000","last_clean":"2024-09-17T08:14:33.261686+0000","last_became_active":"2024-09-17T08:11:32.436438+0000","last_became_peered":"2024-09-17T08:11:32.436438+0000","last_unstale":"2024-09-17T08:14:33.261686+0000","last_undegraded":"2024-09-17T08:14:33.261686+0000","last_fullsized":"2024-09-17T08:14:33.261686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:13:58.963319+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.963319+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.183605+0000","scrub_duration":0.010144426,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.256822+0000","last_change":"2024-09-17T08:14:01.636987+0000","last_active":"2024-09-17T08:14:33.256822+0000","last_peered":"2024-09-17T08:14:33.256822+0000","last_clean":"2024-09-17T08:14:33.256822+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:14:33.256822+0000","last_undegraded":"2024-09-17T08:14:33.256822+0000","last_fullsized":"2024-09-17T08:14:33.256822+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:14:01.636849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.255485+0000","scrub_duration":0.010498624999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":90,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.259034+0000","last_change":"2024-09-17T08:12:32.313526+0000","last_active":"2024-09-17T08:14:33.259034+0000","last_peered":"2024-09-17T08:14:33.259034+0000","last_clean":"2024-09-17T08:14:33.259034+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:14:33.259034+0000","last_undegraded":"2024-09-17T08:14:33.259034+0000","last_fullsized":"2024-09-17T08:14:33.259034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:32.313398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:32.313398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011036097999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":106,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.255157+0000","last_change":"2024-09-17T08:12:43.354919+0000","last_active":"2024-09-17T08:14:33.255157+0000","last_peered":"2024-09-17T08:14:33.255157+0000","last_clean":"2024-09-17T08:14:33.255157+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:14:33.255157+0000","last_undegraded":"2024-09-17T08:14:33.255157+0000","last_fullsized":"2024-09-17T08:14:33.255157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.354544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.354544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053342249999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":127,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.258403+0000","last_change":"2024-09-17T08:13:34.227830+0000","last_active":"2024-09-17T08:14:33.258403+0000","last_peered":"2024-09-17T08:14:33.258403+0000","last_clean":"2024-09-17T08:14:33.258403+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:14:33.258403+0000","last_undegraded":"2024-09-17T08:14:33.258403+0000","last_fullsized":"2024-09-17T08:14:33.258403+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:13:34.227679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.227679+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:36.044847+0000","scrub_duration":0.0073082720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":102,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T08:14:33.257915+0000","last_change":"2024-09-17T08:13:39.253357+0000","last_active":"2024-09-17T08:14:33.257915+0000","last_peered":"2024-09-17T08:14:33.257915+0000","last_clean":"2024-09-17T08:14:33.257915+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:14:33.257915+0000","last_undegraded":"2024-09-17T08:14:33.257915+0000","last_fullsized":"2024-09-17T08:14:33.257915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:39.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:39.253277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:44.680616+0000","scrub_duration":0.0099572529999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":5017436184,"num_objects":76561,"num_object_clones":0,"num_object_copies":229683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76561,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178235,"num_write_kb":8153409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15065952256,"data_stored":15065940040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85890,"ondisk_log_size":85890,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":694,"num_read_kb":451,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421,"ondisk_log_size":421,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":275,"num_read_kb":270,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070526,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1647820,"kb_used_data":1554372,"kb_used_omap":1,"kb_used_meta":93438,"kb_avail":92097332,"statfs":{"total":95995035648,"available":94307667968,"internally_reserved":0,"allocated":1591676928,"data_stored":1591359627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":95680971},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":1.0129999999999999,"15min":0.70499999999999996},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":27.751999999999999,"5min":27.751999999999999,"15min":27.751999999999999},"last":1.895},{"interface":"front","average":{"1min":1.3660000000000001,"5min":0.81699999999999995,"15min":0.63700000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":5.2380000000000004,"5min":5.2380000000000004,"15min":5.2380000000000004},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":0.76200000000000001,"15min":0.65600000000000003},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2959999999999998,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.70699999999999996,"15min":0.60299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.443}]},{"osd":2,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":0.74399999999999999,"15min":0.60899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.6210000000000004,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.173,"5min":0.72699999999999998,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.93700000000000006}]},{"osd":3,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":0.82199999999999995,"15min":0.629},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":11.298,"5min":11.298,"15min":11.298},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.6240000000000001,"5min":0.89100000000000001,"15min":0.69299999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.45500000000000002},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.306,"5min":2.21,"15min":2.21},"last":0.253},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.28599999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2470000000000001,"5min":1.526,"15min":1.526},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.3320000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.34799999999999998}]}]},{"osd":6,"up_from":50,"seq":214748364867,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1623040,"kb_used_data":1533688,"kb_used_omap":1,"kb_used_meta":89342,"kb_avail":92122112,"statfs":{"total":95995035648,"available":94333042688,"internally_reserved":0,"allocated":1570496512,"data_stored":1570190783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":91486663},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.3250000000000002,"5min":2.5569999999999999,"15min":1.1950000000000001},"min":{"1min":0.33200000000000002,"5min":0.216,"15min":0.216},"max":{"1min":28.991,"5min":73.757999999999996,"15min":73.757999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":3.044,"5min":2.6970000000000001,"15min":1.226},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":30.620000000000001,"5min":73.811000000000007,"15min":73.811000000000007},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.2450000000000001,"5min":0.96799999999999997,"15min":0.67100000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":28.288,"5min":28.288,"15min":28.288},"last":0.35299999999999998},{"interface":"front","average":{"1min":2.2949999999999999,"5min":0.94499999999999995,"15min":0.67600000000000005},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":28.939,"5min":28.939,"15min":28.939},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.4710000000000001,"15min":0.84399999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.5220000000000002,"5min":68.165000000000006,"15min":68.165000000000006},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.5449999999999999,"15min":0.85099999999999998},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.8840000000000003,"5min":67.997,"15min":67.997},"last":0.59399999999999997}]},{"osd":3,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.9929999999999999,"5min":2.0350000000000001,"15min":1.0529999999999999},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":29.379999999999999,"5min":77.010999999999996,"15min":77.010999999999996},"last":0.496},{"interface":"front","average":{"1min":2.5840000000000001,"5min":2.0089999999999999,"15min":1.0289999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":28.071999999999999,"5min":76.375,"15min":76.375},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.40400000000000003},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.514,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.433,"15min":0.38900000000000001},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.39500000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.673,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48299999999999998,"15min":0.439},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.645,"15min":1.645},"last":0.435}]},{"osd":7,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47099999999999997}]}]},{"osd":5,"up_from":41,"seq":176093659208,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1296960,"kb_used_data":1225656,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":92448192,"statfs":{"total":95995035648,"available":94666948608,"internally_reserved":0,"allocated":1255071744,"data_stored":1254765506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:02 2024","interfaces":[{"interface":"back","average":{"1min":2.4500000000000002,"5min":0.89800000000000002,"15min":0.63600000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":13.685,"5min":13.685,"15min":13.685},"last":0.316},{"interface":"front","average":{"1min":1.8520000000000001,"5min":0.85399999999999998,"15min":0.60999999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":14.503,"5min":14.503,"15min":14.503},"last":0.436}]},{"osd":1,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":0.74099999999999999,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":3.4319999999999999,"5min":5.056,"15min":5.056},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.26,"5min":0.72999999999999998,"15min":0.57599999999999996},"min":{"1min":0.27700000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.0300000000000002,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.66800000000000004,"15min":0.57199999999999995},"min":{"1min":0.25600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.411,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.3320000000000001,"5min":0.752,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5129999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":3.1539999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":0.95799999999999996,"15min":0.64800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.648,"5min":16.648,"15min":16.648},"last":3.1930000000000001},{"interface":"front","average":{"1min":2.0449999999999999,"5min":0.94399999999999995,"15min":0.66600000000000004},"min":{"1min":0.30099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":16.803000000000001,"5min":16.803000000000001,"15min":16.803000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.19,"15min":0.19},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.29199999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:14:28 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.51000000000000001,"15min":0.438},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.48999999999999999,"15min":0.432},"min":{"1min":0.32800000000000001,"5min":0.254,"15min":0.254},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.34999999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.52700000000000002,"15min":0.48799999999999999},"min":{"1min":0.34899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.53300000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.40400000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1637640,"kb_used_data":1548288,"kb_used_omap":1,"kb_used_meta":89342,"kb_avail":92107512,"statfs":{"total":95995035648,"available":94318092288,"internally_reserved":0,"allocated":1585446912,"data_stored":1585133001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":91486662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:40 2024","interfaces":[{"interface":"back","average":{"1min":1.766,"5min":0.80600000000000005,"15min":0.60199999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":10.227,"5min":10.227,"15min":10.227},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":0.78400000000000003,"15min":0.60399999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":10.286,"5min":10.286,"15min":10.286},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:13:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.58999999999999997,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.532},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.65300000000000002,"15min":0.55100000000000005},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.2469999999999999,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:13:40 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.626,"15min":0.56000000000000005},"min":{"1min":0.31,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.625,"15min":0.53700000000000003},"min":{"1min":0.28199999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:13:40 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":0.98999999999999999,"15min":0.67900000000000005},"min":{"1min":0.318,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":27.952000000000002,"5min":27.952000000000002,"15min":27.952000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":2.6829999999999998,"5min":0.98999999999999999,"15min":0.67800000000000005},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":27.919,"5min":27.919,"15min":27.919},"last":0.50600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:14:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.39800000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.42999999999999999,"15min":0.39700000000000002},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.88200000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.439,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.48999999999999999,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.39900000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48199999999999998,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80000000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60999999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117073,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2901332,"kb_used_data":2757580,"kb_used_omap":1,"kb_used_meta":143486,"kb_avail":90843820,"statfs":{"total":95995035648,"available":93024071680,"internally_reserved":0,"allocated":2823761920,"data_stored":2823435198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":146930120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":3.2810000000000001,"5min":1.1879999999999999,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.189,"15min":0.189},"max":{"1min":8.6829999999999998,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":2.5779999999999998,"5min":0.999,"15min":0.58499999999999996},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":10.362,"5min":10.362,"15min":10.362},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":2.9060000000000001,"5min":1.109,"15min":0.64700000000000002},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":8.4410000000000007,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":5.4089999999999998},{"interface":"front","average":{"1min":2.1389999999999998,"5min":0.90500000000000003,"15min":0.58099999999999996},"min":{"1min":0.28000000000000003,"5min":0.188,"15min":0.188},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":5.4489999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":2.762,"5min":1.113,"15min":0.64300000000000002},"min":{"1min":0.28999999999999998,"5min":0.246,"15min":0.246},"max":{"1min":10.786,"5min":10.786,"15min":10.786},"last":0.60699999999999998},{"interface":"front","average":{"1min":3.165,"5min":1.2729999999999999,"15min":0.69699999999999995},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":8.6440000000000001,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":3.448}]},{"osd":4,"last update":"Tue Sep 17 08:13:39 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":0.75900000000000001,"15min":0.61499999999999999},"min":{"1min":0.30199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.6040000000000001,"5min":0.82999999999999996,"15min":0.63700000000000001},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.89400000000000002,"15min":0.67800000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.1829999999999998,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.4410000000000001,"5min":0.93000000000000005,"15min":0.71099999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":5.5730000000000004,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.4770000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":1.7989999999999999,"5min":1.107,"15min":0.748},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.2830000000000004,"5min":10.945,"15min":10.945},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.0189999999999999,"15min":0.71999999999999997},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":4.665,"5min":5.798,"15min":5.798},"last":3.4079999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":0.93100000000000005,"15min":0.70399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.488,"5min":0.83899999999999997,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.1600000000000001,"5min":5.1600000000000001,"15min":5.1600000000000001},"last":0.625}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1964232,"kb_used_data":1856832,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":91780920,"statfs":{"total":95995035648,"available":93983662080,"internally_reserved":0,"allocated":1901395968,"data_stored":1901082046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":3.5089999999999999,"5min":1.089,"15min":0.58099999999999996},"min":{"1min":0.26100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":22.399000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":4.4630000000000001,"5min":1.2889999999999999,"15min":0.66500000000000004},"min":{"1min":0.27500000000000002,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":23.341000000000001,"5min":23.341000000000001,"15min":23.341000000000001},"last":2.25}]},{"osd":1,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":0.61099999999999999,"15min":0.45100000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.374},{"interface":"front","average":{"1min":1.706,"5min":0.68799999999999994,"15min":0.48499999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":3.907,"5min":3.907,"15min":3.907},"last":2.1789999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:14:19 2024","interfaces":[{"interface":"back","average":{"1min":2.0830000000000002,"5min":1.1870000000000001,"15min":0.67100000000000004},"min":{"1min":0.31,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":4.6159999999999997,"5min":15.065,"15min":15.065},"last":3.335},{"interface":"front","average":{"1min":1.738,"5min":1.0609999999999999,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.219,"15min":0.219},"max":{"1min":3.847,"5min":15.555999999999999,"15min":15.555999999999999},"last":3.121}]},{"osd":4,"last update":"Tue Sep 17 08:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":0.98699999999999999,"15min":0.69499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.383,"5min":5.6150000000000002,"15min":5.6150000000000002},"last":3.3700000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.90400000000000003,"15min":0.67700000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.714,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":2.1509999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:14:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.755,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.2149999999999999,"5min":3.2149999999999999,"15min":3.2149999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.6870000000000001,"5min":0.83599999999999997,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":3.4279999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.99199999999999999,"15min":0.72099999999999997},"min":{"1min":0.371,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.3200000000000003,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":4.1029999999999998},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.93400000000000005,"15min":0.72399999999999998},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8479999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":2.2200000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.74299999999999999,"15min":0.62},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":1.6679999999999999,"5min":0.80300000000000005,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":4.133}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1941076,"kb_used_data":1829580,"kb_used_omap":1,"kb_used_meta":111486,"kb_avail":91804076,"statfs":{"total":95995035648,"available":94007373824,"internally_reserved":0,"allocated":1873489920,"data_stored":1873163482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114162118},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:35 2024","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.04,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":9.1549999999999994,"5min":23.834,"15min":23.834},"last":2.0369999999999999},{"interface":"front","average":{"1min":2.516,"5min":1.032,"15min":0.53600000000000003},"min":{"1min":0.20899999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":9.3629999999999995,"5min":22.402999999999999,"15min":22.402999999999999},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:13:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":0.78300000000000003,"15min":0.49199999999999999},"min":{"1min":0.20100000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":7.2649999999999997,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":6.1399999999999997},{"interface":"front","average":{"1min":2.3180000000000001,"5min":0.85199999999999998,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.3860000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:14:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5649999999999999,"5min":1.038,"15min":0.61799999999999999},"min":{"1min":0.23300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.8499999999999996,"5min":7.8499999999999996,"15min":7.8499999999999996},"last":4.617},{"interface":"front","average":{"1min":2.2360000000000002,"5min":1.006,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.189,"15min":0.189},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:13:43 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":0.74399999999999999,"15min":0.60399999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.1769999999999996,"5min":6.1769999999999996,"15min":6.1769999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.462,"5min":0.80700000000000005,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":8.6709999999999994,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.32100000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.75600000000000001,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.0869999999999997,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":4.657},{"interface":"front","average":{"1min":2.0499999999999998,"5min":0.84999999999999998,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:14:22 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":0.88,"15min":0.64700000000000002},"min":{"1min":0.378,"5min":0.318,"15min":0.318},"max":{"1min":2.8999999999999999,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":0.86899999999999999,"15min":0.68700000000000006},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":4.5640000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":0.81799999999999995,"15min":0.65200000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.3380000000000001,"5min":6.3380000000000001,"15min":6.3380000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.6639999999999999,"5min":0.86499999999999999,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.69999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2628556,"kb_used_data":2484996,"kb_used_omap":1,"kb_used_meta":143486,"kb_avail":91116596,"statfs":{"total":95995035648,"available":93303394304,"internally_reserved":0,"allocated":2544635904,"data_stored":2544318311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146930118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:14:32 2024","interfaces":[{"interface":"back","average":{"1min":2.2469999999999999,"5min":1.02,"15min":0.55300000000000005},"min":{"1min":0.27000000000000002,"5min":0.20399999999999999,"15min":0.19900000000000001},"max":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":2.222,"5min":1.179,"15min":0.61799999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":6.4749999999999996,"5min":6.4749999999999996,"15min":6.4749999999999996},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:13:53 2024","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":1.0309999999999999,"15min":0.60099999999999998},"min":{"1min":0.312,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":12.355,"5min":12.355,"15min":12.355},"last":3.617},{"interface":"front","average":{"1min":2.3450000000000002,"5min":0.93500000000000005,"15min":0.58499999999999996},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":7.8639999999999999,"5min":7.8639999999999999,"15min":7.8639999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Tue Sep 17 08:14:22 2024","interfaces":[{"interface":"back","average":{"1min":2.3900000000000001,"5min":1.3009999999999999,"15min":0.73399999999999999},"min":{"1min":0.35799999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.731999999999999,"5min":12.731999999999999,"15min":12.731999999999999},"last":3.5750000000000002},{"interface":"front","average":{"1min":2.6779999999999999,"5min":1.093,"15min":0.68100000000000005},"min":{"1min":0.312,"5min":0.251,"15min":0.251},"max":{"1min":12.965,"5min":12.965,"15min":12.965},"last":0.71999999999999997}]},{"osd":4,"last update":"Tue Sep 17 08:13:36 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":0.83199999999999996,"15min":0.65100000000000002},"min":{"1min":0.34999999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.0359999999999996,"5min":5.0359999999999996,"15min":5.0359999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.28,"5min":0.82599999999999996,"15min":0.66600000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":6.548}]},{"osd":5,"last update":"Tue Sep 17 08:14:07 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":0.84299999999999997,"15min":0.66000000000000003},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.835,"5min":7.835,"15min":7.835},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.71,"5min":0.97399999999999998,"15min":0.71299999999999997},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.7130000000000001,"5min":6.8789999999999996,"15min":6.8789999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":0.93300000000000005,"15min":0.69799999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.8899999999999997,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.038,"15min":0.73999999999999999},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.628}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":0.95599999999999996,"15min":0.69799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":3.6440000000000001},{"interface":"front","average":{"1min":1.6859999999999999,"5min":0.90200000000000002,"15min":0.68400000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514227200,"data_stored":2514223128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1857163264,"data_stored":1857159192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1885732864,"data_stored":1885732864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2816802816,"data_stored":2816802816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1582759936,"data_stored":1582759936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1252196352,"data_stored":1252196352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1568477184,"data_stored":1568473112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1588592640,"data_stored":1588592640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:14:43.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:14:43.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:43.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:43.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:43.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:43.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:43.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:43 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:14:43.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:43 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:14:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:43 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:14:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:43 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:14:43.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.635 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:43.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:43.649 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:43 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:14:43.649 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:43 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:14:43.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:43.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:43.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:43.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:43.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:43.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:43.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:43.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:44.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:44.135 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:44.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.162 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:44.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:44.165 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:44.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:44.288 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:44.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:44.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:44.441 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:44.446 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:44.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:44.449 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[29565]: 7.2 scrub ok 2024-09-17T08:14:44.450 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[29565]: pgmap v192: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:44.450 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[29565]: 7.a scrub ok 2024-09-17T08:14:44.450 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[29565]: 7.b scrub starts 2024-09-17T08:14:44.453 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:44.453 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[40649]: 7.2 scrub ok 2024-09-17T08:14:44.453 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[40649]: pgmap v192: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:44.453 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[40649]: 7.a scrub ok 2024-09-17T08:14:44.453 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:44 smithi079 ceph-mon[40649]: 7.b scrub starts 2024-09-17T08:14:44.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:44.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.566 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:44.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:44 smithi173 ceph-mon[33187]: 7.2 scrub ok 2024-09-17T08:14:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:44 smithi173 ceph-mon[33187]: pgmap v192: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-17T08:14:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:44 smithi173 ceph-mon[33187]: 7.a scrub ok 2024-09-17T08:14:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:44 smithi173 ceph-mon[33187]: 7.b scrub starts 2024-09-17T08:14:44.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:44.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:44.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:44.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:44.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:44.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:45.099 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:21658 7.d 7:b8f077bb:::benchmark_data_smithi079_83551_object21657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 139.383121195, 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.031442645999999998, 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.113 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 21658 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727080+0000", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727136+0000", 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5928999999999999e-05 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727147+0000", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1473e-05 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727153+0000", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0599999999999998e-06 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727188+0000", 2024-09-17T08:14:45.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5267e-05 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727275+0000", 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7330000000000006e-05 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727369+0000", 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3528000000000003e-05 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.728187+0000", 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00081786399999999996 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.731982+0000", 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003795344 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758465+0000", 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.026482793000000001 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758496+0000", 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0979000000000003e-05 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758524+0000", 2024-09-17T08:14:45.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8070000000000001e-05 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:26092 7.d 7:b0769b2a:::benchmark_data_smithi079_83551_object26091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 137.390820494, 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028404776999999999, 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.120 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 26092 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719381+0000", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719497+0000", 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115707 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719504+0000", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7069999999999992e-06 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719510+0000", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7729999999999996e-06 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721232+0000", 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017213860000000001 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721364+0000", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013278500000000001 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721512+0000", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014775899999999999 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.722256+0000", 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074349699999999997 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.724025+0000", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017694690000000001 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747692+0000", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023667157000000001 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747741+0000", 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9162999999999997e-05 2024-09-17T08:14:45.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747787+0000", 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5074999999999998e-05 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28716 7.7 7:edba59f3:::benchmark_data_smithi079_83551_object28715:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 136.134276548, 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.063303783000000002, 2024-09-17T08:14:45.126 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.127 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.127 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.127 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28716 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:14:45.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975925+0000", 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976073+0000", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014827699999999999 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976086+0000", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2918000000000001e-05 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976096+0000", 2024-09-17T08:14:45.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0256e-05 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976145+0000", 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9243999999999998e-05 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976258+0000", 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011240500000000001 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976379+0000", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121612 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.977391+0000", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001011685 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033061+0000", 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055669996999999999 2024-09-17T08:14:45.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039074+0000", 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0060129399999999996 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039129+0000", 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4897e-05 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039229+0000", 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100722 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28722 7.7 7:e67c57be:::benchmark_data_smithi079_83551_object28721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 136.13096232699999, 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.060131948999999997, 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28722 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979239+0000", 2024-09-17T08:14:45.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979357+0000", 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011785099999999999 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979367+0000", 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0438999999999999e-05 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979374+0000", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0400000000000004e-06 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979415+0000", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1279999999999998e-05 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979502+0000", 2024-09-17T08:14:45.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6149999999999993e-05 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979586+0000", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4332999999999994e-05 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.980492+0000", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090632099999999997 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033104+0000", 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.052611766999999997 2024-09-17T08:14:45.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039310+0000", 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0062060930000000002 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039339+0000", 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8484999999999999e-05 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039372+0000", 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3337000000000003e-05 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29213 7.c 7:3e440d7d:::benchmark_data_smithi079_83551_object29212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.861671417, 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057673473000000003, 2024-09-17T08:14:45.141 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29213 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248530+0000", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248900+0000", 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000370066 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248909+0000", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9749999999999996e-06 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248914+0000", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8389999999999997e-06 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248950+0000", 2024-09-17T08:14:45.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.595e-05 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249061+0000", 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110703 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249260+0000", 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000199182 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252421+0000", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031614400000000002 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252454+0000", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3216999999999997e-05 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306147+0000", 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.053692036999999998 2024-09-17T08:14:45.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306179+0000", 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2332000000000001e-05 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306204+0000", 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5429999999999999e-05 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29214 7.d 7:bb0590f4:::benchmark_data_smithi079_83551_object29213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.86127698799999, 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057144951999999999, 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29214 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249567+0000", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064241899999999998 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249580+0000", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2741e-05 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249586+0000", 2024-09-17T08:14:45.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4869999999999999e-06 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249651+0000", 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4521000000000001e-05 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249739+0000", 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8483e-05 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249811+0000", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1867999999999997e-05 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250845+0000", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001034285 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251620+0000", 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00077503800000000003 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.305985+0000", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.054364282 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306043+0000", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8400999999999999e-05 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306070+0000", 2024-09-17T08:14:45.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7157000000000001e-05 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29221 7.c 7:3e2e93ed:::benchmark_data_smithi079_83551_object29220:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.86027397500001, 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.058164050000000002, 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.154 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29221 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249927+0000", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250103+0000", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581399999999999 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250116+0000", 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3127e-05 2024-09-17T08:14:45.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250124+0000", 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7419999999999997e-06 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251107+0000", 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098316799999999998 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251265+0000", 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015859700000000001 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251404+0000", 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000138496 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252513+0000", 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011087200000000001 2024-09-17T08:14:45.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252648+0000", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000135081 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308041+0000", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055393851000000001 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308072+0000", 2024-09-17T08:14:45.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0790999999999997e-05 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308092+0000", 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0032000000000001e-05 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29227 7.7 7:ebe30d19:::benchmark_data_smithi079_83551_object29226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:14:45.160 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.85702015000001, 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.054569469000000002, 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29227 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:14:45.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253181+0000", 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253277+0000", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5808000000000002e-05 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253286+0000", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8939999999999999e-06 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253290+0000", 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3139999999999997e-06 2024-09-17T08:14:45.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253325+0000", 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4477000000000002e-05 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253406+0000", 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1000000000000004e-05 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253499+0000", 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3505999999999995e-05 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.254519+0000", 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00102016 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306321+0000", 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.051802061000000003 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307680+0000", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013587880000000001 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307712+0000", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1399000000000002e-05 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307752+0000", 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9876999999999998e-05 2024-09-17T08:14:45.166 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30807 7.7 7:e1a9b60f:::benchmark_data_smithi079_83551_object30806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08917394100001, 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15767790800000001, 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.167 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30807 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.168 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021027+0000", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021092+0000", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5506000000000006e-05 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021104+0000", 2024-09-17T08:14:45.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1472e-05 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021115+0000", 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1080000000000001e-05 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021160+0000", 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5367000000000001e-05 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021239+0000", 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8437999999999997e-05 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021321+0000", 2024-09-17T08:14:45.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2540999999999995e-05 2024-09-17T08:14:45.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:45.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:45.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022432+0000", 2024-09-17T08:14:45.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001110457 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025758+0000", 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003326552 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178623+0000", 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.152864151 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178669+0000", 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6896e-05 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178706+0000", 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6767000000000002e-05 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.175 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30809 7.d 7:b03b464e:::benchmark_data_smithi079_83551_object30808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08760772700001, 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15486507099999999, 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30809 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022593+0000", 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022677+0000", 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3442999999999994e-05 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022694+0000", 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7031e-05 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022702+0000", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.028e-06 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022744+0000", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2491999999999999e-05 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022838+0000", 2024-09-17T08:14:45.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3326999999999996e-05 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022969+0000", 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131068 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.023921+0000", 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00095237200000000005 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175140+0000", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15121870300000001 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177383+0000", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022433729999999999 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177423+0000", 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9823000000000001e-05 2024-09-17T08:14:45.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177460+0000", 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6634999999999997e-05 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30813 7.7 7:e02af656:::benchmark_data_smithi079_83551_object30812:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.086150105, 2024-09-17T08:14:45.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15475839, 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30813 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024046+0000", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999938 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024174+0000", 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000128133 2024-09-17T08:14:45.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024192+0000", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8287999999999998e-05 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024200+0000", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3900000000000004e-06 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024259+0000", 2024-09-17T08:14:45.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9525999999999999e-05 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024392+0000", 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132946 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024549+0000", 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015661000000000001 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178587+0000", 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.154037904 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178743+0000", 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015638800000000001 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178750+0000", 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5570000000000001e-06 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178783+0000", 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3241999999999998e-05 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178811+0000", 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7668999999999999e-05 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30816 7.d 7:bfa0a9cd:::benchmark_data_smithi079_83551_object30815:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.188 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.085989458, 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.153399859, 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30816 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:14:45.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024212+0000", 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024268+0000", 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5563000000000003e-05 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024283+0000", 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5176e-05 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024290+0000", 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0079999999999996e-06 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.191 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024333+0000", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2933999999999999e-05 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024447+0000", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114708 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024706+0000", 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025843399999999999 2024-09-17T08:14:45.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177353+0000", 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15264659 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177528+0000", 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581300000000001 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177538+0000", 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6139999999999998e-06 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177581+0000", 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2889999999999998e-05 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177613+0000", 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1819999999999997e-05 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.085953975, 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084112399999999, 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024247+0000", 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024253+0000", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3569999999999997e-06 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024263+0000", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0562e-05 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024279+0000", 2024-09-17T08:14:45.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5896000000000002e-05 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025745+0000", 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001465767 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025753+0000", 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9100000000000005e-06 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175089+0000", 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.149336513 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30809 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08563105900001, 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15057779399999999, 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:14:45.199 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024570+0000", 2024-09-17T08:14:45.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024602+0000", 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1566000000000002e-05 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024612+0000", 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9510000000000001e-06 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024618+0000", 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2609999999999999e-06 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175046+0000", 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.150427808 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175130+0000", 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3523000000000007e-05 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175149+0000", 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9442000000000001e-05 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30817 7.c 7:3940e3c2:::benchmark_data_smithi079_83551_object30816:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.085601029, 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15284112799999999, 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.203 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30817 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:14:45.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024590+0000", 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999989 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024982+0000", 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000392063 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024994+0000", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1858000000000001e-05 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025005+0000", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0288999999999999e-05 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025057+0000", 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2791999999999999e-05 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025172+0000", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011491499999999999 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025294+0000", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012128 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174906+0000", 2024-09-17T08:14:45.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961192100000001 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177330+0000", 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002424538 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177361+0000", 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1254999999999998e-05 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177406+0000", 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4196000000000002e-05 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177442+0000", 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6826999999999998e-05 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08440207999999, 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15291164700000001, 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025799+0000", 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025805+0000", 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1129999999999997e-06 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175419+0000", 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961376500000001 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175431+0000", 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2131e-05 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178612+0000", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031804939999999999 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9100000000000002e-06 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178712+0000", 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4178999999999994e-05 2024-09-17T08:14:45.213 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30816 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08368999699999, 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084441900000001, 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026517+0000", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5160000000000002e-06 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174997+0000", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14847996599999999 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175009+0000", 2024-09-17T08:14:45.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1552999999999999e-05 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177340+0000", 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023309110000000002 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177347+0000", 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3560000000000004e-06 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177357+0000", 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7340000000000003e-06 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30820 7.d 7:b5459376:::benchmark_data_smithi079_83551_object30819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 135.08333999600001, 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15420378100000001, 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.218 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30820 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026861+0000", 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026944+0000", 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3053999999999995e-05 2024-09-17T08:14:45.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174921+0000", 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14797671100000001 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174933+0000", 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1926000000000001e-05 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176058+0000", 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011258220000000001 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176161+0000", 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102187 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176275+0000", 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114832 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177625+0000", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001349996 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.179664+0000", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002038467 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.180981+0000", 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013172679999999999 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181030+0000", 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9128999999999997e-05 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181066+0000", 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5720999999999999e-05 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:32607 7.7 7:e4b06c5d:::benchmark_data_smithi079_83551_object32606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 134.138365343, 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.05146564, 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 32607 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971836+0000", 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971928+0000", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1908000000000002e-05 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971941+0000", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3608000000000001e-05 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971948+0000", 2024-09-17T08:14:45.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5150000000000003e-06 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971992+0000", 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3769000000000001e-05 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.972081+0000", 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9649999999999997e-05 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.972166+0000", 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4598000000000006e-05 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973794+0000", 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016286359999999999 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.974962+0000", 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00116788 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023221+0000", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.048258471999999997 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023268+0000", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7465e-05 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023303+0000", 2024-09-17T08:14:45.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4328000000000001e-05 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:33202 7.7 7:e6f9aec6:::benchmark_data_smithi079_83551_object33201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 133.807528362, 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.030856884000000001, 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:14:45.231 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 33202 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:45.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302673+0000", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302828+0000", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015509399999999999 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:45.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302840+0000", 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2853e-05 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302846+0000", 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0370000000000002e-06 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303493+0000", 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064765700000000005 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303615+0000", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012233599999999999 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303760+0000", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014483899999999999 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.304637+0000", 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00087705499999999996 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.309162+0000", 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0045242379999999999 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333437+0000", 2024-09-17T08:14:45.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024274917999999999 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333488+0000", 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1175999999999999e-05 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333531+0000", 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3040999999999997e-05 2024-09-17T08:14:45.237 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.238 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:45.238 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:45.238 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:45.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:45.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:45.299 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:45.372 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:14:45.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:45.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:45.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:14:45.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:14:45.498 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: 7.3 scrub starts 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2658124904' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: 7.3 scrub starts 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2658124904' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:14:45.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:45 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:14:45.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:45.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:45.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:45.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:45.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:45.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: 7.3 scrub starts 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2658124904' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:14:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:45 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:14:45.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:45.832 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:45.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:45.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:45.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:45.970 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:45.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:46.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:46.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:14:46.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:46.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:46.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:46.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:46.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:46.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:46.368 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:46.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:46.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:46.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:46.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:46.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:46.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:46.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:46.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: 7.b scrub ok 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: pgmap v193: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 928 B/s rd, 0 B/s wr, 916 op/s 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:46.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:46 smithi173 ceph-mon[33187]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T08:14:46.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:46.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:46.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: 7.b scrub ok 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: pgmap v193: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 928 B/s rd, 0 B/s wr, 916 op/s 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:14:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[29565]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: 7.b scrub ok 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: pgmap v193: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 928 B/s rd, 0 B/s wr, 916 op/s 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:14:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:46 smithi079 ceph-mon[40649]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T08:14:46.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:46.916 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:46.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:46.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:47.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.056 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:47.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:47.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:47.190 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:47.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:47.292 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:47.320 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:47.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:47.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:47.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:47.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:47.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:47.594 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 7.3 scrub ok 2024-09-17T08:14:47.594 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 5.d deep-scrub starts 2024-09-17T08:14:47.594 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 5.d deep-scrub ok 2024-09-17T08:14:47.594 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 4.5 deep-scrub starts 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 4.5 deep-scrub ok 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:14:47.595 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[40649]: pgmap v195: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 7.3 scrub ok 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 5.d deep-scrub starts 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 5.d deep-scrub ok 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 4.5 deep-scrub starts 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 4.5 deep-scrub ok 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:14:47.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:47 smithi079 ceph-mon[29565]: pgmap v195: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:14:47.598 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:47.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 7.3 scrub ok 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 5.d deep-scrub starts 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 5.d deep-scrub ok 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 4.5 deep-scrub starts 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 4.5 deep-scrub ok 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:14:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:47 smithi173 ceph-mon[33187]: pgmap v195: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:14:47.700 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:47.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:47.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:47.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:47.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.867 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:47.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:47.967 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:47.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:47.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:47.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:48.097 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:48.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.133 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:48.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:48.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:48.248 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24698.0:189555 7.f e139/91 7:feedbb57:::benchmark_data_smithi079_83551_object62155:head v 139'11903, mlcod=139'11901)", 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:48.247393+0000", 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00032287199999999998, 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035029199999999999, 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247393+0000", 2024-09-17T08:14:48.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247393+0000", 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247393+0000", 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:14:48.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247398+0000", 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3229999999999997e-06 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247406+0000", 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3939999999999996e-06 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247411+0000", 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1840000000000001e-06 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247460+0000", 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9382999999999998e-05 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:48.247481+0000", 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0709000000000001e-05 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:14:48.252 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:14:48.253 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2024-09-17T08:14:48.253 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:48.276 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:48.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:48.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:48.406 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:48.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:48.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:14:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:14:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:14:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:14:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:14:48.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:48 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:14:48.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:48.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:48.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:48.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:48.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:48.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:48.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:14:48.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:14:48.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:48 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:14:48.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:48.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:48.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:48.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:48.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:48.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:48.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:49.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:49.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.093 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:49.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:49.193 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:49.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:49.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:49.349 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:49.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:49.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.479 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:49.579 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:49.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:49.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:14:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:14:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:49 smithi173 ceph-mon[33187]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:14:49.729 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:49.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:14:49.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[40649]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:14:49.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:14:49.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:49 smithi079 ceph-mon[29565]: pgmap v196: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:14:49.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:49.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:14:49.857 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:49.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:49.987 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:49.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:49.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:50.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:14:50.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:50.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.118 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:50.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:50.118 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:50.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:50.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:50.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:50.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:50.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:50.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:50.550 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:50.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:50.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:50.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.679 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:50.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:14:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:14:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T08:14:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:14:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:50 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:14:50.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:50.805 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:50.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:50.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:14:50.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:14:50.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T08:14:50.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:14:50.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:50 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:14:50.907 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:50.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:50.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:50.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:51.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:51.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:51.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:51.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:51.202 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:51.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:51.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:51.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:51.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:51.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:51.461 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:51.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:14:51.479 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:14:51.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:51.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:51.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:51.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:51.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:51.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:51 smithi173 ceph-mon[33187]: 4.8 deep-scrub starts 2024-09-17T08:14:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:51 smithi173 ceph-mon[33187]: 4.8 deep-scrub ok 2024-09-17T08:14:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:51 smithi173 ceph-mon[33187]: pgmap v198: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 635 op/s 2024-09-17T08:14:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:51 smithi173 ceph-mon[33187]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:51.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:51.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[29565]: 4.8 deep-scrub starts 2024-09-17T08:14:51.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[29565]: 4.8 deep-scrub ok 2024-09-17T08:14:51.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[29565]: pgmap v198: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 635 op/s 2024-09-17T08:14:51.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[29565]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:51.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[40649]: 4.8 deep-scrub starts 2024-09-17T08:14:51.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[40649]: 4.8 deep-scrub ok 2024-09-17T08:14:51.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[40649]: pgmap v198: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 635 op/s 2024-09-17T08:14:51.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:51 smithi079 ceph-mon[40649]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:14:51.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:51.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:51.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:51.994 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:51.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:51.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:52.096 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:52.126 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:52.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:52.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:52.260 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:52.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:52.300 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-17T08:14:51.297043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4334557486,"num_objects":66328,"num_object_clones":0,"num_object_copies":198984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66328,"num_whiteouts":0,"num_read":1204,"num_read_kb":1050,"num_write":189273,"num_write_kb":8155868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86359,"ondisk_log_size":86359,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13694784,"kb_used_data":12803184,"kb_used_omap":12,"kb_used_meta":891123,"kb_avail":736266432,"statfs":{"total":767960285184,"available":753936826368,"internally_reserved":0,"allocated":13110460416,"data_stored":13107899330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":912510520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,5,10,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-332988416,"num_objects":-5081,"num_object_clones":0,"num_object_copies":-15243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5081,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5081,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003433"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.295426+0000","last_change":"2024-09-17T08:14:03.327276+0000","last_active":"2024-09-17T08:14:34.295426+0000","last_peered":"2024-09-17T08:14:34.295426+0000","last_clean":"2024-09-17T08:14:34.295426+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:14:34.295426+0000","last_undegraded":"2024-09-17T08:14:34.295426+0000","last_fullsized":"2024-09-17T08:14:34.295426+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.327169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.327169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:26.263265+0000","scrub_duration":0.013886001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:36.791881+0000","last_change":"2024-09-17T08:14:36.791881+0000","last_active":"2024-09-17T08:14:36.791881+0000","last_peered":"2024-09-17T08:14:36.791881+0000","last_clean":"2024-09-17T08:14:36.791881+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:14:36.791881+0000","last_undegraded":"2024-09-17T08:14:36.791881+0000","last_fullsized":"2024-09-17T08:14:36.791881+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:36.791645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.791645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:39.635176+0000","scrub_duration":0.013350902,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294645+0000","last_change":"2024-09-17T08:12:03.141440+0000","last_active":"2024-09-17T08:14:34.294645+0000","last_peered":"2024-09-17T08:14:34.294645+0000","last_clean":"2024-09-17T08:14:34.294645+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:14:34.294645+0000","last_undegraded":"2024-09-17T08:14:34.294645+0000","last_fullsized":"2024-09-17T08:14:34.294645+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:12:03.141349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:03.141349+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026318309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263623+0000","last_change":"2024-09-17T08:13:37.294653+0000","last_active":"2024-09-17T08:14:34.263623+0000","last_peered":"2024-09-17T08:14:34.263623+0000","last_clean":"2024-09-17T08:14:34.263623+0000","last_became_active":"2024-09-17T08:09:58.506145+0000","last_became_peered":"2024-09-17T08:09:58.506145+0000","last_unstale":"2024-09-17T08:14:34.263623+0000","last_undegraded":"2024-09-17T08:14:34.263623+0000","last_fullsized":"2024-09-17T08:14:34.263623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.294551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.294551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:45.707563+0000","scrub_duration":0.0012183000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":97,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.352722+0000","last_change":"2024-09-17T08:12:33.271632+0000","last_active":"2024-09-17T08:14:34.352722+0000","last_peered":"2024-09-17T08:14:34.352722+0000","last_clean":"2024-09-17T08:14:34.352722+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:14:34.352722+0000","last_undegraded":"2024-09-17T08:14:34.352722+0000","last_fullsized":"2024-09-17T08:14:34.352722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:33.271466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:33.271466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057842520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":144,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263782+0000","last_change":"2024-09-17T08:13:50.305904+0000","last_active":"2024-09-17T08:14:34.263782+0000","last_peered":"2024-09-17T08:14:34.263782+0000","last_clean":"2024-09-17T08:14:34.263782+0000","last_became_active":"2024-09-17T08:11:32.433797+0000","last_became_peered":"2024-09-17T08:11:32.433797+0000","last_unstale":"2024-09-17T08:14:34.263782+0000","last_undegraded":"2024-09-17T08:14:34.263782+0000","last_fullsized":"2024-09-17T08:14:34.263782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:13:50.305732+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:13:50.305732+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:04.124060+0000","scrub_duration":0.020356910999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274665+0000","last_change":"2024-09-17T08:13:47.758152+0000","last_active":"2024-09-17T08:14:34.274665+0000","last_peered":"2024-09-17T08:14:34.274665+0000","last_clean":"2024-09-17T08:14:34.274665+0000","last_became_active":"2024-09-17T08:09:58.507345+0000","last_became_peered":"2024-09-17T08:09:58.507345+0000","last_unstale":"2024-09-17T08:14:34.274665+0000","last_undegraded":"2024-09-17T08:14:34.274665+0000","last_fullsized":"2024-09-17T08:14:34.274665+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.757894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.757894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:51.494786+0000","scrub_duration":0.00167778,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":112,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263679+0000","last_change":"2024-09-17T08:14:32.183940+0000","last_active":"2024-09-17T08:14:34.263679+0000","last_peered":"2024-09-17T08:14:34.263679+0000","last_clean":"2024-09-17T08:14:34.263679+0000","last_became_active":"2024-09-17T08:10:00.525046+0000","last_became_peered":"2024-09-17T08:10:00.525046+0000","last_unstale":"2024-09-17T08:14:34.263679+0000","last_undegraded":"2024-09-17T08:14:34.263679+0000","last_fullsized":"2024-09-17T08:14:34.263679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:32.183821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.183821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.373034+0000","scrub_duration":0.001688852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.282016+0000","last_change":"2024-09-17T08:13:47.009995+0000","last_active":"2024-09-17T08:14:34.282016+0000","last_peered":"2024-09-17T08:14:34.282016+0000","last_clean":"2024-09-17T08:14:34.282016+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:14:34.282016+0000","last_undegraded":"2024-09-17T08:14:34.282016+0000","last_fullsized":"2024-09-17T08:14:34.282016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.009857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.009857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.755878+0000","scrub_duration":0.002516679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":117,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263699+0000","last_change":"2024-09-17T08:13:40.204062+0000","last_active":"2024-09-17T08:14:34.263699+0000","last_peered":"2024-09-17T08:14:34.263699+0000","last_clean":"2024-09-17T08:14:34.263699+0000","last_became_active":"2024-09-17T08:09:56.503186+0000","last_became_peered":"2024-09-17T08:09:56.503186+0000","last_unstale":"2024-09-17T08:14:34.263699+0000","last_undegraded":"2024-09-17T08:14:34.263699+0000","last_fullsized":"2024-09-17T08:14:34.263699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:13:40.203920+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:13:40.203920+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:47.482974+0000","scrub_duration":0.0058130819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":114,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:44.954450+0000","last_change":"2024-09-17T08:14:44.954450+0000","last_active":"2024-09-17T08:14:44.954450+0000","last_peered":"2024-09-17T08:14:44.954450+0000","last_clean":"2024-09-17T08:14:44.954450+0000","last_became_active":"2024-09-17T08:09:58.513381+0000","last_became_peered":"2024-09-17T08:09:58.513381+0000","last_unstale":"2024-09-17T08:14:44.954450+0000","last_undegraded":"2024-09-17T08:14:44.954450+0000","last_fullsized":"2024-09-17T08:14:44.954450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:44.954375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:44.954375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:10.398031+0000","scrub_duration":0.001471621,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.289482+0000","last_change":"2024-09-17T08:13:53.151626+0000","last_active":"2024-09-17T08:14:34.289482+0000","last_peered":"2024-09-17T08:14:34.289482+0000","last_clean":"2024-09-17T08:14:34.289482+0000","last_became_active":"2024-09-17T08:10:00.516734+0000","last_became_peered":"2024-09-17T08:10:00.516734+0000","last_unstale":"2024-09-17T08:14:34.289482+0000","last_undegraded":"2024-09-17T08:14:34.289482+0000","last_fullsized":"2024-09-17T08:14:34.289482+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.151500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.151500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.803456+0000","scrub_duration":0.0089419549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281890+0000","last_change":"2024-09-17T08:13:57.066226+0000","last_active":"2024-09-17T08:14:34.281890+0000","last_peered":"2024-09-17T08:14:34.281890+0000","last_clean":"2024-09-17T08:14:34.281890+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:14:34.281890+0000","last_undegraded":"2024-09-17T08:14:34.281890+0000","last_fullsized":"2024-09-17T08:14:34.281890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:57.066069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.066069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:59.438287+0000","scrub_duration":0.0018716270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":132,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263949+0000","last_change":"2024-09-17T08:14:34.212267+0000","last_active":"2024-09-17T08:14:34.263949+0000","last_peered":"2024-09-17T08:14:34.263949+0000","last_clean":"2024-09-17T08:14:34.263949+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:14:34.263949+0000","last_undegraded":"2024-09-17T08:14:34.263949+0000","last_fullsized":"2024-09-17T08:14:34.263949+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:14:34.212100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:34.212100+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:35.674727+0000","scrub_duration":0.0066844529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.291572+0000","last_change":"2024-09-17T08:13:54.189394+0000","last_active":"2024-09-17T08:14:34.291572+0000","last_peered":"2024-09-17T08:14:34.291572+0000","last_clean":"2024-09-17T08:14:34.291572+0000","last_became_active":"2024-09-17T08:09:58.512577+0000","last_became_peered":"2024-09-17T08:09:58.512577+0000","last_unstale":"2024-09-17T08:14:34.291572+0000","last_undegraded":"2024-09-17T08:14:34.291572+0000","last_fullsized":"2024-09-17T08:14:34.291572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:54.189279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.189279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.336622+0000","scrub_duration":0.002245637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.288497+0000","last_change":"2024-09-17T08:13:40.588846+0000","last_active":"2024-09-17T08:14:34.288497+0000","last_peered":"2024-09-17T08:14:34.288497+0000","last_clean":"2024-09-17T08:14:34.288497+0000","last_became_active":"2024-09-17T08:10:00.520594+0000","last_became_peered":"2024-09-17T08:10:00.520594+0000","last_unstale":"2024-09-17T08:14:34.288497+0000","last_undegraded":"2024-09-17T08:14:34.288497+0000","last_fullsized":"2024-09-17T08:14:34.288497+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:40.588715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:40.588715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:08.948015+0000","scrub_duration":0.0021501789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"139'11981","reported_seq":12150,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.507884+0000","last_change":"2024-09-17T08:14:36.071289+0000","last_active":"2024-09-17T08:14:47.507884+0000","last_peered":"2024-09-17T08:14:47.507884+0000","last_clean":"2024-09-17T08:14:47.507884+0000","last_became_active":"2024-09-17T08:12:05.207481+0000","last_became_peered":"2024-09-17T08:12:05.207481+0000","last_unstale":"2024-09-17T08:14:47.507884+0000","last_undegraded":"2024-09-17T08:14:47.507884+0000","last_fullsized":"2024-09-17T08:14:47.507884+0000","mapping_epoch":91,"log_start":"108'5400","ondisk_log_start":"108'5400","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11226","last_scrub_stamp":"2024-09-17T08:14:36.068645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.068645+0000","objects_scrubbed":5035,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:05.867385+0000","scrub_duration":2.2070316010000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276627456,"num_objects":4221,"num_object_clones":0,"num_object_copies":12663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11981,"num_write_kb":518464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":134,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264568+0000","last_change":"2024-09-17T08:13:33.288306+0000","last_active":"2024-09-17T08:14:34.264568+0000","last_peered":"2024-09-17T08:14:34.264568+0000","last_clean":"2024-09-17T08:14:34.264568+0000","last_became_active":"2024-09-17T08:09:56.497454+0000","last_became_peered":"2024-09-17T08:09:56.497454+0000","last_unstale":"2024-09-17T08:14:34.264568+0000","last_undegraded":"2024-09-17T08:14:34.264568+0000","last_fullsized":"2024-09-17T08:14:34.264568+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:13:33.288160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:33.288160+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:58.878404+0000","scrub_duration":0.0089040130000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281957+0000","last_change":"2024-09-17T08:13:46.010229+0000","last_active":"2024-09-17T08:14:34.281957+0000","last_peered":"2024-09-17T08:14:34.281957+0000","last_clean":"2024-09-17T08:14:34.281957+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:14:34.281957+0000","last_undegraded":"2024-09-17T08:14:34.281957+0000","last_fullsized":"2024-09-17T08:14:34.281957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:46.010103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:46.010103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:52.201701+0000","scrub_duration":0.0016522850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274949+0000","last_change":"2024-09-17T08:12:48.882990+0000","last_active":"2024-09-17T08:14:34.274949+0000","last_peered":"2024-09-17T08:14:34.274949+0000","last_clean":"2024-09-17T08:14:34.274949+0000","last_became_active":"2024-09-17T08:09:58.506355+0000","last_became_peered":"2024-09-17T08:09:58.506355+0000","last_unstale":"2024-09-17T08:14:34.274949+0000","last_undegraded":"2024-09-17T08:14:34.274949+0000","last_fullsized":"2024-09-17T08:14:34.274949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:48.881035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:48.881035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014848926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.501757+0000","last_change":"2024-09-17T08:12:43.301341+0000","last_active":"2024-09-17T08:14:37.501757+0000","last_peered":"2024-09-17T08:14:37.501757+0000","last_clean":"2024-09-17T08:14:37.501757+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:14:37.501757+0000","last_undegraded":"2024-09-17T08:14:37.501757+0000","last_fullsized":"2024-09-17T08:14:37.501757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.301210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.301210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005881786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"139'11860","reported_seq":12029,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.650917+0000","last_change":"2024-09-17T08:14:36.164221+0000","last_active":"2024-09-17T08:14:47.650917+0000","last_peered":"2024-09-17T08:14:47.650917+0000","last_clean":"2024-09-17T08:14:47.650917+0000","last_became_active":"2024-09-17T08:12:05.207933+0000","last_became_peered":"2024-09-17T08:12:05.207933+0000","last_unstale":"2024-09-17T08:14:47.650917+0000","last_undegraded":"2024-09-17T08:14:47.650917+0000","last_fullsized":"2024-09-17T08:14:47.650917+0000","mapping_epoch":91,"log_start":"114'6400","ondisk_log_start":"114'6400","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11122","last_scrub_stamp":"2024-09-17T08:14:36.163963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.163963+0000","objects_scrubbed":5049,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:39.578774+0000","scrub_duration":2.3961925420000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":278265856,"num_objects":4246,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11860,"num_write_kb":515392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":132,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.199095+0000","last_change":"2024-09-17T08:14:37.199095+0000","last_active":"2024-09-17T08:14:37.199095+0000","last_peered":"2024-09-17T08:14:37.199095+0000","last_clean":"2024-09-17T08:14:37.199095+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:14:37.199095+0000","last_undegraded":"2024-09-17T08:14:37.199095+0000","last_fullsized":"2024-09-17T08:14:37.199095+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:14:37.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:37.198955+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:46.937610+0000","scrub_duration":0.0087823410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290467+0000","last_change":"2024-09-17T08:12:39.889514+0000","last_active":"2024-09-17T08:14:34.290467+0000","last_peered":"2024-09-17T08:14:34.290467+0000","last_clean":"2024-09-17T08:14:34.290467+0000","last_became_active":"2024-09-17T08:09:54.497113+0000","last_became_peered":"2024-09-17T08:09:54.497113+0000","last_unstale":"2024-09-17T08:14:34.290467+0000","last_undegraded":"2024-09-17T08:14:34.290467+0000","last_fullsized":"2024-09-17T08:14:34.290467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:39.889220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:39.889220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033635470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":182,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.993854+0000","last_change":"2024-09-17T08:14:45.994439+0000","last_active":"2024-09-17T08:14:45.993854+0000","last_peered":"2024-09-17T08:14:45.993854+0000","last_clean":"2024-09-17T08:14:45.993854+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:14:45.993854+0000","last_undegraded":"2024-09-17T08:14:45.993854+0000","last_fullsized":"2024-09-17T08:14:45.993854+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:14:45.993827+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:14:45.993827+0000","last_clean_scrub_stamp":"2024-09-17T08:14:45.993827+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:48.282684+0000","scrub_duration":0.016070165000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264469+0000","last_change":"2024-09-17T08:13:59.358631+0000","last_active":"2024-09-17T08:14:34.264469+0000","last_peered":"2024-09-17T08:14:34.264469+0000","last_clean":"2024-09-17T08:14:34.264469+0000","last_became_active":"2024-09-17T08:10:00.523543+0000","last_became_peered":"2024-09-17T08:10:00.523543+0000","last_unstale":"2024-09-17T08:14:34.264469+0000","last_undegraded":"2024-09-17T08:14:34.264469+0000","last_fullsized":"2024-09-17T08:14:34.264469+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:59.358484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:13:59.358484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:09.991516+0000","scrub_duration":0.0089018259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"139'12132","reported_seq":12196,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.545487+0000","last_change":"2024-09-17T08:12:46.191819+0000","last_active":"2024-09-17T08:14:47.545487+0000","last_peered":"2024-09-17T08:14:47.545487+0000","last_clean":"2024-09-17T08:14:47.545487+0000","last_became_active":"2024-09-17T08:12:46.185461+0000","last_became_peered":"2024-09-17T08:12:46.185461+0000","last_unstale":"2024-09-17T08:14:47.545487+0000","last_undegraded":"2024-09-17T08:14:47.545487+0000","last_fullsized":"2024-09-17T08:14:47.545487+0000","mapping_epoch":103,"log_start":"120'7400","ondisk_log_start":"120'7400","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275382272,"num_objects":4202,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12132,"num_write_kb":522688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":138,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274860+0000","last_change":"2024-09-17T08:12:51.952863+0000","last_active":"2024-09-17T08:14:34.274860+0000","last_peered":"2024-09-17T08:14:34.274860+0000","last_clean":"2024-09-17T08:14:34.274860+0000","last_became_active":"2024-09-17T08:09:56.502162+0000","last_became_peered":"2024-09-17T08:09:56.502162+0000","last_unstale":"2024-09-17T08:14:34.274860+0000","last_undegraded":"2024-09-17T08:14:34.274860+0000","last_fullsized":"2024-09-17T08:14:34.274860+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:12:51.952706+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021231191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274842+0000","last_change":"2024-09-17T08:13:58.754546+0000","last_active":"2024-09-17T08:14:34.274842+0000","last_peered":"2024-09-17T08:14:34.274842+0000","last_clean":"2024-09-17T08:14:34.274842+0000","last_became_active":"2024-09-17T08:09:54.497444+0000","last_became_peered":"2024-09-17T08:09:54.497444+0000","last_unstale":"2024-09-17T08:14:34.274842+0000","last_undegraded":"2024-09-17T08:14:34.274842+0000","last_fullsized":"2024-09-17T08:14:34.274842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:58.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:25.006130+0000","scrub_duration":0.050368468,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":170,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.883230+0000","last_change":"2024-09-17T08:13:43.244011+0000","last_active":"2024-09-17T08:14:42.883230+0000","last_peered":"2024-09-17T08:14:42.883230+0000","last_clean":"2024-09-17T08:14:42.883230+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:14:42.883230+0000","last_undegraded":"2024-09-17T08:14:42.883230+0000","last_fullsized":"2024-09-17T08:14:42.883230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:43.243896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:43.243896+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:52.639156+0000","scrub_duration":0.0044011149999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290076+0000","last_change":"2024-09-17T08:13:57.216118+0000","last_active":"2024-09-17T08:14:34.290076+0000","last_peered":"2024-09-17T08:14:34.290076+0000","last_clean":"2024-09-17T08:14:34.290076+0000","last_became_active":"2024-09-17T08:10:00.517590+0000","last_became_peered":"2024-09-17T08:10:00.517590+0000","last_unstale":"2024-09-17T08:14:34.290076+0000","last_undegraded":"2024-09-17T08:14:34.290076+0000","last_fullsized":"2024-09-17T08:14:34.290076+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:57.216011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.216011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.931644+0000","scrub_duration":0.0045278590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"139'11655","reported_seq":11721,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.351535+0000","last_change":"2024-09-17T08:14:34.806303+0000","last_active":"2024-09-17T08:14:47.351535+0000","last_peered":"2024-09-17T08:14:47.351535+0000","last_clean":"2024-09-17T08:14:47.351535+0000","last_became_active":"2024-09-17T08:12:46.183640+0000","last_became_peered":"2024-09-17T08:12:46.183640+0000","last_unstale":"2024-09-17T08:14:47.351535+0000","last_undegraded":"2024-09-17T08:14:47.351535+0000","last_fullsized":"2024-09-17T08:14:47.351535+0000","mapping_epoch":103,"log_start":"115'6500","ondisk_log_start":"115'6500","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":5155,"log_dups_size":0,"ondisk_log_size":5155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267780120,"num_objects":4087,"num_object_clones":0,"num_object_copies":12261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11655,"num_write_kb":503681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"72'11","reported_seq":125,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264766+0000","last_change":"2024-09-17T08:13:51.259561+0000","last_active":"2024-09-17T08:14:34.264766+0000","last_peered":"2024-09-17T08:14:34.264766+0000","last_clean":"2024-09-17T08:14:34.264766+0000","last_became_active":"2024-09-17T08:09:56.500115+0000","last_became_peered":"2024-09-17T08:09:56.500115+0000","last_unstale":"2024-09-17T08:14:34.264766+0000","last_undegraded":"2024-09-17T08:14:34.264766+0000","last_fullsized":"2024-09-17T08:14:34.264766+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:13:51.259464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:51.259464+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.498439+0000","scrub_duration":0.0023729620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281475+0000","last_change":"2024-09-17T08:13:49.008226+0000","last_active":"2024-09-17T08:14:34.281475+0000","last_peered":"2024-09-17T08:14:34.281475+0000","last_clean":"2024-09-17T08:14:34.281475+0000","last_became_active":"2024-09-17T08:09:54.506592+0000","last_became_peered":"2024-09-17T08:09:54.506592+0000","last_unstale":"2024-09-17T08:14:34.281475+0000","last_undegraded":"2024-09-17T08:14:34.281475+0000","last_fullsized":"2024-09-17T08:14:34.281475+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.008096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.008096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.255380+0000","scrub_duration":0.010394446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290306+0000","last_change":"2024-09-17T08:12:42.124247+0000","last_active":"2024-09-17T08:14:34.290306+0000","last_peered":"2024-09-17T08:14:34.290306+0000","last_clean":"2024-09-17T08:14:34.290306+0000","last_became_active":"2024-09-17T08:09:58.505823+0000","last_became_peered":"2024-09-17T08:09:58.505823+0000","last_unstale":"2024-09-17T08:14:34.290306+0000","last_undegraded":"2024-09-17T08:14:34.290306+0000","last_fullsized":"2024-09-17T08:14:34.290306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.124120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.124120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001609774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":91,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290695+0000","last_change":"2024-09-17T08:12:00.142321+0000","last_active":"2024-09-17T08:14:34.290695+0000","last_peered":"2024-09-17T08:14:34.290695+0000","last_clean":"2024-09-17T08:14:34.290695+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:14:34.290695+0000","last_undegraded":"2024-09-17T08:14:34.290695+0000","last_fullsized":"2024-09-17T08:14:34.290695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:00.142246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:00.142246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001285509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"139'11595","reported_seq":11753,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.517952+0000","last_change":"2024-09-17T08:14:42.713759+0000","last_active":"2024-09-17T08:14:47.517952+0000","last_peered":"2024-09-17T08:14:47.517952+0000","last_clean":"2024-09-17T08:14:47.517952+0000","last_became_active":"2024-09-17T08:12:05.208839+0000","last_became_peered":"2024-09-17T08:12:05.208839+0000","last_unstale":"2024-09-17T08:14:47.517952+0000","last_undegraded":"2024-09-17T08:14:47.517952+0000","last_fullsized":"2024-09-17T08:14:47.517952+0000","mapping_epoch":91,"log_start":"119'6900","ondisk_log_start":"119'6900","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11323","last_scrub_stamp":"2024-09-17T08:14:42.713564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:42.713564+0000","objects_scrubbed":4364,"log_size":4695,"log_dups_size":0,"ondisk_log_size":4695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:45.958230+0000","scrub_duration":2.3382766560000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263258112,"num_objects":4017,"num_object_clones":0,"num_object_copies":12051,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11595,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":127,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263684+0000","last_change":"2024-09-17T08:13:34.301444+0000","last_active":"2024-09-17T08:14:34.263684+0000","last_peered":"2024-09-17T08:14:34.263684+0000","last_clean":"2024-09-17T08:14:34.263684+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:14:34.263684+0000","last_undegraded":"2024-09-17T08:14:34.263684+0000","last_fullsized":"2024-09-17T08:14:34.263684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:13:34.301346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.301346+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.346147+0000","scrub_duration":0.0022927440000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:38.159618+0000","last_change":"2024-09-17T08:14:38.159618+0000","last_active":"2024-09-17T08:14:38.159618+0000","last_peered":"2024-09-17T08:14:38.159618+0000","last_clean":"2024-09-17T08:14:38.159618+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:14:38.159618+0000","last_undegraded":"2024-09-17T08:14:38.159618+0000","last_fullsized":"2024-09-17T08:14:38.159618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:38.159509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:38.159509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:43.492339+0000","scrub_duration":0.0051691410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'8","reported_seq":173,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.883551+0000","last_change":"2024-09-17T08:13:55.128984+0000","last_active":"2024-09-17T08:14:42.883551+0000","last_peered":"2024-09-17T08:14:42.883551+0000","last_clean":"2024-09-17T08:14:42.883551+0000","last_became_active":"2024-09-17T08:09:58.512854+0000","last_became_peered":"2024-09-17T08:09:58.512854+0000","last_unstale":"2024-09-17T08:14:42.883551+0000","last_undegraded":"2024-09-17T08:14:42.883551+0000","last_fullsized":"2024-09-17T08:14:42.883551+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:55.128587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:55.128587+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:03.969779+0000","scrub_duration":0.0087543629999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":113,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:46.983151+0000","last_change":"2024-09-17T08:14:46.983248+0000","last_active":"2024-09-17T08:14:46.983151+0000","last_peered":"2024-09-17T08:14:46.983151+0000","last_clean":"2024-09-17T08:14:46.983151+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:14:46.983151+0000","last_undegraded":"2024-09-17T08:14:46.983151+0000","last_fullsized":"2024-09-17T08:14:46.983151+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:46.983131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:46.983131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:56.154143+0000","scrub_duration":0.0016336930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"139'11905","reported_seq":12058,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.531592+0000","last_change":"2024-09-17T08:14:45.967112+0000","last_active":"2024-09-17T08:14:47.531592+0000","last_peered":"2024-09-17T08:14:47.531592+0000","last_clean":"2024-09-17T08:14:47.531592+0000","last_became_active":"2024-09-17T08:12:05.209256+0000","last_became_peered":"2024-09-17T08:12:05.209256+0000","last_unstale":"2024-09-17T08:14:47.531592+0000","last_undegraded":"2024-09-17T08:14:47.531592+0000","last_fullsized":"2024-09-17T08:14:47.531592+0000","mapping_epoch":91,"log_start":"119'7200","ondisk_log_start":"119'7200","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11806","last_scrub_stamp":"2024-09-17T08:14:45.966913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:45.966913+0000","objects_scrubbed":4266,"log_size":4705,"log_dups_size":0,"ondisk_log_size":4705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:07.387800+0000","scrub_duration":1.553772436,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269680640,"num_objects":4115,"num_object_clones":0,"num_object_copies":12345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":512640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":138,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.295833+0000","last_change":"2024-09-17T08:12:56.366089+0000","last_active":"2024-09-17T08:14:34.295833+0000","last_peered":"2024-09-17T08:14:34.295833+0000","last_clean":"2024-09-17T08:14:34.295833+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:14:34.295833+0000","last_undegraded":"2024-09-17T08:14:34.295833+0000","last_fullsized":"2024-09-17T08:14:34.295833+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:12:56.365721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.365721+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012144524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274389+0000","last_change":"2024-09-17T08:14:03.639239+0000","last_active":"2024-09-17T08:14:34.274389+0000","last_peered":"2024-09-17T08:14:34.274389+0000","last_clean":"2024-09-17T08:14:34.274389+0000","last_became_active":"2024-09-17T08:09:54.498352+0000","last_became_peered":"2024-09-17T08:09:54.498352+0000","last_unstale":"2024-09-17T08:14:34.274389+0000","last_undegraded":"2024-09-17T08:14:34.274389+0000","last_fullsized":"2024-09-17T08:14:34.274389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.639130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.639130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.754192+0000","scrub_duration":0.0025313649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263587+0000","last_change":"2024-09-17T08:13:52.280824+0000","last_active":"2024-09-17T08:14:34.263587+0000","last_peered":"2024-09-17T08:14:34.263587+0000","last_clean":"2024-09-17T08:14:34.263587+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:14:34.263587+0000","last_undegraded":"2024-09-17T08:14:34.263587+0000","last_fullsized":"2024-09-17T08:14:34.263587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:52.280710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:13:52.280710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:54.376108+0000","scrub_duration":0.0055970719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263865+0000","last_change":"2024-09-17T08:13:46.308939+0000","last_active":"2024-09-17T08:14:34.263865+0000","last_peered":"2024-09-17T08:14:34.263865+0000","last_clean":"2024-09-17T08:14:34.263865+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:14:34.263865+0000","last_undegraded":"2024-09-17T08:14:34.263865+0000","last_fullsized":"2024-09-17T08:14:34.263865+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:46.308829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:46.308829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:48.009682+0000","scrub_duration":0.0085156869999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"139'13691","reported_seq":14047,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.284005+0000","last_change":"2024-09-17T08:14:32.435613+0000","last_active":"2024-09-17T08:14:47.284005+0000","last_peered":"2024-09-17T08:14:47.284005+0000","last_clean":"2024-09-17T08:14:47.284005+0000","last_became_active":"2024-09-17T08:12:05.208659+0000","last_became_peered":"2024-09-17T08:12:05.208659+0000","last_unstale":"2024-09-17T08:14:47.284005+0000","last_undegraded":"2024-09-17T08:14:47.284005+0000","last_fullsized":"2024-09-17T08:14:47.284005+0000","mapping_epoch":91,"log_start":"125'8001","ondisk_log_start":"125'8001","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'12787","last_scrub_stamp":"2024-09-17T08:14:32.435450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.435450+0000","objects_scrubbed":4928,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:52.936186+0000","scrub_duration":1.1622392939999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261423104,"num_objects":3989,"num_object_clones":0,"num_object_copies":11967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11535,"num_write_kb":496768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":153,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:46.714494+0000","last_change":"2024-09-17T08:14:46.714607+0000","last_active":"2024-09-17T08:14:46.714494+0000","last_peered":"2024-09-17T08:14:46.714494+0000","last_clean":"2024-09-17T08:14:46.714494+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:14:46.714494+0000","last_undegraded":"2024-09-17T08:14:46.714494+0000","last_fullsized":"2024-09-17T08:14:46.714494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:14:46.714472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:46.714472+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:04.921737+0000","scrub_duration":0.0080937649999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.282087+0000","last_change":"2024-09-17T08:12:43.593151+0000","last_active":"2024-09-17T08:14:34.282087+0000","last_peered":"2024-09-17T08:14:34.282087+0000","last_clean":"2024-09-17T08:14:34.282087+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:14:34.282087+0000","last_undegraded":"2024-09-17T08:14:34.282087+0000","last_fullsized":"2024-09-17T08:14:34.282087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:12:43.592847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.592847+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024350980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":59,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.286783+0000","last_change":"2024-09-17T08:13:59.677721+0000","last_active":"2024-09-17T08:14:34.286783+0000","last_peered":"2024-09-17T08:14:34.286783+0000","last_clean":"2024-09-17T08:14:34.286783+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:14:34.286783+0000","last_undegraded":"2024-09-17T08:14:34.286783+0000","last_fullsized":"2024-09-17T08:14:34.286783+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:59.677603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:59.677603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.145100+0000","scrub_duration":0.003242782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274598+0000","last_change":"2024-09-17T08:12:43.884405+0000","last_active":"2024-09-17T08:14:34.274598+0000","last_peered":"2024-09-17T08:14:34.274598+0000","last_clean":"2024-09-17T08:14:34.274598+0000","last_became_active":"2024-09-17T08:10:00.597846+0000","last_became_peered":"2024-09-17T08:10:00.597846+0000","last_unstale":"2024-09-17T08:14:34.274598+0000","last_undegraded":"2024-09-17T08:14:34.274598+0000","last_fullsized":"2024-09-17T08:14:34.274598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.884245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.884245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044087800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"139'11701","reported_seq":11916,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.242069+0000","last_change":"2024-09-17T08:13:58.900421+0000","last_active":"2024-09-17T08:14:47.242069+0000","last_peered":"2024-09-17T08:14:47.242069+0000","last_clean":"2024-09-17T08:14:47.242069+0000","last_became_active":"2024-09-17T08:12:05.206361+0000","last_became_peered":"2024-09-17T08:12:05.206361+0000","last_unstale":"2024-09-17T08:14:47.242069+0000","last_undegraded":"2024-09-17T08:14:47.242069+0000","last_fullsized":"2024-09-17T08:14:47.242069+0000","mapping_epoch":91,"log_start":"114'6400","ondisk_log_start":"114'6400","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8703","last_scrub_stamp":"2024-09-17T08:13:58.900297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.900297+0000","objects_scrubbed":7257,"log_size":5301,"log_dups_size":0,"ondisk_log_size":5301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:59.838343+0000","scrub_duration":1.5784367939999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276234240,"num_objects":4215,"num_object_clones":0,"num_object_copies":12645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11701,"num_write_kb":509312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"72'10","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290344+0000","last_change":"2024-09-17T08:12:53.896268+0000","last_active":"2024-09-17T08:14:34.290344+0000","last_peered":"2024-09-17T08:14:34.290344+0000","last_clean":"2024-09-17T08:14:34.290344+0000","last_became_active":"2024-09-17T08:09:56.500060+0000","last_became_peered":"2024-09-17T08:09:56.500060+0000","last_unstale":"2024-09-17T08:14:34.290344+0000","last_undegraded":"2024-09-17T08:14:34.290344+0000","last_fullsized":"2024-09-17T08:14:34.290344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:12:53.894036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:53.894036+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022172299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.275087+0000","last_change":"2024-09-17T08:12:32.839208+0000","last_active":"2024-09-17T08:14:34.275087+0000","last_peered":"2024-09-17T08:14:34.275087+0000","last_clean":"2024-09-17T08:14:34.275087+0000","last_became_active":"2024-09-17T08:09:54.496231+0000","last_became_peered":"2024-09-17T08:09:54.496231+0000","last_unstale":"2024-09-17T08:14:34.275087+0000","last_undegraded":"2024-09-17T08:14:34.275087+0000","last_fullsized":"2024-09-17T08:14:34.275087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:32.838985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:32.838985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088459530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":170,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.883154+0000","last_change":"2024-09-17T08:12:51.719764+0000","last_active":"2024-09-17T08:14:42.883154+0000","last_peered":"2024-09-17T08:14:42.883154+0000","last_clean":"2024-09-17T08:14:42.883154+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:14:42.883154+0000","last_undegraded":"2024-09-17T08:14:42.883154+0000","last_fullsized":"2024-09-17T08:14:42.883154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:12:51.719648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:51.719648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007910116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.284299+0000","last_change":"2024-09-17T08:13:53.683916+0000","last_active":"2024-09-17T08:14:34.284299+0000","last_peered":"2024-09-17T08:14:34.284299+0000","last_clean":"2024-09-17T08:14:34.284299+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:14:34.284299+0000","last_undegraded":"2024-09-17T08:14:34.284299+0000","last_fullsized":"2024-09-17T08:14:34.284299+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.683777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.683777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.587530+0000","scrub_duration":0.007378796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"139'11729","reported_seq":11950,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.629912+0000","last_change":"2024-09-17T08:14:34.008301+0000","last_active":"2024-09-17T08:14:47.629912+0000","last_peered":"2024-09-17T08:14:47.629912+0000","last_clean":"2024-09-17T08:14:47.629912+0000","last_became_active":"2024-09-17T08:12:46.185093+0000","last_became_peered":"2024-09-17T08:12:46.185093+0000","last_unstale":"2024-09-17T08:14:47.629912+0000","last_undegraded":"2024-09-17T08:14:47.629912+0000","last_fullsized":"2024-09-17T08:14:47.629912+0000","mapping_epoch":103,"log_start":"113'6200","ondisk_log_start":"113'6200","created":91,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'10882","last_scrub_stamp":"2024-09-17T08:14:34.007989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:34.007989+0000","objects_scrubbed":4991,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:44.112073+0000","scrub_duration":2.270366321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266272768,"num_objects":4063,"num_object_clones":0,"num_object_copies":12189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11729,"num_write_kb":505344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"72'28","reported_seq":212,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.884449+0000","last_change":"2024-09-17T08:12:49.895324+0000","last_active":"2024-09-17T08:14:42.884449+0000","last_peered":"2024-09-17T08:14:42.884449+0000","last_clean":"2024-09-17T08:14:42.884449+0000","last_became_active":"2024-09-17T08:09:56.503418+0000","last_became_peered":"2024-09-17T08:09:56.503418+0000","last_unstale":"2024-09-17T08:14:42.884449+0000","last_undegraded":"2024-09-17T08:14:42.884449+0000","last_fullsized":"2024-09-17T08:14:42.884449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:12:49.895130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.895130+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076594480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.289778+0000","last_change":"2024-09-17T08:14:03.251865+0000","last_active":"2024-09-17T08:14:34.289778+0000","last_peered":"2024-09-17T08:14:34.289778+0000","last_clean":"2024-09-17T08:14:34.289778+0000","last_became_active":"2024-09-17T08:09:54.499602+0000","last_became_peered":"2024-09-17T08:09:54.499602+0000","last_unstale":"2024-09-17T08:14:34.289778+0000","last_undegraded":"2024-09-17T08:14:34.289778+0000","last_fullsized":"2024-09-17T08:14:34.289778+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.251783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:03.251783+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.251783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.513602+0000","scrub_duration":0.002663585,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294005+0000","last_change":"2024-09-17T08:13:49.610395+0000","last_active":"2024-09-17T08:14:34.294005+0000","last_peered":"2024-09-17T08:14:34.294005+0000","last_clean":"2024-09-17T08:14:34.294005+0000","last_became_active":"2024-09-17T08:09:58.511643+0000","last_became_peered":"2024-09-17T08:09:58.511643+0000","last_unstale":"2024-09-17T08:14:34.294005+0000","last_undegraded":"2024-09-17T08:14:34.294005+0000","last_fullsized":"2024-09-17T08:14:34.294005+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.610090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.610090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:50.329505+0000","scrub_duration":0.0081973259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.296640+0000","last_change":"2024-09-17T08:12:52.393950+0000","last_active":"2024-09-17T08:14:34.296640+0000","last_peered":"2024-09-17T08:14:34.296640+0000","last_clean":"2024-09-17T08:14:34.296640+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:14:34.296640+0000","last_undegraded":"2024-09-17T08:14:34.296640+0000","last_fullsized":"2024-09-17T08:14:34.296640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:52.393815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:52.393815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010050285000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"139'11764","reported_seq":11973,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.618404+0000","last_change":"2024-09-17T08:14:04.288383+0000","last_active":"2024-09-17T08:14:47.618404+0000","last_peered":"2024-09-17T08:14:47.618404+0000","last_clean":"2024-09-17T08:14:47.618404+0000","last_became_active":"2024-09-17T08:12:05.206469+0000","last_became_peered":"2024-09-17T08:12:05.206469+0000","last_unstale":"2024-09-17T08:14:47.618404+0000","last_undegraded":"2024-09-17T08:14:47.618404+0000","last_fullsized":"2024-09-17T08:14:47.618404+0000","mapping_epoch":91,"log_start":"114'6300","ondisk_log_start":"114'6300","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8997","last_scrub_stamp":"2024-09-17T08:14:04.288198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.288198+0000","objects_scrubbed":6921,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:32.382198+0000","scrub_duration":2.5911683210000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267255808,"num_objects":4078,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11764,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":98,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.277103+0000","last_change":"2024-09-17T08:13:39.910053+0000","last_active":"2024-09-17T08:14:34.277103+0000","last_peered":"2024-09-17T08:14:34.277103+0000","last_clean":"2024-09-17T08:14:34.277103+0000","last_became_active":"2024-09-17T08:11:32.435377+0000","last_became_peered":"2024-09-17T08:11:32.435377+0000","last_unstale":"2024-09-17T08:14:34.277103+0000","last_undegraded":"2024-09-17T08:14:34.277103+0000","last_fullsized":"2024-09-17T08:14:34.277103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:13:39.909755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:39.909755+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:05.391461+0000","scrub_duration":0.0076279349999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":401,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.998020+0000","last_change":"2024-09-17T08:12:47.905087+0000","last_active":"2024-09-17T08:14:45.998020+0000","last_peered":"2024-09-17T08:14:45.998020+0000","last_clean":"2024-09-17T08:14:45.998020+0000","last_became_active":"2024-09-17T08:09:54.497685+0000","last_became_peered":"2024-09-17T08:09:54.497685+0000","last_unstale":"2024-09-17T08:14:45.998020+0000","last_undegraded":"2024-09-17T08:14:45.998020+0000","last_fullsized":"2024-09-17T08:14:45.998020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:12:47.904961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:47.904961+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015034595,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":280,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.922587+0000","last_change":"2024-09-17T08:14:37.922587+0000","last_active":"2024-09-17T08:14:37.922587+0000","last_peered":"2024-09-17T08:14:37.922587+0000","last_clean":"2024-09-17T08:14:37.922587+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:14:37.922587+0000","last_undegraded":"2024-09-17T08:14:37.922587+0000","last_fullsized":"2024-09-17T08:14:37.922587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:37.922444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:14:37.922444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:01.092123+0000","scrub_duration":0.012442091000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.354234+0000","last_change":"2024-09-17T08:12:55.327814+0000","last_active":"2024-09-17T08:14:34.354234+0000","last_peered":"2024-09-17T08:14:34.354234+0000","last_clean":"2024-09-17T08:14:34.354234+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:14:34.354234+0000","last_undegraded":"2024-09-17T08:14:34.354234+0000","last_fullsized":"2024-09-17T08:14:34.354234+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:55.327282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.327282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066521990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:43.948340+0000","last_change":"2024-09-17T08:14:43.948340+0000","last_active":"2024-09-17T08:14:43.948340+0000","last_peered":"2024-09-17T08:14:43.948340+0000","last_clean":"2024-09-17T08:14:43.948340+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:14:43.948340+0000","last_undegraded":"2024-09-17T08:14:43.948340+0000","last_fullsized":"2024-09-17T08:14:43.948340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:43.948197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:14:43.948197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:47.068628+0000","scrub_duration":0.012406833000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"139'11936","reported_seq":11995,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.257943+0000","last_change":"2024-09-17T08:14:35.242009+0000","last_active":"2024-09-17T08:14:47.257943+0000","last_peered":"2024-09-17T08:14:47.257943+0000","last_clean":"2024-09-17T08:14:47.257943+0000","last_became_active":"2024-09-17T08:12:05.207217+0000","last_became_peered":"2024-09-17T08:12:05.207217+0000","last_unstale":"2024-09-17T08:14:47.257943+0000","last_undegraded":"2024-09-17T08:14:47.257943+0000","last_fullsized":"2024-09-17T08:14:47.257943+0000","mapping_epoch":91,"log_start":"117'7000","ondisk_log_start":"117'7000","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:12:04.173119+0000","objects_scrubbed":0,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270794752,"num_objects":4132,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11936,"num_write_kb":514176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":130,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294809+0000","last_change":"2024-09-17T08:13:43.584287+0000","last_active":"2024-09-17T08:14:34.294809+0000","last_peered":"2024-09-17T08:14:34.294809+0000","last_clean":"2024-09-17T08:14:34.294809+0000","last_became_active":"2024-09-17T08:09:56.508177+0000","last_became_peered":"2024-09-17T08:09:56.508177+0000","last_unstale":"2024-09-17T08:14:34.294809+0000","last_undegraded":"2024-09-17T08:14:34.294809+0000","last_fullsized":"2024-09-17T08:14:34.294809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:13:43.584153+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:13:43.584153+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.644811+0000","scrub_duration":0.017778021000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"67'1","reported_seq":114,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.288724+0000","last_change":"2024-09-17T08:12:55.802841+0000","last_active":"2024-09-17T08:14:34.288724+0000","last_peered":"2024-09-17T08:14:34.288724+0000","last_clean":"2024-09-17T08:14:34.288724+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:14:34.288724+0000","last_undegraded":"2024-09-17T08:14:34.288724+0000","last_fullsized":"2024-09-17T08:14:34.288724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'1","last_scrub_stamp":"2024-09-17T08:12:55.802644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.802644+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010970435000000001,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281596+0000","last_change":"2024-09-17T08:13:56.115539+0000","last_active":"2024-09-17T08:14:34.281596+0000","last_peered":"2024-09-17T08:14:34.281596+0000","last_clean":"2024-09-17T08:14:34.281596+0000","last_became_active":"2024-09-17T08:09:54.507184+0000","last_became_peered":"2024-09-17T08:09:54.507184+0000","last_unstale":"2024-09-17T08:14:34.281596+0000","last_undegraded":"2024-09-17T08:14:34.281596+0000","last_fullsized":"2024-09-17T08:14:34.281596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:13:56.115447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:21.526055+0000","scrub_duration":0.0045415680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290322+0000","last_change":"2024-09-17T08:12:43.075970+0000","last_active":"2024-09-17T08:14:34.290322+0000","last_peered":"2024-09-17T08:14:34.290322+0000","last_clean":"2024-09-17T08:14:34.290322+0000","last_became_active":"2024-09-17T08:09:58.502074+0000","last_became_peered":"2024-09-17T08:09:58.502074+0000","last_unstale":"2024-09-17T08:14:34.290322+0000","last_undegraded":"2024-09-17T08:14:34.290322+0000","last_fullsized":"2024-09-17T08:14:34.290322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.075863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.075863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013502640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.274928+0000","last_change":"2024-09-17T08:13:44.744020+0000","last_active":"2024-09-17T08:14:34.274928+0000","last_peered":"2024-09-17T08:14:34.274928+0000","last_clean":"2024-09-17T08:14:34.274928+0000","last_became_active":"2024-09-17T08:10:00.518926+0000","last_became_peered":"2024-09-17T08:10:00.518926+0000","last_unstale":"2024-09-17T08:14:34.274928+0000","last_undegraded":"2024-09-17T08:14:34.274928+0000","last_fullsized":"2024-09-17T08:14:34.274928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:44.743795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:44.743795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:03.716597+0000","scrub_duration":0.0038700259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"139'11722","reported_seq":11935,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.636401+0000","last_change":"2024-09-17T08:14:03.362812+0000","last_active":"2024-09-17T08:14:47.636401+0000","last_peered":"2024-09-17T08:14:47.636401+0000","last_clean":"2024-09-17T08:14:47.636401+0000","last_became_active":"2024-09-17T08:12:05.208529+0000","last_became_peered":"2024-09-17T08:12:05.208529+0000","last_unstale":"2024-09-17T08:14:47.636401+0000","last_undegraded":"2024-09-17T08:14:47.636401+0000","last_fullsized":"2024-09-17T08:14:47.636401+0000","mapping_epoch":91,"log_start":"113'6200","ondisk_log_start":"113'6200","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'8967","last_scrub_stamp":"2024-09-17T08:14:03.362611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.362611+0000","objects_scrubbed":6991,"log_size":5522,"log_dups_size":0,"ondisk_log_size":5522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:19.667692+0000","scrub_duration":2.7054181179999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272367616,"num_objects":4156,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11722,"num_write_kb":508096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"139'21","reported_seq":100,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:46.101981+0000","last_change":"2024-09-17T08:14:46.102116+0000","last_active":"2024-09-17T08:14:46.101981+0000","last_peered":"2024-09-17T08:14:46.101981+0000","last_clean":"2024-09-17T08:14:46.101981+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:14:46.101981+0000","last_undegraded":"2024-09-17T08:14:46.101981+0000","last_fullsized":"2024-09-17T08:14:46.101981+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'21","last_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:14:46.101949+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:07.379304+0000","scrub_duration":0.011657381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.293877+0000","last_change":"2024-09-17T08:14:33.367441+0000","last_active":"2024-09-17T08:14:34.293877+0000","last_peered":"2024-09-17T08:14:34.293877+0000","last_clean":"2024-09-17T08:14:34.293877+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:14:34.293877+0000","last_undegraded":"2024-09-17T08:14:34.293877+0000","last_fullsized":"2024-09-17T08:14:34.293877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:14:33.367113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:33.367113+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:43.951220+0000","scrub_duration":0.0052554730000000001,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.291687+0000","last_change":"2024-09-17T08:12:38.112848+0000","last_active":"2024-09-17T08:14:34.291687+0000","last_peered":"2024-09-17T08:14:34.291687+0000","last_clean":"2024-09-17T08:14:34.291687+0000","last_became_active":"2024-09-17T08:09:54.502210+0000","last_became_peered":"2024-09-17T08:09:54.502210+0000","last_unstale":"2024-09-17T08:14:34.291687+0000","last_undegraded":"2024-09-17T08:14:34.291687+0000","last_fullsized":"2024-09-17T08:14:34.291687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:38.112582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:38.112582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027848069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":369,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281688+0000","last_change":"2024-09-17T08:14:32.196268+0000","last_active":"2024-09-17T08:14:34.281688+0000","last_peered":"2024-09-17T08:14:34.281688+0000","last_clean":"2024-09-17T08:14:34.281688+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:14:34.281688+0000","last_undegraded":"2024-09-17T08:14:34.281688+0000","last_fullsized":"2024-09-17T08:14:34.281688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:14:32.196101+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:14:32.196101+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:32.599164+0000","scrub_duration":0.0029103029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281713+0000","last_change":"2024-09-17T08:14:04.052308+0000","last_active":"2024-09-17T08:14:34.281713+0000","last_peered":"2024-09-17T08:14:34.281713+0000","last_clean":"2024-09-17T08:14:34.281713+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:14:34.281713+0000","last_undegraded":"2024-09-17T08:14:34.281713+0000","last_fullsized":"2024-09-17T08:14:34.281713+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:04.052233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.052233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:07.637042+0000","scrub_duration":0.0041087229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290391+0000","last_change":"2024-09-17T08:13:34.022691+0000","last_active":"2024-09-17T08:14:34.290391+0000","last_peered":"2024-09-17T08:14:34.290391+0000","last_clean":"2024-09-17T08:14:34.290391+0000","last_became_active":"2024-09-17T08:10:00.521988+0000","last_became_peered":"2024-09-17T08:10:00.521988+0000","last_unstale":"2024-09-17T08:14:34.290391+0000","last_undegraded":"2024-09-17T08:14:34.290391+0000","last_fullsized":"2024-09-17T08:14:34.290391+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:34.022556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.022556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065771129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"139'11826","reported_seq":11995,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.630439+0000","last_change":"2024-09-17T08:14:36.586011+0000","last_active":"2024-09-17T08:14:47.630439+0000","last_peered":"2024-09-17T08:14:47.630439+0000","last_clean":"2024-09-17T08:14:47.630439+0000","last_became_active":"2024-09-17T08:12:05.207111+0000","last_became_peered":"2024-09-17T08:12:05.207111+0000","last_unstale":"2024-09-17T08:14:47.630439+0000","last_undegraded":"2024-09-17T08:14:47.630439+0000","last_fullsized":"2024-09-17T08:14:47.630439+0000","mapping_epoch":91,"log_start":"113'6300","ondisk_log_start":"113'6300","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11070","last_scrub_stamp":"2024-09-17T08:14:36.585116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.585116+0000","objects_scrubbed":4966,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:49.382672+0000","scrub_duration":1.821818428,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272236544,"num_objects":4154,"num_object_clones":0,"num_object_copies":12462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":511360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290190+0000","last_change":"2024-09-17T08:12:56.917131+0000","last_active":"2024-09-17T08:14:34.290190+0000","last_peered":"2024-09-17T08:14:34.290190+0000","last_clean":"2024-09-17T08:14:34.290190+0000","last_became_active":"2024-09-17T08:12:46.183692+0000","last_became_peered":"2024-09-17T08:12:46.183692+0000","last_unstale":"2024-09-17T08:14:34.290190+0000","last_undegraded":"2024-09-17T08:14:34.290190+0000","last_fullsized":"2024-09-17T08:14:34.290190+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:12:56.916985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.916985+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011500321000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.277457+0000","last_change":"2024-09-17T08:12:49.325243+0000","last_active":"2024-09-17T08:14:34.277457+0000","last_peered":"2024-09-17T08:14:34.277457+0000","last_clean":"2024-09-17T08:14:34.277457+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:14:34.277457+0000","last_undegraded":"2024-09-17T08:14:34.277457+0000","last_fullsized":"2024-09-17T08:14:34.277457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.325122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.325122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003561631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.277165+0000","last_change":"2024-09-17T08:12:50.355594+0000","last_active":"2024-09-17T08:14:34.277165+0000","last_peered":"2024-09-17T08:14:34.277165+0000","last_clean":"2024-09-17T08:14:34.277165+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:14:34.277165+0000","last_undegraded":"2024-09-17T08:14:34.277165+0000","last_fullsized":"2024-09-17T08:14:34.277165+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:50.355473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:50.355473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094222349999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281588+0000","last_change":"2024-09-17T08:13:41.903624+0000","last_active":"2024-09-17T08:14:34.281588+0000","last_peered":"2024-09-17T08:14:34.281588+0000","last_clean":"2024-09-17T08:14:34.281588+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:14:34.281588+0000","last_undegraded":"2024-09-17T08:14:34.281588+0000","last_fullsized":"2024-09-17T08:14:34.281588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:41.903070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:41.903070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:51.229180+0000","scrub_duration":0.0063924560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"139'11736","reported_seq":11949,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.473716+0000","last_change":"2024-09-17T08:14:36.549130+0000","last_active":"2024-09-17T08:14:47.473716+0000","last_peered":"2024-09-17T08:14:47.473716+0000","last_clean":"2024-09-17T08:14:47.473716+0000","last_became_active":"2024-09-17T08:12:05.203964+0000","last_became_peered":"2024-09-17T08:12:05.203964+0000","last_unstale":"2024-09-17T08:14:47.473716+0000","last_undegraded":"2024-09-17T08:14:47.473716+0000","last_fullsized":"2024-09-17T08:14:47.473716+0000","mapping_epoch":91,"log_start":"108'5200","ondisk_log_start":"108'5200","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11018","last_scrub_stamp":"2024-09-17T08:14:36.548737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:36.548737+0000","objects_scrubbed":4930,"log_size":6536,"log_dups_size":0,"ondisk_log_size":6536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:51.619556+0000","scrub_duration":1.6783069149999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272498688,"num_objects":4158,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11736,"num_write_kb":508608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":122,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.287615+0000","last_change":"2024-09-17T08:13:54.660609+0000","last_active":"2024-09-17T08:14:34.287615+0000","last_peered":"2024-09-17T08:14:34.287615+0000","last_clean":"2024-09-17T08:14:34.287615+0000","last_became_active":"2024-09-17T08:09:56.506142+0000","last_became_peered":"2024-09-17T08:09:56.506142+0000","last_unstale":"2024-09-17T08:14:34.287615+0000","last_undegraded":"2024-09-17T08:14:34.287615+0000","last_fullsized":"2024-09-17T08:14:34.287615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:13:54.660479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.660479+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.788584+0000","scrub_duration":0.015027664,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.285419+0000","last_change":"2024-09-17T08:12:37.930254+0000","last_active":"2024-09-17T08:14:34.285419+0000","last_peered":"2024-09-17T08:14:34.285419+0000","last_clean":"2024-09-17T08:14:34.285419+0000","last_became_active":"2024-09-17T08:09:54.498859+0000","last_became_peered":"2024-09-17T08:09:54.498859+0000","last_unstale":"2024-09-17T08:14:34.285419+0000","last_undegraded":"2024-09-17T08:14:34.285419+0000","last_fullsized":"2024-09-17T08:14:34.285419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:37.929953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:37.929953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031686209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.279695+0000","last_change":"2024-09-17T08:13:54.967053+0000","last_active":"2024-09-17T08:14:34.279695+0000","last_peered":"2024-09-17T08:14:34.279695+0000","last_clean":"2024-09-17T08:14:34.279695+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:14:34.279695+0000","last_undegraded":"2024-09-17T08:14:34.279695+0000","last_fullsized":"2024-09-17T08:14:34.279695+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:54.966885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:13:54.966885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:09.852302+0000","scrub_duration":0.014432926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.288635+0000","last_change":"2024-09-17T08:12:49.737151+0000","last_active":"2024-09-17T08:14:34.288635+0000","last_peered":"2024-09-17T08:14:34.288635+0000","last_clean":"2024-09-17T08:14:34.288635+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:14:34.288635+0000","last_undegraded":"2024-09-17T08:14:34.288635+0000","last_fullsized":"2024-09-17T08:14:34.288635+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.736793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.736793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004027793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"139'11544","reported_seq":11699,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.334112+0000","last_change":"2024-09-17T08:14:45.130827+0000","last_active":"2024-09-17T08:14:47.334112+0000","last_peered":"2024-09-17T08:14:47.334112+0000","last_clean":"2024-09-17T08:14:47.334112+0000","last_became_active":"2024-09-17T08:12:05.200128+0000","last_became_peered":"2024-09-17T08:12:05.200128+0000","last_unstale":"2024-09-17T08:14:47.334112+0000","last_undegraded":"2024-09-17T08:14:47.334112+0000","last_fullsized":"2024-09-17T08:14:47.334112+0000","mapping_epoch":91,"log_start":"114'6400","ondisk_log_start":"114'6400","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11403","last_scrub_stamp":"2024-09-17T08:14:45.130346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:45.130346+0000","objects_scrubbed":4261,"log_size":5144,"log_dups_size":0,"ondisk_log_size":5144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:14.062663+0000","scrub_duration":1.2710336330000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":4096,"num_object_clones":0,"num_object_copies":12288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11544,"num_write_kb":500480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"72'15","reported_seq":131,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.289458+0000","last_change":"2024-09-17T08:12:45.062371+0000","last_active":"2024-09-17T08:14:34.289458+0000","last_peered":"2024-09-17T08:14:34.289458+0000","last_clean":"2024-09-17T08:14:34.289458+0000","last_became_active":"2024-09-17T08:09:57.040783+0000","last_became_peered":"2024-09-17T08:09:57.040783+0000","last_unstale":"2024-09-17T08:14:34.289458+0000","last_undegraded":"2024-09-17T08:14:34.289458+0000","last_fullsized":"2024-09-17T08:14:34.289458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:12:45.062287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:45.062287+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020883730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.354303+0000","last_change":"2024-09-17T08:12:37.141169+0000","last_active":"2024-09-17T08:14:34.354303+0000","last_peered":"2024-09-17T08:14:34.354303+0000","last_clean":"2024-09-17T08:14:34.354303+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:14:34.354303+0000","last_undegraded":"2024-09-17T08:14:34.354303+0000","last_fullsized":"2024-09-17T08:14:34.354303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:37.141052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:37.141052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020501897000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":170,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.884470+0000","last_change":"2024-09-17T08:13:36.798097+0000","last_active":"2024-09-17T08:14:42.884470+0000","last_peered":"2024-09-17T08:14:42.884470+0000","last_clean":"2024-09-17T08:14:42.884470+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:14:42.884470+0000","last_undegraded":"2024-09-17T08:14:42.884470+0000","last_fullsized":"2024-09-17T08:14:42.884470+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:36.797958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:36.797958+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.004868+0000","scrub_duration":0.0023544389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.289510+0000","last_change":"2024-09-17T08:14:00.164468+0000","last_active":"2024-09-17T08:14:34.289510+0000","last_peered":"2024-09-17T08:14:34.289510+0000","last_clean":"2024-09-17T08:14:34.289510+0000","last_became_active":"2024-09-17T08:10:00.598697+0000","last_became_peered":"2024-09-17T08:10:00.598697+0000","last_unstale":"2024-09-17T08:14:34.289510+0000","last_undegraded":"2024-09-17T08:14:34.289510+0000","last_fullsized":"2024-09-17T08:14:34.289510+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:00.164369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:00.164369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:07.156293+0000","scrub_duration":0.001471577,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"139'12043","reported_seq":12204,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.523745+0000","last_change":"2024-09-17T08:14:43.599107+0000","last_active":"2024-09-17T08:14:47.523745+0000","last_peered":"2024-09-17T08:14:47.523745+0000","last_clean":"2024-09-17T08:14:47.523745+0000","last_became_active":"2024-09-17T08:12:05.209262+0000","last_became_peered":"2024-09-17T08:12:05.209262+0000","last_unstale":"2024-09-17T08:14:47.523745+0000","last_undegraded":"2024-09-17T08:14:47.523745+0000","last_fullsized":"2024-09-17T08:14:47.523745+0000","mapping_epoch":91,"log_start":"119'7300","ondisk_log_start":"119'7300","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11777","last_scrub_stamp":"2024-09-17T08:14:43.598891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:04.173119+0000","last_clean_scrub_stamp":"2024-09-17T08:14:43.598891+0000","objects_scrubbed":4521,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:45.758447+0000","scrub_duration":2.1873042140000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275447808,"num_objects":4203,"num_object_clones":0,"num_object_copies":12609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12043,"num_write_kb":519872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":140,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:38.189031+0000","last_change":"2024-09-17T08:14:38.189031+0000","last_active":"2024-09-17T08:14:38.189031+0000","last_peered":"2024-09-17T08:14:38.189031+0000","last_clean":"2024-09-17T08:14:38.189031+0000","last_became_active":"2024-09-17T08:09:56.503246+0000","last_became_peered":"2024-09-17T08:09:56.503246+0000","last_unstale":"2024-09-17T08:14:38.189031+0000","last_undegraded":"2024-09-17T08:14:38.189031+0000","last_fullsized":"2024-09-17T08:14:38.189031+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:14:38.188915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:38.188915+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:47.577655+0000","scrub_duration":0.0039895130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":126,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264991+0000","last_change":"2024-09-17T08:14:05.379019+0000","last_active":"2024-09-17T08:14:34.264991+0000","last_peered":"2024-09-17T08:14:34.264991+0000","last_clean":"2024-09-17T08:14:34.264991+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:14:34.264991+0000","last_undegraded":"2024-09-17T08:14:34.264991+0000","last_fullsized":"2024-09-17T08:14:34.264991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:14:05.378907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:05.378907+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:08.195877+0000","scrub_duration":0.0051126039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:41.917322+0000","last_change":"2024-09-17T08:14:41.917322+0000","last_active":"2024-09-17T08:14:41.917322+0000","last_peered":"2024-09-17T08:14:41.917322+0000","last_clean":"2024-09-17T08:14:41.917322+0000","last_became_active":"2024-09-17T08:09:58.506540+0000","last_became_peered":"2024-09-17T08:09:58.506540+0000","last_unstale":"2024-09-17T08:14:41.917322+0000","last_undegraded":"2024-09-17T08:14:41.917322+0000","last_fullsized":"2024-09-17T08:14:41.917322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:41.916809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:14:41.916809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:10.593553+0000","scrub_duration":0.0031579490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.295021+0000","last_change":"2024-09-17T08:12:46.344419+0000","last_active":"2024-09-17T08:14:34.295021+0000","last_peered":"2024-09-17T08:14:34.295021+0000","last_clean":"2024-09-17T08:14:34.295021+0000","last_became_active":"2024-09-17T08:10:00.522082+0000","last_became_peered":"2024-09-17T08:10:00.522082+0000","last_unstale":"2024-09-17T08:14:34.295021+0000","last_undegraded":"2024-09-17T08:14:34.295021+0000","last_fullsized":"2024-09-17T08:14:34.295021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:46.344243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:46.344243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010636784999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":126,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.379704+0000","last_change":"2024-09-17T08:13:57.680834+0000","last_active":"2024-09-17T08:14:45.379704+0000","last_peered":"2024-09-17T08:14:45.379704+0000","last_clean":"2024-09-17T08:14:45.379704+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:14:45.379704+0000","last_undegraded":"2024-09-17T08:14:45.379704+0000","last_fullsized":"2024-09-17T08:14:45.379704+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:13:57.680700+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:13:57.680700+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.392665+0000","scrub_duration":0.031491026999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.501661+0000","last_change":"2024-09-17T08:12:35.149784+0000","last_active":"2024-09-17T08:14:37.501661+0000","last_peered":"2024-09-17T08:14:37.501661+0000","last_clean":"2024-09-17T08:14:37.501661+0000","last_became_active":"2024-09-17T08:09:54.501595+0000","last_became_peered":"2024-09-17T08:09:54.501595+0000","last_unstale":"2024-09-17T08:14:37.501661+0000","last_undegraded":"2024-09-17T08:14:37.501661+0000","last_fullsized":"2024-09-17T08:14:37.501661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:35.149490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:35.149490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013490703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":93,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.388183+0000","last_change":"2024-09-17T08:12:35.295415+0000","last_active":"2024-09-17T08:14:45.388183+0000","last_peered":"2024-09-17T08:14:45.388183+0000","last_clean":"2024-09-17T08:14:45.388183+0000","last_became_active":"2024-09-17T08:09:58.518594+0000","last_became_peered":"2024-09-17T08:09:58.518594+0000","last_unstale":"2024-09-17T08:14:45.388183+0000","last_undegraded":"2024-09-17T08:14:45.388183+0000","last_fullsized":"2024-09-17T08:14:45.388183+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:35.295240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:35.295240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075020399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":113,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:46.960715+0000","last_change":"2024-09-17T08:14:46.960715+0000","last_active":"2024-09-17T08:14:46.960715+0000","last_peered":"2024-09-17T08:14:46.960715+0000","last_clean":"2024-09-17T08:14:46.960715+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:14:46.960715+0000","last_undegraded":"2024-09-17T08:14:46.960715+0000","last_fullsized":"2024-09-17T08:14:46.960715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:46.960570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:46.960570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:06.234782+0000","scrub_duration":0.0030097190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":58,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263770+0000","last_change":"2024-09-17T08:13:36.335503+0000","last_active":"2024-09-17T08:14:34.263770+0000","last_peered":"2024-09-17T08:14:34.263770+0000","last_clean":"2024-09-17T08:14:34.263770+0000","last_became_active":"2024-09-17T08:12:46.186606+0000","last_became_peered":"2024-09-17T08:12:46.186606+0000","last_unstale":"2024-09-17T08:14:34.263770+0000","last_undegraded":"2024-09-17T08:14:34.263770+0000","last_fullsized":"2024-09-17T08:14:34.263770+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:36.335413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:36.335413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:54.653797+0000","scrub_duration":0.0030218530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":115,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.388318+0000","last_change":"2024-09-17T08:13:42.209088+0000","last_active":"2024-09-17T08:14:45.388318+0000","last_peered":"2024-09-17T08:14:45.388318+0000","last_clean":"2024-09-17T08:14:45.388318+0000","last_became_active":"2024-09-17T08:09:57.043084+0000","last_became_peered":"2024-09-17T08:09:57.043084+0000","last_unstale":"2024-09-17T08:14:45.388318+0000","last_undegraded":"2024-09-17T08:14:45.388318+0000","last_fullsized":"2024-09-17T08:14:45.388318+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:13:42.208919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:42.208919+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099883109999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":104,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.375967+0000","last_change":"2024-09-17T08:12:50.030274+0000","last_active":"2024-09-17T08:14:45.375967+0000","last_peered":"2024-09-17T08:14:45.375967+0000","last_clean":"2024-09-17T08:14:45.375967+0000","last_became_active":"2024-09-17T08:09:58.506937+0000","last_became_peered":"2024-09-17T08:09:58.506937+0000","last_unstale":"2024-09-17T08:14:45.375967+0000","last_undegraded":"2024-09-17T08:14:45.375967+0000","last_fullsized":"2024-09-17T08:14:45.375967+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:50.030147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:50.030147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063418609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":102,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.372073+0000","last_change":"2024-09-17T08:12:56.560628+0000","last_active":"2024-09-17T08:14:45.372073+0000","last_peered":"2024-09-17T08:14:45.372073+0000","last_clean":"2024-09-17T08:14:45.372073+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:14:45.372073+0000","last_undegraded":"2024-09-17T08:14:45.372073+0000","last_fullsized":"2024-09-17T08:14:45.372073+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:56.560534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:56.560534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001610879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.545698+0000","last_change":"2024-09-17T08:14:01.307387+0000","last_active":"2024-09-17T08:14:42.545698+0000","last_peered":"2024-09-17T08:14:42.545698+0000","last_clean":"2024-09-17T08:14:42.545698+0000","last_became_active":"2024-09-17T08:09:57.040876+0000","last_became_peered":"2024-09-17T08:09:57.040876+0000","last_unstale":"2024-09-17T08:14:42.545698+0000","last_undegraded":"2024-09-17T08:14:42.545698+0000","last_fullsized":"2024-09-17T08:14:42.545698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:14:01.307065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:01.307065+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.149862+0000","scrub_duration":0.01517745,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":128,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:47.207312+0000","last_change":"2024-09-17T08:14:47.207426+0000","last_active":"2024-09-17T08:14:47.207312+0000","last_peered":"2024-09-17T08:14:47.207312+0000","last_clean":"2024-09-17T08:14:47.207312+0000","last_became_active":"2024-09-17T08:11:32.437420+0000","last_became_peered":"2024-09-17T08:11:32.437420+0000","last_unstale":"2024-09-17T08:14:47.207312+0000","last_undegraded":"2024-09-17T08:14:47.207312+0000","last_fullsized":"2024-09-17T08:14:47.207312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:47.207288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:47.207288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:14.538317+0000","scrub_duration":0.013954416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"72'8","reported_seq":170,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.372547+0000","last_change":"2024-09-17T08:13:38.125058+0000","last_active":"2024-09-17T08:14:45.372547+0000","last_peered":"2024-09-17T08:14:45.372547+0000","last_clean":"2024-09-17T08:14:45.372547+0000","last_became_active":"2024-09-17T08:09:58.505903+0000","last_became_peered":"2024-09-17T08:09:58.505903+0000","last_unstale":"2024-09-17T08:14:45.372547+0000","last_undegraded":"2024-09-17T08:14:45.372547+0000","last_fullsized":"2024-09-17T08:14:45.372547+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:13:38.124922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:38.124922+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012576178,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294351+0000","last_change":"2024-09-17T08:13:51.666223+0000","last_active":"2024-09-17T08:14:34.294351+0000","last_peered":"2024-09-17T08:14:34.294351+0000","last_clean":"2024-09-17T08:14:34.294351+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:14:34.294351+0000","last_undegraded":"2024-09-17T08:14:34.294351+0000","last_fullsized":"2024-09-17T08:14:34.294351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:51.665999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:51.665999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:15.188087+0000","scrub_duration":0.01465203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'9","reported_seq":122,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.372616+0000","last_change":"2024-09-17T08:12:36.171494+0000","last_active":"2024-09-17T08:14:45.372616+0000","last_peered":"2024-09-17T08:14:45.372616+0000","last_clean":"2024-09-17T08:14:45.372616+0000","last_became_active":"2024-09-17T08:09:57.041601+0000","last_became_peered":"2024-09-17T08:09:57.041601+0000","last_unstale":"2024-09-17T08:14:45.372616+0000","last_undegraded":"2024-09-17T08:14:45.372616+0000","last_fullsized":"2024-09-17T08:14:45.372616+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:12:36.171403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:36.171403+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.060433385999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:44.732923+0000","last_change":"2024-09-17T08:14:44.732923+0000","last_active":"2024-09-17T08:14:44.732923+0000","last_peered":"2024-09-17T08:14:44.732923+0000","last_clean":"2024-09-17T08:14:44.732923+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:14:44.732923+0000","last_undegraded":"2024-09-17T08:14:44.732923+0000","last_fullsized":"2024-09-17T08:14:44.732923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:44.732823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:44.732823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:51.853239+0000","scrub_duration":0.0084282269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":171,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.883284+0000","last_change":"2024-09-17T08:14:39.413470+0000","last_active":"2024-09-17T08:14:42.883284+0000","last_peered":"2024-09-17T08:14:42.883284+0000","last_clean":"2024-09-17T08:14:42.883284+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:14:42.883284+0000","last_undegraded":"2024-09-17T08:14:42.883284+0000","last_fullsized":"2024-09-17T08:14:42.883284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:14:39.413074+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:14:39.413074+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:40.621683+0000","scrub_duration":0.015248731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264835+0000","last_change":"2024-09-17T08:13:49.324546+0000","last_active":"2024-09-17T08:14:34.264835+0000","last_peered":"2024-09-17T08:14:34.264835+0000","last_clean":"2024-09-17T08:14:34.264835+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:14:34.264835+0000","last_undegraded":"2024-09-17T08:14:34.264835+0000","last_fullsized":"2024-09-17T08:14:34.264835+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:49.324431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:49.324431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.346128+0000","scrub_duration":0.0098997129999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":133,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.351433+0000","last_change":"2024-09-17T08:14:35.754010+0000","last_active":"2024-09-17T08:14:42.351433+0000","last_peered":"2024-09-17T08:14:42.351433+0000","last_clean":"2024-09-17T08:14:42.351433+0000","last_became_active":"2024-09-17T08:09:56.499411+0000","last_became_peered":"2024-09-17T08:09:56.499411+0000","last_unstale":"2024-09-17T08:14:42.351433+0000","last_undegraded":"2024-09-17T08:14:42.351433+0000","last_fullsized":"2024-09-17T08:14:42.351433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:14:35.753807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.753807+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:54.769940+0000","scrub_duration":0.028091901999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":58,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.291272+0000","last_change":"2024-09-17T08:12:53.362491+0000","last_active":"2024-09-17T08:14:34.291272+0000","last_peered":"2024-09-17T08:14:34.291272+0000","last_clean":"2024-09-17T08:14:34.291272+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:14:34.291272+0000","last_undegraded":"2024-09-17T08:14:34.291272+0000","last_fullsized":"2024-09-17T08:14:34.291272+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:53.362370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:53.362370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011603571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.545638+0000","last_change":"2024-09-17T08:13:58.267374+0000","last_active":"2024-09-17T08:14:42.545638+0000","last_peered":"2024-09-17T08:14:42.545638+0000","last_clean":"2024-09-17T08:14:42.545638+0000","last_became_active":"2024-09-17T08:09:58.508071+0000","last_became_peered":"2024-09-17T08:09:58.508071+0000","last_unstale":"2024-09-17T08:14:42.545638+0000","last_undegraded":"2024-09-17T08:14:42.545638+0000","last_fullsized":"2024-09-17T08:14:42.545638+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:58.267204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.267204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:12.437180+0000","scrub_duration":0.0072680419999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.635855+0000","last_change":"2024-09-17T08:13:47.616836+0000","last_active":"2024-09-17T08:14:42.635855+0000","last_peered":"2024-09-17T08:14:42.635855+0000","last_clean":"2024-09-17T08:14:42.635855+0000","last_became_active":"2024-09-17T08:10:00.517746+0000","last_became_peered":"2024-09-17T08:10:00.517746+0000","last_unstale":"2024-09-17T08:14:42.635855+0000","last_undegraded":"2024-09-17T08:14:42.635855+0000","last_fullsized":"2024-09-17T08:14:42.635855+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:47.616708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:47.616708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.493191+0000","scrub_duration":0.0071559839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":136,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.263358+0000","last_change":"2024-09-17T08:14:39.209379+0000","last_active":"2024-09-17T08:14:42.263358+0000","last_peered":"2024-09-17T08:14:42.263358+0000","last_clean":"2024-09-17T08:14:42.263358+0000","last_became_active":"2024-09-17T08:09:56.498415+0000","last_became_peered":"2024-09-17T08:09:56.498415+0000","last_unstale":"2024-09-17T08:14:42.263358+0000","last_undegraded":"2024-09-17T08:14:42.263358+0000","last_fullsized":"2024-09-17T08:14:42.263358+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:14:39.209237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:39.209237+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:02.384048+0000","scrub_duration":0.0034689090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.351488+0000","last_change":"2024-09-17T08:13:56.658599+0000","last_active":"2024-09-17T08:14:42.351488+0000","last_peered":"2024-09-17T08:14:42.351488+0000","last_clean":"2024-09-17T08:14:42.351488+0000","last_became_active":"2024-09-17T08:09:54.496370+0000","last_became_peered":"2024-09-17T08:09:54.496370+0000","last_unstale":"2024-09-17T08:14:42.351488+0000","last_undegraded":"2024-09-17T08:14:42.351488+0000","last_fullsized":"2024-09-17T08:14:42.351488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:56.658473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:56.658473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:25.037189+0000","scrub_duration":0.0071637649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.351418+0000","last_change":"2024-09-17T08:14:39.816478+0000","last_active":"2024-09-17T08:14:42.351418+0000","last_peered":"2024-09-17T08:14:42.351418+0000","last_clean":"2024-09-17T08:14:42.351418+0000","last_became_active":"2024-09-17T08:09:58.508277+0000","last_became_peered":"2024-09-17T08:09:58.508277+0000","last_unstale":"2024-09-17T08:14:42.351418+0000","last_undegraded":"2024-09-17T08:14:42.351418+0000","last_fullsized":"2024-09-17T08:14:42.351418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:39.816110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:39.816110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:03.262965+0000","scrub_duration":0.0065387529999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.544995+0000","last_change":"2024-09-17T08:13:37.215381+0000","last_active":"2024-09-17T08:14:42.544995+0000","last_peered":"2024-09-17T08:14:42.544995+0000","last_clean":"2024-09-17T08:14:42.544995+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:14:42.544995+0000","last_undegraded":"2024-09-17T08:14:42.544995+0000","last_fullsized":"2024-09-17T08:14:42.544995+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.215249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.215249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:57.556524+0000","scrub_duration":0.035524337000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":112,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.502273+0000","last_change":"2024-09-17T08:14:35.434487+0000","last_active":"2024-09-17T08:14:37.502273+0000","last_peered":"2024-09-17T08:14:37.502273+0000","last_clean":"2024-09-17T08:14:37.502273+0000","last_became_active":"2024-09-17T08:09:56.506543+0000","last_became_peered":"2024-09-17T08:09:56.506543+0000","last_unstale":"2024-09-17T08:14:37.502273+0000","last_undegraded":"2024-09-17T08:14:37.502273+0000","last_fullsized":"2024-09-17T08:14:37.502273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:14:35.433619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.433619+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:53.671533+0000","scrub_duration":0.0079958110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":119,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.310414+0000","last_change":"2024-09-17T08:14:40.283924+0000","last_active":"2024-09-17T08:14:42.310414+0000","last_peered":"2024-09-17T08:14:42.310414+0000","last_clean":"2024-09-17T08:14:42.310414+0000","last_became_active":"2024-09-17T08:09:54.500168+0000","last_became_peered":"2024-09-17T08:09:54.500168+0000","last_unstale":"2024-09-17T08:14:42.310414+0000","last_undegraded":"2024-09-17T08:14:42.310414+0000","last_fullsized":"2024-09-17T08:14:42.310414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:40.283755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:40.283755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:08.085247+0000","scrub_duration":0.008578947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":59,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.263552+0000","last_change":"2024-09-17T08:12:55.327121+0000","last_active":"2024-09-17T08:14:34.263552+0000","last_peered":"2024-09-17T08:14:34.263552+0000","last_clean":"2024-09-17T08:14:34.263552+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:14:34.263552+0000","last_undegraded":"2024-09-17T08:14:34.263552+0000","last_fullsized":"2024-09-17T08:14:34.263552+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:55.327041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:08.310979+0000","last_clean_scrub_stamp":"2024-09-17T08:12:55.327041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013685990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.265618+0000","last_change":"2024-09-17T08:12:49.448378+0000","last_active":"2024-09-17T08:14:42.265618+0000","last_peered":"2024-09-17T08:14:42.265618+0000","last_clean":"2024-09-17T08:14:42.265618+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:14:42.265618+0000","last_undegraded":"2024-09-17T08:14:42.265618+0000","last_fullsized":"2024-09-17T08:14:42.265618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:49.448287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:12:49.448287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026602499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":125,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.502306+0000","last_change":"2024-09-17T08:13:48.179594+0000","last_active":"2024-09-17T08:14:37.502306+0000","last_peered":"2024-09-17T08:14:37.502306+0000","last_clean":"2024-09-17T08:14:37.502306+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:14:37.502306+0000","last_undegraded":"2024-09-17T08:14:37.502306+0000","last_fullsized":"2024-09-17T08:14:37.502306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:13:48.179449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:48.179449+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:17.230628+0000","scrub_duration":0.0059153540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":85,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290459+0000","last_change":"2024-09-17T08:12:46.184677+0000","last_active":"2024-09-17T08:14:34.290459+0000","last_peered":"2024-09-17T08:14:34.290459+0000","last_clean":"2024-09-17T08:14:34.290459+0000","last_became_active":"2024-09-17T08:12:46.183935+0000","last_became_peered":"2024-09-17T08:12:46.183935+0000","last_unstale":"2024-09-17T08:14:34.290459+0000","last_undegraded":"2024-09-17T08:14:34.290459+0000","last_fullsized":"2024-09-17T08:14:34.290459+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:12:39.460413+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:12:39.460413+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:37.262108+0000","last_change":"2024-09-17T08:13:33.776436+0000","last_active":"2024-09-17T08:14:37.262108+0000","last_peered":"2024-09-17T08:14:37.262108+0000","last_clean":"2024-09-17T08:14:37.262108+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:14:37.262108+0000","last_undegraded":"2024-09-17T08:14:37.262108+0000","last_fullsized":"2024-09-17T08:14:37.262108+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:33.776333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:33.776333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:00.865312+0000","scrub_duration":0.0016169979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":91,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.389235+0000","last_change":"2024-09-17T08:12:40.337179+0000","last_active":"2024-09-17T08:14:45.389235+0000","last_peered":"2024-09-17T08:14:45.389235+0000","last_clean":"2024-09-17T08:14:45.389235+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:14:45.389235+0000","last_undegraded":"2024-09-17T08:14:45.389235+0000","last_fullsized":"2024-09-17T08:14:45.389235+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:40.336921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:40.336921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014625262999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.281695+0000","last_change":"2024-09-17T08:14:05.076766+0000","last_active":"2024-09-17T08:14:34.281695+0000","last_peered":"2024-09-17T08:14:34.281695+0000","last_clean":"2024-09-17T08:14:34.281695+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:14:34.281695+0000","last_undegraded":"2024-09-17T08:14:34.281695+0000","last_fullsized":"2024-09-17T08:14:34.281695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:14:05.076681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:05.076681+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.231207+0000","scrub_duration":0.0076773190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":108,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T08:14:45.371411+0000","last_change":"2024-09-17T08:13:37.089663+0000","last_active":"2024-09-17T08:14:45.371411+0000","last_peered":"2024-09-17T08:14:45.371411+0000","last_clean":"2024-09-17T08:14:45.371411+0000","last_became_active":"2024-09-17T08:09:54.500210+0000","last_became_peered":"2024-09-17T08:09:54.500210+0000","last_unstale":"2024-09-17T08:14:45.371411+0000","last_undegraded":"2024-09-17T08:14:45.371411+0000","last_fullsized":"2024-09-17T08:14:45.371411+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:37.089515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:13:37.089515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016432529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294651+0000","last_change":"2024-09-17T08:12:54.884143+0000","last_active":"2024-09-17T08:14:34.294651+0000","last_peered":"2024-09-17T08:14:34.294651+0000","last_clean":"2024-09-17T08:14:34.294651+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:14:34.294651+0000","last_undegraded":"2024-09-17T08:14:34.294651+0000","last_fullsized":"2024-09-17T08:14:34.294651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:12:54.883622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:54.883622+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.125166219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:42.883167+0000","last_change":"2024-09-17T08:14:33.194618+0000","last_active":"2024-09-17T08:14:42.883167+0000","last_peered":"2024-09-17T08:14:42.883167+0000","last_clean":"2024-09-17T08:14:42.883167+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:14:42.883167+0000","last_undegraded":"2024-09-17T08:14:42.883167+0000","last_fullsized":"2024-09-17T08:14:42.883167+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:14:33.194496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:33.194496+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:45.510334+0000","scrub_duration":0.0014499820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":101,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.284835+0000","last_change":"2024-09-17T08:12:42.853739+0000","last_active":"2024-09-17T08:14:34.284835+0000","last_peered":"2024-09-17T08:14:34.284835+0000","last_clean":"2024-09-17T08:14:34.284835+0000","last_became_active":"2024-09-17T08:10:00.523634+0000","last_became_peered":"2024-09-17T08:10:00.523634+0000","last_unstale":"2024-09-17T08:14:34.284835+0000","last_undegraded":"2024-09-17T08:14:34.284835+0000","last_fullsized":"2024-09-17T08:14:34.284835+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:42.853611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:42.853611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046570830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294866+0000","last_change":"2024-09-17T08:14:04.365914+0000","last_active":"2024-09-17T08:14:34.294866+0000","last_peered":"2024-09-17T08:14:34.294866+0000","last_clean":"2024-09-17T08:14:34.294866+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:14:34.294866+0000","last_undegraded":"2024-09-17T08:14:34.294866+0000","last_fullsized":"2024-09-17T08:14:34.294866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:04.365774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:04.365774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:10.392202+0000","scrub_duration":0.0083787500000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":90,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290380+0000","last_change":"2024-09-17T08:12:47.057999+0000","last_active":"2024-09-17T08:14:34.290380+0000","last_peered":"2024-09-17T08:14:34.290380+0000","last_clean":"2024-09-17T08:14:34.290380+0000","last_became_active":"2024-09-17T08:12:46.184351+0000","last_became_peered":"2024-09-17T08:12:46.184351+0000","last_unstale":"2024-09-17T08:14:34.290380+0000","last_undegraded":"2024-09-17T08:14:34.290380+0000","last_fullsized":"2024-09-17T08:14:34.290380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:12:47.057855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:12:47.057855+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066983809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":104,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.294578+0000","last_change":"2024-09-17T08:12:45.754294+0000","last_active":"2024-09-17T08:14:34.294578+0000","last_peered":"2024-09-17T08:14:34.294578+0000","last_clean":"2024-09-17T08:14:34.294578+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:14:34.294578+0000","last_undegraded":"2024-09-17T08:14:34.294578+0000","last_fullsized":"2024-09-17T08:14:34.294578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:45.754216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:12:45.754216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001170784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":102,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264704+0000","last_change":"2024-09-17T08:13:53.296955+0000","last_active":"2024-09-17T08:14:34.264704+0000","last_peered":"2024-09-17T08:14:34.264704+0000","last_clean":"2024-09-17T08:14:34.264704+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:14:34.264704+0000","last_undegraded":"2024-09-17T08:14:34.264704+0000","last_fullsized":"2024-09-17T08:14:34.264704+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:53.296823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:13:53.296823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.838694+0000","scrub_duration":0.0020070050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":58,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264019+0000","last_change":"2024-09-17T08:12:46.386848+0000","last_active":"2024-09-17T08:14:34.264019+0000","last_peered":"2024-09-17T08:14:34.264019+0000","last_clean":"2024-09-17T08:14:34.264019+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:14:34.264019+0000","last_undegraded":"2024-09-17T08:14:34.264019+0000","last_fullsized":"2024-09-17T08:14:34.264019+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:12:46.386759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014658869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":162,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.277519+0000","last_change":"2024-09-17T08:13:58.963656+0000","last_active":"2024-09-17T08:14:34.277519+0000","last_peered":"2024-09-17T08:14:34.277519+0000","last_clean":"2024-09-17T08:14:34.277519+0000","last_became_active":"2024-09-17T08:11:32.436438+0000","last_became_peered":"2024-09-17T08:11:32.436438+0000","last_unstale":"2024-09-17T08:14:34.277519+0000","last_undegraded":"2024-09-17T08:14:34.277519+0000","last_fullsized":"2024-09-17T08:14:34.277519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:13:58.963319+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.963319+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:01.183605+0000","scrub_duration":0.010144426,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.285077+0000","last_change":"2024-09-17T08:14:01.636987+0000","last_active":"2024-09-17T08:14:34.285077+0000","last_peered":"2024-09-17T08:14:34.285077+0000","last_clean":"2024-09-17T08:14:34.285077+0000","last_became_active":"2024-09-17T08:09:58.506942+0000","last_became_peered":"2024-09-17T08:09:58.506942+0000","last_unstale":"2024-09-17T08:14:34.285077+0000","last_undegraded":"2024-09-17T08:14:34.285077+0000","last_fullsized":"2024-09-17T08:14:34.285077+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:14:01.636849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:20.255485+0000","scrub_duration":0.010498624999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":91,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.295882+0000","last_change":"2024-09-17T08:12:32.313526+0000","last_active":"2024-09-17T08:14:34.295882+0000","last_peered":"2024-09-17T08:14:34.295882+0000","last_clean":"2024-09-17T08:14:34.295882+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:14:34.295882+0000","last_undegraded":"2024-09-17T08:14:34.295882+0000","last_fullsized":"2024-09-17T08:14:34.295882+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:32.313398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:12:32.313398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011036097999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.295342+0000","last_change":"2024-09-17T08:12:43.354919+0000","last_active":"2024-09-17T08:14:34.295342+0000","last_peered":"2024-09-17T08:14:34.295342+0000","last_clean":"2024-09-17T08:14:34.295342+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:14:34.295342+0000","last_undegraded":"2024-09-17T08:14:34.295342+0000","last_fullsized":"2024-09-17T08:14:34.295342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:12:43.354544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:12:43.354544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053342249999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":128,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.290167+0000","last_change":"2024-09-17T08:13:34.227830+0000","last_active":"2024-09-17T08:14:34.290167+0000","last_peered":"2024-09-17T08:14:34.290167+0000","last_clean":"2024-09-17T08:14:34.290167+0000","last_became_active":"2024-09-17T08:09:56.505113+0000","last_became_peered":"2024-09-17T08:09:56.505113+0000","last_unstale":"2024-09-17T08:14:34.290167+0000","last_undegraded":"2024-09-17T08:14:34.290167+0000","last_fullsized":"2024-09-17T08:14:34.290167+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:13:34.227679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:34.227679+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073082720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":103,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-17T08:14:34.264374+0000","last_change":"2024-09-17T08:13:39.253357+0000","last_active":"2024-09-17T08:14:34.264374+0000","last_peered":"2024-09-17T08:14:34.264374+0000","last_clean":"2024-09-17T08:14:34.264374+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:14:34.264374+0000","last_undegraded":"2024-09-17T08:14:34.264374+0000","last_fullsized":"2024-09-17T08:14:34.264374+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:39.253277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:39.253277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099572529999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":4333961240,"num_objects":66132,"num_object_clones":0,"num_object_copies":198396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188664,"num_write_kb":8153409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13019525120,"data_stored":13019512904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85719,"ondisk_log_size":85719,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":694,"num_read_kb":451,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":285,"num_read_kb":280,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070528,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1458780,"kb_used_data":1351376,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":92286372,"statfs":{"total":95995035648,"available":94501244928,"internally_reserved":0,"allocated":1383809024,"data_stored":1383487569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":109967819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":1.0129999999999999,"15min":0.70499999999999996},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":27.751999999999999,"5min":27.751999999999999,"15min":27.751999999999999},"last":1.075},{"interface":"front","average":{"1min":1.3660000000000001,"5min":0.81699999999999995,"15min":0.63700000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":5.2380000000000004,"5min":5.2380000000000004,"15min":5.2380000000000004},"last":0.72899999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":0.76200000000000001,"15min":0.65600000000000003},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2959999999999998,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.70699999999999996,"15min":0.60299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.82299999999999995}]},{"osd":2,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":0.74399999999999999,"15min":0.60899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.6210000000000004,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.173,"5min":0.72699999999999998,"15min":0.64600000000000002},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":0.82199999999999995,"15min":0.629},"min":{"1min":0.314,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":11.298,"5min":11.298,"15min":11.298},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.6240000000000001,"5min":0.89100000000000001,"15min":0.69299999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.45500000000000002},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.306,"5min":2.21,"15min":2.21},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.307}]},{"osd":5,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2470000000000001,"5min":1.526,"15min":1.526},"last":0.40699999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.3320000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.35999999999999999}]}]},{"osd":6,"up_from":50,"seq":214748364869,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1433420,"kb_used_data":1330116,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":92311732,"statfs":{"total":95995035648,"available":94527213568,"internally_reserved":0,"allocated":1362038784,"data_stored":1361728901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":105773511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.3250000000000002,"5min":2.5569999999999999,"15min":1.1950000000000001},"min":{"1min":0.33200000000000002,"5min":0.216,"15min":0.216},"max":{"1min":28.991,"5min":73.757999999999996,"15min":73.757999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":3.044,"5min":2.6970000000000001,"15min":1.226},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":30.620000000000001,"5min":73.811000000000007,"15min":73.811000000000007},"last":0.24299999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.2450000000000001,"5min":0.96799999999999997,"15min":0.67100000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":28.288,"5min":28.288,"15min":28.288},"last":0.26900000000000002},{"interface":"front","average":{"1min":2.2949999999999999,"5min":0.94499999999999995,"15min":0.67600000000000005},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":28.939,"5min":28.939,"15min":28.939},"last":5.9560000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.4710000000000001,"15min":0.84399999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":5.5220000000000002,"5min":68.165000000000006,"15min":68.165000000000006},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.5449999999999999,"15min":0.85099999999999998},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.8840000000000003,"5min":67.997,"15min":67.997},"last":6.3410000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":2.9929999999999999,"5min":2.0350000000000001,"15min":1.0529999999999999},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":29.379999999999999,"5min":77.010999999999996,"15min":77.010999999999996},"last":5.6310000000000002},{"interface":"front","average":{"1min":2.5840000000000001,"5min":2.0089999999999999,"15min":1.0289999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":28.071999999999999,"5min":76.375,"15min":76.375},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.40400000000000003},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.514,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.181},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.433,"15min":0.38900000000000001},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.32100000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.673,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48299999999999998,"15min":0.439},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":1.645,"15min":1.645},"last":0.84499999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:13:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.34999999999999998}]}]},{"osd":5,"up_from":41,"seq":176093659210,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1129296,"kb_used_data":1062084,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92615856,"statfs":{"total":95995035648,"available":94838636544,"internally_reserved":0,"allocated":1087574016,"data_stored":1087263624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:02 2024","interfaces":[{"interface":"back","average":{"1min":2.4500000000000002,"5min":0.89800000000000002,"15min":0.63600000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":13.685,"5min":13.685,"15min":13.685},"last":1.778},{"interface":"front","average":{"1min":1.8520000000000001,"5min":0.85399999999999998,"15min":0.60999999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":14.503,"5min":14.503,"15min":14.503},"last":1.5269999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":0.74099999999999999,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":3.4319999999999999,"5min":5.056,"15min":5.056},"last":2.6459999999999999},{"interface":"front","average":{"1min":1.26,"5min":0.72999999999999998,"15min":0.57599999999999996},"min":{"1min":0.27700000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.0300000000000002,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.66800000000000004,"15min":0.57199999999999995},"min":{"1min":0.25600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.411,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":1.954},{"interface":"front","average":{"1min":1.3320000000000001,"5min":0.752,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5129999999999999,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":0.95799999999999996,"15min":0.64800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.648,"5min":16.648,"15min":16.648},"last":0.86499999999999999},{"interface":"front","average":{"1min":2.0449999999999999,"5min":0.94399999999999995,"15min":0.66600000000000004},"min":{"1min":0.30099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":16.803000000000001,"5min":16.803000000000001,"15min":16.803000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.374},"min":{"1min":0.27200000000000002,"5min":0.19,"15min":0.19},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.32600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:14:28 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.51000000000000001,"15min":0.438},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.48999999999999999,"15min":0.432},"min":{"1min":0.32800000000000001,"5min":0.254,"15min":0.254},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.34599999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.52700000000000002,"15min":0.48799999999999999},"min":{"1min":0.34899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.53300000000000003,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.51700000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1439316,"kb_used_data":1336012,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":92305836,"statfs":{"total":95995035648,"available":94521176064,"internally_reserved":0,"allocated":1368076288,"data_stored":1367758223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105773510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:41 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":1.268,"15min":0.75600000000000001},"min":{"1min":0.22600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":23.786000000000001,"5min":23.786000000000001,"15min":23.786000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":2.3769999999999998,"5min":1.157,"15min":0.72899999999999998},"min":{"1min":0.25700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":23.245000000000001,"5min":23.245000000000001,"15min":23.245000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:14:41 2024","interfaces":[{"interface":"back","average":{"1min":2.222,"5min":0.92900000000000005,"15min":0.66100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":23.135000000000002,"5min":23.135000000000002,"15min":23.135000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.026,"15min":0.67600000000000005},"min":{"1min":0.25600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":23.434999999999999,"5min":23.434999999999999,"15min":23.434999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Sep 17 08:14:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67100000000000004,"15min":0.57499999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.9620000000000002,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.70499999999999996,"15min":0.56399999999999995},"min":{"1min":0.30199999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.6739999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:14:41 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":2.972,"15min":1.3400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":154.137,"5min":154.137,"15min":154.137},"last":8.3010000000000002},{"interface":"front","average":{"1min":10.247999999999999,"5min":2.9350000000000001,"15min":1.3260000000000001},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":153.54400000000001,"5min":153.54400000000001,"15min":153.54400000000001},"last":8.2219999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:14:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.39800000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.42999999999999999,"15min":0.39700000000000002},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.33300000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.439,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.48999999999999999,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:14:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51200000000000001,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.504,"15min":0.46899999999999997},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.377}]}]},{"osd":3,"up_from":27,"seq":115964117075,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2534816,"kb_used_data":2386904,"kb_used_omap":1,"kb_used_meta":147582,"kb_avail":91210336,"statfs":{"total":95995035648,"available":93399384064,"internally_reserved":0,"allocated":2444189696,"data_stored":2443858820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151124424},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":3.2810000000000001,"5min":1.1879999999999999,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.189,"15min":0.189},"max":{"1min":8.6829999999999998,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":2.5779999999999998,"5min":0.999,"15min":0.58499999999999996},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":10.362,"5min":10.362,"15min":10.362},"last":1.325}]},{"osd":1,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":2.9060000000000001,"5min":1.109,"15min":0.64700000000000002},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":8.4410000000000007,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":1.4319999999999999},{"interface":"front","average":{"1min":2.1389999999999998,"5min":0.90500000000000003,"15min":0.58099999999999996},"min":{"1min":0.28000000000000003,"5min":0.188,"15min":0.188},"max":{"1min":5.2039999999999997,"5min":5.2039999999999997,"15min":5.2039999999999997},"last":1.01}]},{"osd":2,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":2.762,"5min":1.113,"15min":0.64300000000000002},"min":{"1min":0.28999999999999998,"5min":0.246,"15min":0.246},"max":{"1min":10.786,"5min":10.786,"15min":10.786},"last":0.376},{"interface":"front","average":{"1min":3.165,"5min":1.2729999999999999,"15min":0.69699999999999995},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":8.6440000000000001,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":7.9219999999999997}]},{"osd":4,"last update":"Tue Sep 17 08:14:39 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":0.94799999999999995,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":5.1299999999999999,"5min":5.1299999999999999,"15min":5.1299999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.3009999999999999,"5min":0.98199999999999998,"15min":0.68799999999999994},"min":{"1min":0.35299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.226,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":1.3740000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.903,"5min":0.89400000000000002,"15min":0.67800000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.1829999999999998,"5min":5.1829999999999998,"15min":5.1829999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.4410000000000001,"5min":0.93000000000000005,"15min":0.71099999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":5.5730000000000004,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.482}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":1.7989999999999999,"5min":1.107,"15min":0.748},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.2830000000000004,"5min":10.945,"15min":10.945},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.0189999999999999,"15min":0.71999999999999997},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":4.665,"5min":5.798,"15min":5.798},"last":1.5169999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":0.93100000000000005,"15min":0.70399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.488,"5min":0.83899999999999997,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":5.1600000000000001,"5min":5.1600000000000001,"15min":5.1600000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1709528,"kb_used_data":1606220,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":92035624,"statfs":{"total":95995035648,"available":94244478976,"internally_reserved":0,"allocated":1644769280,"data_stored":1644451204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105773510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":3.5089999999999999,"5min":1.089,"15min":0.58099999999999996},"min":{"1min":0.26100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":22.399000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":4.4630000000000001,"5min":1.2889999999999999,"15min":0.66500000000000004},"min":{"1min":0.27500000000000002,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":23.341000000000001,"5min":23.341000000000001,"15min":23.341000000000001},"last":3.2480000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":0.61099999999999999,"15min":0.45100000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":4.2469999999999999},{"interface":"front","average":{"1min":1.706,"5min":0.68799999999999994,"15min":0.48499999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":3.907,"5min":3.907,"15min":3.907},"last":3.3039999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:14:19 2024","interfaces":[{"interface":"back","average":{"1min":2.0830000000000002,"5min":1.1870000000000001,"15min":0.67100000000000004},"min":{"1min":0.31,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":4.6159999999999997,"5min":15.065,"15min":15.065},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.738,"5min":1.0609999999999999,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.219,"15min":0.219},"max":{"1min":3.847,"5min":15.555999999999999,"15min":15.555999999999999},"last":3.2810000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":0.98699999999999999,"15min":0.69499999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.383,"5min":5.6150000000000002,"15min":5.6150000000000002},"last":0.499},{"interface":"front","average":{"1min":1.1870000000000001,"5min":0.90400000000000003,"15min":0.67700000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.714,"5min":4.9820000000000002,"15min":4.9820000000000002},"last":0.39600000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:14:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.755,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.2149999999999999,"5min":3.2149999999999999,"15min":3.2149999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":0.83599999999999997,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.52000000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:14:24 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.99199999999999999,"15min":0.72099999999999997},"min":{"1min":0.371,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.3200000000000003,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.93400000000000005,"15min":0.72399999999999998},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8479999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.42799999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:13:55 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.74299999999999999,"15min":0.62},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":1.542},{"interface":"front","average":{"1min":1.6679999999999999,"5min":0.80300000000000005,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":0.47999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1691488,"kb_used_data":1579992,"kb_used_omap":1,"kb_used_meta":111486,"kb_avail":92053664,"statfs":{"total":95995035648,"available":94262951936,"internally_reserved":0,"allocated":1617911808,"data_stored":1617581216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114162118},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:14:38 2024","interfaces":[{"interface":"back","average":{"1min":2.3620000000000001,"5min":1.4430000000000001,"15min":0.66600000000000004},"min":{"1min":0.19,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":7.8470000000000004,"5min":23.834,"15min":23.834},"last":0.745},{"interface":"front","average":{"1min":2.5470000000000002,"5min":1.4630000000000001,"15min":0.68600000000000005},"min":{"1min":0.22900000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":7.5730000000000004,"5min":22.402999999999999,"15min":22.402999999999999},"last":3.0169999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:13:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":0.78300000000000003,"15min":0.49199999999999999},"min":{"1min":0.20100000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":7.2649999999999997,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":1.1899999999999999},{"interface":"front","average":{"1min":2.3180000000000001,"5min":0.85199999999999998,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.3860000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":2.9740000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:14:19 2024","interfaces":[{"interface":"back","average":{"1min":2.5649999999999999,"5min":1.038,"15min":0.61799999999999999},"min":{"1min":0.23300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.8499999999999996,"5min":7.8499999999999996,"15min":7.8499999999999996},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.2360000000000002,"5min":1.006,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.189,"15min":0.189},"max":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":2.9390000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:14:45 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":0.98499999999999999,"15min":0.68500000000000005},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":7.4859999999999998,"5min":7.4859999999999998,"15min":7.4859999999999998},"last":2.9969999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.88800000000000001,"15min":0.66400000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9319999999999999,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:14:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.75600000000000001,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.0869999999999997,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.0589999999999999},{"interface":"front","average":{"1min":2.0499999999999998,"5min":0.84999999999999998,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":3.0369999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:14:22 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":0.88,"15min":0.64700000000000002},"min":{"1min":0.378,"5min":0.318,"15min":0.318},"max":{"1min":2.8999999999999999,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3240000000000001,"5min":0.86899999999999999,"15min":0.68700000000000006},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.98699999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":0.81799999999999995,"15min":0.65200000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":6.3380000000000001,"5min":6.3380000000000001,"15min":6.3380000000000001},"last":3.0579999999999998},{"interface":"front","average":{"1min":1.6639999999999999,"5min":0.86499999999999999,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2298140,"kb_used_data":2150480,"kb_used_omap":1,"kb_used_meta":147582,"kb_avail":91447012,"statfs":{"total":95995035648,"available":93641740288,"internally_reserved":0,"allocated":2202091520,"data_stored":2201769773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151124422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,6,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:14:32 2024","interfaces":[{"interface":"back","average":{"1min":2.2469999999999999,"5min":1.02,"15min":0.55300000000000005},"min":{"1min":0.27000000000000002,"5min":0.20399999999999999,"15min":0.19900000000000001},"max":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":2.222,"5min":1.179,"15min":0.61799999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":6.4749999999999996,"5min":6.4749999999999996,"15min":6.4749999999999996},"last":0.88}]},{"osd":2,"last update":"Tue Sep 17 08:13:53 2024","interfaces":[{"interface":"back","average":{"1min":2.7869999999999999,"5min":1.0309999999999999,"15min":0.60099999999999998},"min":{"1min":0.312,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":12.355,"5min":12.355,"15min":12.355},"last":0.746},{"interface":"front","average":{"1min":2.3450000000000002,"5min":0.93500000000000005,"15min":0.58499999999999996},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":7.8639999999999999,"5min":7.8639999999999999,"15min":7.8639999999999999},"last":0.97799999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:14:22 2024","interfaces":[{"interface":"back","average":{"1min":2.3900000000000001,"5min":1.3009999999999999,"15min":0.73399999999999999},"min":{"1min":0.35799999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.731999999999999,"5min":12.731999999999999,"15min":12.731999999999999},"last":1.7629999999999999},{"interface":"front","average":{"1min":2.6779999999999999,"5min":1.093,"15min":0.68100000000000005},"min":{"1min":0.312,"5min":0.251,"15min":0.251},"max":{"1min":12.965,"5min":12.965,"15min":12.965},"last":1.296}]},{"osd":4,"last update":"Tue Sep 17 08:14:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.054,"15min":0.72499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.6890000000000001,"5min":5.6890000000000001,"15min":5.6890000000000001},"last":0.313},{"interface":"front","average":{"1min":1.877,"5min":1.0840000000000001,"15min":0.752},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.548,"5min":6.548,"15min":6.548},"last":0.39300000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:14:07 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":0.84299999999999997,"15min":0.66000000000000003},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.835,"5min":7.835,"15min":7.835},"last":0.28899999999999998},{"interface":"front","average":{"1min":1.71,"5min":0.97399999999999998,"15min":0.71299999999999997},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.7130000000000001,"5min":6.8789999999999996,"15min":6.8789999999999996},"last":0.34200000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:14:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":0.93300000000000005,"15min":0.69799999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.8899999999999997,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.038,"15min":0.73999999999999999},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.36799999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:13:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":0.95599999999999996,"15min":0.69799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.6859999999999999,"5min":0.90200000000000002,"15min":0.68400000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":0.44400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2167541760,"data_stored":2167537688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1603997696,"data_stored":1603993624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1627586560,"data_stored":1627586560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2436956160,"data_stored":2436956160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364721664,"data_stored":1364721664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1084227584,"data_stored":1084227584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1358696448,"data_stored":1358692376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1375797248,"data_stored":1375797248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:14:52.308 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-17T08:14:52.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:14:52.308 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:14:52.309 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-17T08:14:52.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 5 0 2024-09-17T08:14:52.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.434 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:52.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:52.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:52.578 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:14:52.578 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:14:52.578 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:14:52.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:14:52.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:14:52.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:14:52.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 2.a deep-scrub starts 2024-09-17T08:14:52.579 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[29565]: 2.a deep-scrub ok 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 2.a deep-scrub starts 2024-09-17T08:14:52.580 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:52 smithi079 ceph-mon[40649]: 2.a deep-scrub ok 2024-09-17T08:14:52.586 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:52.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:52.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:52.616 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:14:52.617 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:14:52.660 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:14:52.660 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:14:52.696 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:14:52.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:14:52.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:52.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 2.a deep-scrub starts 2024-09-17T08:14:52.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:52 smithi173 ceph-mon[33187]: 2.a deep-scrub ok 2024-09-17T08:14:52.725 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:14:52.725 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:14:52.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.735 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:52.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:52.736 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:14:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:14:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:14:52.736 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:14:52.736 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:14:52.760 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:14:52.760 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:14:52.793 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:14:52.793 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:14:52.825 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:14:52.825 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:14:52.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:52.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:52.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:52.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:52.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:53.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:53.111 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:53.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:14:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:14:53.136 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:53.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:53.275 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:53.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:53.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:53.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:14:53.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:53.558 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 5.11 deep-scrub starts 2024-09-17T08:14:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 5.11 deep-scrub ok 2024-09-17T08:14:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3129210682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[40649]: pgmap v199: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 5.11 deep-scrub starts 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 5.11 deep-scrub ok 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3129210682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:53.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:14:53.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:14:53.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:53 smithi079 ceph-mon[29565]: pgmap v199: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:14:53.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:53.695 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:53.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 5.11 deep-scrub starts 2024-09-17T08:14:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 5.11 deep-scrub ok 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3129210682' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:14:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:53 smithi173 ceph-mon[33187]: pgmap v199: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:14:53.781 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:14:53.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:53.832 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:53.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:53.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:53.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:53.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:54.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:54.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:54.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:54.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:54.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:54.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:54.356 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:54.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:54.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:54.423 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:54.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:54.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:54.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:54.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:54.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:54.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:54.741 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:14:54.741 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:14:54.741 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:14:54.741 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:14:54.742 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:14:54.742 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:14:54.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:54.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:54.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:54.884 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:54 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:14:54.884 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:54 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:14:54.884 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:14:54.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:54.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:54.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:54.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:55.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:55.117 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:55.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:55.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:55.258 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:55.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:55.360 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:55.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:55.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:55.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:55.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.530 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:55.530 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:55.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:55.655 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:55.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:55.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:55.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:55.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:55.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:55 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:14:55.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:55 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:14:55.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:55 smithi173 ceph-mon[33187]: 5.1e deep-scrub starts 2024-09-17T08:14:55.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:55 smithi173 ceph-mon[33187]: 5.1e deep-scrub ok 2024-09-17T08:14:55.805 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:55 smithi173 ceph-mon[33187]: pgmap v200: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:14:55.838 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:14:55.838 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[40649]: 5.1e deep-scrub starts 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[40649]: 5.1e deep-scrub ok 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[40649]: pgmap v200: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[29565]: 5.1e deep-scrub starts 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[29565]: 5.1e deep-scrub ok 2024-09-17T08:14:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:55 smithi079 ceph-mon[29565]: pgmap v200: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:14:55.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:55.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:55.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:55.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:55.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:14:56.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:56.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:56.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:56.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:56.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:56.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:56.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:56.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:56.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:56.365 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:14:56.365 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:14:56.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:56.496 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:56.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:56.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:56.713 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:14:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:14:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2264250298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T08:14:56.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2264250298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:14:56.731 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2264250298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:14:56.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:56.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:56.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:56.905 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:56.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:56.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:57.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:57.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.048 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:57.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:57.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:14:57.149 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:57.174 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:57.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:57.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:57.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:57.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:57.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:57.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:57.556 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:57.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:57.633 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:57.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-17T08:14:57.735 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:57.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:57.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:57.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/219183757' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[40649]: pgmap v203: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 647 B/s rd, 0 B/s wr, 765 op/s 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/219183757' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:57 smithi079 ceph-mon[29565]: pgmap v203: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 647 B/s rd, 0 B/s wr, 765 op/s 2024-09-17T08:14:57.883 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:57.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:57.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:14:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:14:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/219183757' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-17T08:14:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:57 smithi173 ceph-mon[33187]: pgmap v203: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 647 B/s rd, 0 B/s wr, 765 op/s 2024-09-17T08:14:57.985 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:58.007 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:58.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:58.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:58.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:58.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:58.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:58.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:58.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:58.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:14:58.377 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:58.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:14:58.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:14:58.502 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:58.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:14:58.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:58.626 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:58.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:58.750 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:58.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:58.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:14:58.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:58.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:58.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:14:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:58 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-17T08:14:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:58 smithi173 ceph-mon[33187]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T08:14:58.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:59.004 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:59.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:59.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:58 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-17T08:14:59.085 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:58 smithi079 ceph-mon[29565]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T08:14:59.085 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:58 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-17T08:14:59.085 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:58 smithi079 ceph-mon[40649]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T08:14:59.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:59.144 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:59.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:59.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:59.284 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:59.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:59.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:14:59.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:59.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:14:59.480 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:14:59.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:14:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.572 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:59.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:14:59.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:14:59.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:14:59.717 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:14:59.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:14:59.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:14:59.844 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:14:59.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:14:59.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:14:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:59 smithi079 ceph-mon[29565]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T08:14:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:14:59 smithi079 ceph-mon[29565]: pgmap v206: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:14:59.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:59 smithi079 ceph-mon[40649]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T08:14:59.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:14:59 smithi079 ceph-mon[40649]: pgmap v206: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:14:59.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:00.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:00.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:00.111 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:00.137 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:00.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:00.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:59 smithi173 ceph-mon[33187]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T08:15:00.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:14:59 smithi173 ceph-mon[33187]: pgmap v206: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:15:00.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:00.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.285 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:00.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:00.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:00.410 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:00.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:00.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:00.552 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:00.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.553 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:00.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:00.681 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:00.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:00.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:00.810 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:00.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.811 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:00.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:00.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:00.951 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:00.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:01.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:01.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:01.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:01.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:01.243 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:01.244 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:15:01.244 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:15:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:01.285 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:15:01.286 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:15:01.315 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2591637785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:01.316 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2591637785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:01.324 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:15:01.324 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:15:01.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:01.357 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:15:01.357 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:15:01.370 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:01.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:01.389 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:15:01.389 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:15:01.421 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:15:01.421 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:15:01.452 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:15:01.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:15:01.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:01.488 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:15:01.489 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:15:01.506 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:01.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:01.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:01.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:01.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2591637785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:01.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:01.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.773 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:01.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:01.859 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:01.874 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:01.897 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:01.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:01.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:02.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:02.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.039 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:02.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:02.141 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:02.167 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:02.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:02.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:02.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:02.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:02.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:02.423 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:02.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:02.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:02.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:02.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:02.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[29565]: pgmap v207: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 947 op/s 2024-09-17T08:15:02.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:02.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[40649]: pgmap v207: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 947 op/s 2024-09-17T08:15:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:02.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:02.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:02.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:02 smithi173 ceph-mon[33187]: pgmap v207: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 947 op/s 2024-09-17T08:15:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:02.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:02.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:02.819 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:02.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:02.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:02.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:02.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:02.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:204703 7.6 7.ef30156 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.010980+0000", 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0011493650000000001, 2024-09-17T08:15:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011753060000000001, 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 204703 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010980+0000", 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010980+0000", 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010979+0000", 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010985+0000", 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010996+0000", 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1202e-05 2024-09-17T08:15:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011003+0000", 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:204704 7.9 7:9ee86439:::benchmark_data_smithi079_83551_object77304:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-17T08:15:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.011009+0000", 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0011211940000000001, 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001207533, 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 204704 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011009+0000", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011009+0000", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011008+0000", 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011010+0000", 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011016+0000", 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999997e-06 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011020+0000", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.279e-06 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011060+0000", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0027000000000001e-05 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011179+0000", 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011948899999999999 2024-09-17T08:15:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011288+0000", 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109028 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:204706 7.5 e145/91 7:a5fff9ac:::benchmark_data_smithi079_83551_object77306:head v 145'12917, mlcod=145'12915)", 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.011717+0000", 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.000412855, 2024-09-17T08:15:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00056556399999999995, 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011717+0000", 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011717+0000", 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011716+0000", 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011724+0000", 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0660000000000004e-06 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011735+0000", 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0703000000000001e-05 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011742+0000", 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5690000000000001e-06 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011861+0000", 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119009 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.011917+0000", 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6427e-05 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:204697 7.7 7:e65c2cb5:::benchmark_data_smithi079_83551_object77297:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.007145+0000", 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0049850820000000001, 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051933040000000001, 2024-09-17T08:15:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 204697 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007145+0000", 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007145+0000", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007143+0000", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007150+0000", 2024-09-17T08:15:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.99e-06 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007161+0000", 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.146e-05 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007168+0000", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1010000000000003e-06 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007213+0000", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4477999999999997e-05 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007336+0000", 2024-09-17T08:15:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122872 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.007460+0000", 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012437799999999999 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010911+0000", 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034507489999999999 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:204696 7.d e145/103 7:b193518f:::benchmark_data_smithi079_83551_object77296:head v 145'12699, mlcod=145'12697)", 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.008083+0000", 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0040467740000000004, 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043265730000000002, 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008083+0000", 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008083+0000", 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008083+0000", 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008087+0000", 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3549999999999998e-06 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008093+0000", 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2349999999999996e-06 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008098+0000", 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2490000000000003e-06 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008660+0000", 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00056240700000000001 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.008678+0000", 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8108999999999999e-05 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:204695 7.e e145/91 7:71de9711:::benchmark_data_smithi079_83551_object77295:head v 145'12987, mlcod=145'12984)", 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:03.010620+0000", 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0015100070000000001, 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018423910000000001, 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010620+0000", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010620+0000", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010619+0000", 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010624+0000", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6090000000000003e-06 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010630+0000", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8579999999999996e-06 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010634+0000", 2024-09-17T08:15:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9280000000000003e-06 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010918+0000", 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028442499999999999 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:03.010934+0000", 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5690000000000001e-05 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:15:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 6 2024-09-17T08:15:03.040 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:15:03.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:03.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:03.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:03.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:03.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:03.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.238 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:03.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:03.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:03.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:03.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:03.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:03.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.503 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:03.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:03.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:03.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[40649]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T08:15:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:03 smithi079 ceph-mon[29565]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T08:15:03.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:03.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:03.715 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:15:03.715 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:15:03.715 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:15:03.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd out 1 2024-09-17T08:15:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:03 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:03 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:03 smithi173 ceph-mon[33187]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T08:15:03.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:03.754 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:03.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:03.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:03.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:03.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:03.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:04.007 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:04.032 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:04.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:04.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:04.167 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:04.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:04.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:04.294 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:04.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:04.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:04.420 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:04.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:04.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:04.562 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:04.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:04 smithi079 ceph-mon[40649]: pgmap v209: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-17T08:15:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:04 smithi079 ceph-mon[29565]: pgmap v209: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-17T08:15:04.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:04.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:04.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:04 smithi173 ceph-mon[33187]: pgmap v209: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-17T08:15:04.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:04.817 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:04.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:04.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:04.952 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:04.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:04.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:05.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:05.087 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:05.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.088 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:05.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:05.214 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:05.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:05.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:05.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:05.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:05.469 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:05.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:05.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:05.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.616 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:05.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:05.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:15:05.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:05.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:05.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:05.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:05.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:05.868 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:05.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:05.969 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:06.014 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:06.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:06.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:06.154 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:06.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:06.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:06.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:06.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:06.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:06.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:06.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:06.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:06.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:06.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:06.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:06 smithi079 ceph-mon[40649]: pgmap v210: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 539 B/s rd, 0 B/s wr, 849 op/s 2024-09-17T08:15:06.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:06 smithi079 ceph-mon[29565]: pgmap v210: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 539 B/s rd, 0 B/s wr, 849 op/s 2024-09-17T08:15:06.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:06.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:06 smithi173 ceph-mon[33187]: pgmap v210: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 539 B/s rd, 0 B/s wr, 849 op/s 2024-09-17T08:15:06.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:06.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:06.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:06.893 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:06.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:06.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:06.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:06.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:06.929 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:07.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:07.058 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:07.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:07.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:07.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:07.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:07.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:07.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:07.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:07.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:07.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:07.683 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:07.710 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:07.812 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:07.838 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:07.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:07.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:07.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:07.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:07.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:08.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:08.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:08.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:08.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:08.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:08.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:08.364 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:08.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:15:08.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:08.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:08.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:08.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:08.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:08 smithi079 ceph-mon[29565]: pgmap v211: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 476 B/s rd, 0 B/s wr, 628 op/s 2024-09-17T08:15:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:08 smithi079 ceph-mon[40649]: pgmap v211: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 476 B/s rd, 0 B/s wr, 628 op/s 2024-09-17T08:15:08.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:08.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:08.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:08 smithi173 ceph-mon[33187]: pgmap v211: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 476 B/s rd, 0 B/s wr, 628 op/s 2024-09-17T08:15:08.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:08.786 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:08.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:08.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:08.935 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:08.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:08.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:09.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:09.059 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:09.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:09.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:09.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.200 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:09.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:09.301 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:09.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:09.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:09.404 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1407358101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.404 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:09 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.405 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1407358101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.405 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:09 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:09.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:09.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:09.479 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:15:09.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:09.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:09.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:09.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:09.705 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:09.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1407358101' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:09 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:09.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:09.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:09.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:09.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:09.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:09.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:10.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:10.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:10.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:10.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:10.247 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:10.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:10.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:10.391 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:10.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:10.528 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:10.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:10.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:10.657 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:10.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:10 smithi173 ceph-mon[33187]: pgmap v212: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:15:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:10 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:10 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:10 smithi173 ceph-mon[33187]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T08:15:10.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:10.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:10.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[29565]: pgmap v212: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[29565]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[40649]: pgmap v212: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:15:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:10 smithi079 ceph-mon[40649]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T08:15:10.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:10.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:10.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:10.923 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:11.024 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:11.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:11.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:11.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:11.186 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:11.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:11.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:11.324 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:11.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.325 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:11.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:11.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:11.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:11.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:11.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:11.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:11.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:11.705 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:11.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:11.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:11.844 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:11.946 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:11.972 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:11.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:11.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:12.074 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:12.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:12.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:12.204 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:12.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.233 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:12.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:12.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:12.372 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:12.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:12.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:12.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:12.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:12.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:12 smithi079 ceph-mon[29565]: pgmap v214: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 586 B/s rd, 0 B/s wr, 817 op/s 2024-09-17T08:15:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:12 smithi079 ceph-mon[40649]: pgmap v214: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 586 B/s rd, 0 B/s wr, 817 op/s 2024-09-17T08:15:12.646 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:12.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:12 smithi173 ceph-mon[33187]: pgmap v214: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 586 B/s rd, 0 B/s wr, 817 op/s 2024-09-17T08:15:12.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:12.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:12.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.783 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:12.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:12.884 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:12.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:12.910 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:12.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:13.011 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:13.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.038 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:13.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:13.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.163 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:13.264 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:13.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.291 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:13.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:13.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.421 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:13.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:13.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:13.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:13.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:13.679 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:13.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:13.806 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:13.908 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:13.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:13.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:13.935 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:14.036 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:14.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:14.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:14.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:14.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:14.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:14.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:15:14.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.347 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:14.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:14.422 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:15:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:15:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:15:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:15:14.422 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:15:14.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:14.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:14.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:14.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:14.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:14.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:14 smithi079 ceph-mon[29565]: pgmap v215: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:14.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:14.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:14 smithi079 ceph-mon[40649]: pgmap v215: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:14.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:14.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:14.696 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:14 smithi173 ceph-mon[33187]: pgmap v215: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:14.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:14.803 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:14.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:14.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:14.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:14.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:15.045 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:15.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:15.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:15.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:14 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:15:14.676+0000 7f02ec2c3640 -1 mon.a@0(leader).osd e146 definitely_dead 0 2024-09-17T08:15:15.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:15.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:15:15.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:15.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:15.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:15.419 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.1. 2024-09-17T08:15:15.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:15.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.502 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:15.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:15.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:15.626 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:15.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/653486564' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[40649]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-17T08:15:15.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[40649]: pgmap v216: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:15.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/653486564' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[29565]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-17T08:15:15.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:15 smithi079 ceph-mon[29565]: pgmap v216: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/653486564' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:15 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-17T08:15:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:15 smithi173 ceph-mon[33187]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-17T08:15:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:15 smithi173 ceph-mon[33187]: pgmap v216: 148 pgs: 148 active+clean; 2.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-17T08:15:15.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:15.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:15.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:15.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:15.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:15.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:15.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:15.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:16.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:16.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:16.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:16.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:16.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:16.367 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:16.391 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:16.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:16.493 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:16.517 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:16.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:16.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[40649]: osdmap e147: 8 total, 8 up, 7 in 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[40649]: osdmap e148: 8 total, 8 up, 7 in 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[29565]: osdmap e147: 8 total, 8 up, 7 in 2024-09-17T08:15:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:16 smithi079 ceph-mon[29565]: osdmap e148: 8 total, 8 up, 7 in 2024-09-17T08:15:16.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:16.643 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:16 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-17T08:15:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:16 smithi173 ceph-mon[33187]: osdmap e147: 8 total, 8 up, 7 in 2024-09-17T08:15:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:16 smithi173 ceph-mon[33187]: osdmap e148: 8 total, 8 up, 7 in 2024-09-17T08:15:16.743 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:16.769 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:16.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:16.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:16.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:16.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:16.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:16.996 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:17.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:17.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:17.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:17.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:17.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:17.381 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:17.409 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:17.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:17.539 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[40649]: osdmap e149: 8 total, 8 up, 7 in 2024-09-17T08:15:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[40649]: pgmap v220: 148 pgs: 148 active+clean; 2.4 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 999 op/s; 146 B/s, 7 objects/s recovering 2024-09-17T08:15:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[29565]: osdmap e149: 8 total, 8 up, 7 in 2024-09-17T08:15:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:17 smithi079 ceph-mon[29565]: pgmap v220: 148 pgs: 148 active+clean; 2.4 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 999 op/s; 146 B/s, 7 objects/s recovering 2024-09-17T08:15:17.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:17.669 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:17.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:17.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:17 smithi173 ceph-mon[33187]: osdmap e149: 8 total, 8 up, 7 in 2024-09-17T08:15:17.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:17 smithi173 ceph-mon[33187]: pgmap v220: 148 pgs: 148 active+clean; 2.4 GiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 999 op/s; 146 B/s, 7 objects/s recovering 2024-09-17T08:15:17.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:17.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:17.840 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:17.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:17.927 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:17.929 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:17.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:17.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:18.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:18.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:18.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:18.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:18.215 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:18.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:18.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:18.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.348 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:18.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:18.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:18.490 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:18.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:18.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:18.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:18.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 146'13391 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 146'13556 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 146'15515 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13375 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13738 2024-09-17T08:15:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:18 smithi173 ceph-mon[33187]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 146'13535 2024-09-17T08:15:18.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:18.768 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:18.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:18.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 146'13391 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 146'13556 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 146'15515 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13375 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13738 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[40649]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 146'13535 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 146'13391 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 146'13556 2024-09-17T08:15:18.840 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 146'15515 2024-09-17T08:15:18.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13375 2024-09-17T08:15:18.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 146'13738 2024-09-17T08:15:18.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:18 smithi079 ceph-mon[29565]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 146'13535 2024-09-17T08:15:18.871 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:18.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:18.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:18.899 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:18.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:19.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:19.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:19.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:19.165 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:19.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:19.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.307 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:19.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:19.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:19.434 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:19.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:19.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:19.562 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:19.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:19.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:19.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:19.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:19.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:19 smithi173 ceph-mon[33187]: pgmap v221: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 558 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:19.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:19.804 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:19 smithi079 ceph-mon[29565]: pgmap v221: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 558 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:19.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:19 smithi079 ceph-mon[40649]: pgmap v221: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 558 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:19.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:19.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.846 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:19.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:19.985 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:19.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:19.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:20.087 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:20.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:20.214 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:20.240 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:20.342 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:20.369 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.370 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:20.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:20.497 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:20.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:20.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:20 smithi079 ceph-mon[29565]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:15:20.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:20 smithi079 ceph-mon[40649]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:15:20.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.623 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:20 smithi173 ceph-mon[33187]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:15:20.724 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:20.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.750 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:20.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:20.875 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:20.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:20.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:21.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.021 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:21.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:21.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:21.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:21.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:21.265 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:21.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:21.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:21.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:21.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:21.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:21.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:21.579 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:21.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.580 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:21.681 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:21.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.705 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:21.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:21.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:21.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:21.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:21.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:21.958 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:21.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:21.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:22.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:22.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:22.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:22.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:22.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:22.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:22.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:22.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:22.505 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:22 smithi173 ceph-mon[33187]: pgmap v222: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 559 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:22.505 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1653509736' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:22.513 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:22.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:22 smithi079 ceph-mon[29565]: pgmap v222: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 559 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1653509736' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:22 smithi079 ceph-mon[40649]: pgmap v222: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 2.2 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 559 op/s; 13127/108633 objects misplaced (12.084%); 1.8 MiB/s, 46 objects/s recovering 2024-09-17T08:15:22.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1653509736' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:22.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:22.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:22.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:22.834 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:15:22.834 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:15:22.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:15:22.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:15:22.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:15:22.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:15:22.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:15:22.870 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:15:22.870 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:15:22.870 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-17T08:15:22.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd in 1 2024-09-17T08:15:22.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:22.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:22.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:22.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:22.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:23.059 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:23.082 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:23.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:23.183 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:23.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:23.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:23.283 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:23.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:23.331 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:15:23.351 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:23.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:23.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:23.489 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:23.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.490 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:23.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:23.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1721382013' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1721382013' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:23.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1721382013' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:15:23.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:23.769 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:23.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:23.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:23.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:23.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:23.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:24.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:24.055 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:24.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:24.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:24.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:24.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:24.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:24.321 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:24.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:24.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:24.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:24.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:24.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:24.603 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:24.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[29565]: pgmap v223: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 357 objects/s recovering 2024-09-17T08:15:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:15:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[29565]: osdmap e150: 8 total, 8 up, 7 in 2024-09-17T08:15:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[40649]: pgmap v223: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 357 objects/s recovering 2024-09-17T08:15:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:15:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:24 smithi079 ceph-mon[40649]: osdmap e150: 8 total, 8 up, 7 in 2024-09-17T08:15:24.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:24 smithi173 ceph-mon[33187]: pgmap v223: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 357 objects/s recovering 2024-09-17T08:15:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:24 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:15:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:24 smithi173 ceph-mon[33187]: osdmap e150: 8 total, 8 up, 7 in 2024-09-17T08:15:24.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.749 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:24.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:24.822 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:24.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:24.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:24.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:24.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:24.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:25.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:25.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:25.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:25.171 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:25.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:25.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:25.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:25.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:25.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:25.428 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:25.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:25.464 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:25 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:15:25.465 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:25 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:15:25.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:25.573 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:25.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:25.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:25.701 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:25.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:25 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:15:25.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:25.842 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:25.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:25.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:25.972 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:25.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:25.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:26.073 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:26.100 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:26.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.101 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:26.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:26.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:26.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:26.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:15:26.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:26.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:26.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:26.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:26.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:26.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:26.585 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:26 smithi079 ceph-mon[29565]: pgmap v225: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 983 op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 348 objects/s recovering 2024-09-17T08:15:26.585 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:26 smithi079 ceph-mon[40649]: pgmap v225: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 983 op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 348 objects/s recovering 2024-09-17T08:15:26.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:26.642 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:15:26.642 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:15:26.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:26.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:26.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:26 smithi173 ceph-mon[33187]: pgmap v225: 148 pgs: 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 142 active+clean; 1.9 GiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 983 op/s; 9106/94854 objects misplaced (9.600%); 21 MiB/s, 348 objects/s recovering 2024-09-17T08:15:26.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:26.796 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:26.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:26.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:15:26.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:26.945 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:26.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:26.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:27.047 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:27.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:27.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:27.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.204 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:27.305 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:27.401 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:27.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:27.537 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:27.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:27.639 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:27.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:27.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:27.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:27.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:27.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:27.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:27.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:28.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:28.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:28.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:28.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:28.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:28.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:28.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:28.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:28.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:28.333 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:27 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:15:27.977+0000 7f02ec2c3640 -1 mon.a@0(leader).osd e150 definitely_dead 0 2024-09-17T08:15:28.349 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.1. 2024-09-17T08:15:28.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:28.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:28.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:28.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:28.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:28.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[29565]: pgmap v226: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.9 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 808 op/s; 9106/93147 objects misplaced (9.776%); 17 MiB/s, 277 objects/s recovering 2024-09-17T08:15:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3497974175' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[40649]: pgmap v226: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.9 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 808 op/s; 9106/93147 objects misplaced (9.776%); 17 MiB/s, 277 objects/s recovering 2024-09-17T08:15:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3497974175' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:28 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:28.670 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:28.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:28 smithi173 ceph-mon[33187]: pgmap v226: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.9 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 808 op/s; 9106/93147 objects misplaced (9.776%); 17 MiB/s, 277 objects/s recovering 2024-09-17T08:15:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:28 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3497974175' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:28 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-17T08:15:28.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:28.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:28.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:28.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:15:28.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:28.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:28.920 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:28.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:29.021 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:29.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:29.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:29.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:29.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:29.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:29.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:29.294 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:29.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:29.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:29.429 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:29.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:29.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:29.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:29.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:29 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-17T08:15:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:29 smithi079 ceph-mon[29565]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T08:15:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:29 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-17T08:15:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:29 smithi079 ceph-mon[40649]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T08:15:29.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:29.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:29.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:29 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-17T08:15:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:29 smithi173 ceph-mon[33187]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T08:15:29.778 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:29.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:29.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:29.887 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-17T08:15:29.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:29.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:29.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:29.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:29.956 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:30.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:30.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.048 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:30.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:30.149 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:30.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:30.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.173 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:30.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:30.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:30.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:15:30.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:15:30.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[40649]: pgmap v228: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 5124/80547 objects misplaced (6.362%); 42 MiB/s, 666 objects/s recovering 2024-09-17T08:15:30.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[40649]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T08:15:30.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:15:30.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:15:30.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[29565]: pgmap v228: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 5124/80547 objects misplaced (6.362%); 42 MiB/s, 666 objects/s recovering 2024-09-17T08:15:30.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:30 smithi079 ceph-mon[29565]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T08:15:30.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:30.427 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:30.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:30.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:30.565 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:30.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.566 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:30.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:30.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:30.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.704 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:30 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:15:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:30 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:15:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:30 smithi173 ceph-mon[33187]: pgmap v228: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 5124/80547 objects misplaced (6.362%); 42 MiB/s, 666 objects/s recovering 2024-09-17T08:15:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:30 smithi173 ceph-mon[33187]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T08:15:30.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:30.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.846 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:30.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:30.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:30.973 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:30.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:30.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:31.065 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:31.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:31.100 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:31.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:31.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:31.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:31.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:31.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:31.366 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:31.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:31.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:31.497 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:31.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[40649]: 7.5 scrub starts 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[40649]: 7.5 scrub ok 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[40649]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[29565]: 7.5 scrub starts 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[29565]: 7.5 scrub ok 2024-09-17T08:15:31.509 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:31 smithi079 ceph-mon[29565]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T08:15:31.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:31.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:31.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:31.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:31 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:15:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:31 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:15:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:31 smithi173 ceph-mon[33187]: 7.5 scrub starts 2024-09-17T08:15:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:31 smithi173 ceph-mon[33187]: 7.5 scrub ok 2024-09-17T08:15:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:31 smithi173 ceph-mon[33187]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T08:15:31.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:31.762 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:31.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:31.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:31.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:31.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:32.002 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:32.027 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:32.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:32.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:32.164 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:32.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:32.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:32.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.314 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:32.314 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:32.415 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:32.440 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:32.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.441 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:32.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:15:32.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:15:32.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[29565]: pgmap v231: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 795 op/s; 5124/80547 objects misplaced (6.362%); 29 MiB/s, 466 objects/s recovering 2024-09-17T08:15:32.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[40649]: pgmap v231: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 795 op/s; 5124/80547 objects misplaced (6.362%); 29 MiB/s, 466 objects/s recovering 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:32.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:32.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:32.576 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:32.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:32.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:32.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:32.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:15:32.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:15:32.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-mon[33187]: pgmap v231: 148 pgs: 1 active+clean+scrubbing, 5 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 141 active+clean; 1.6 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 795 op/s; 5124/80547 objects misplaced (6.362%); 29 MiB/s, 466 objects/s recovering 2024-09-17T08:15:32.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:32.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:32.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:32.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:32.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:32.949 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:32.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:32.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:32.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:33.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:33.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.099 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:33.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:33.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:33.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:33.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:33.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:33.347 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:33.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:33.363 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:33.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:33.470 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:33.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:33.572 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:33.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 7.d scrub starts 2024-09-17T08:15:33.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 7.c scrub starts 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[29565]: 7.d scrub ok 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 7.d scrub starts 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 7.c scrub starts 2024-09-17T08:15:33.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:33 smithi079 ceph-mon[40649]: 7.d scrub ok 2024-09-17T08:15:33.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 7.d scrub starts 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 7.c scrub starts 2024-09-17T08:15:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:33 smithi173 ceph-mon[33187]: 7.d scrub ok 2024-09-17T08:15:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:33.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:33.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:33.860 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:33.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:33.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:33.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:33.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:33.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:34.097 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:34.121 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:34.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:34.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:34.248 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:34.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:34.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:34.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:34.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:34.474 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: 7.c scrub ok 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: pgmap v232: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s; 32 MiB/s, 531 objects/s recovering 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:15:34.475 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: 7.c scrub ok 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: pgmap v232: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s; 32 MiB/s, 531 objects/s recovering 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:15:34.476 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:34 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:15:34.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:34.525 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:34.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:34.552 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:15:34.552 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:15:34.587 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:15:34.587 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:15:34.623 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:15:34.623 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:15:34.627 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:34.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:34.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:34.658 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:15:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:15:34.689 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:15:34.689 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:15:34.720 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:15:34.720 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:15:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: 7.c scrub ok 2024-09-17T08:15:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:15:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:15:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: pgmap v232: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s; 32 MiB/s, 531 objects/s recovering 2024-09-17T08:15:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:15:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:15:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:34 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:15:34.750 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:15:34.751 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:15:34.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:34.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:15:34.782 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:15:34.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:34.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:34.888 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:15:34.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-17T08:15:34.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:34.977 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:34.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:34.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:35.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:35.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:35.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:35.231 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:35.333 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:35.359 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:35.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:35.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:35.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:35.613 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:35.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:35 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:15:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:35 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:15:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:35 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:15:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:35 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:15:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:35 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1370126860' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:35.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:35.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:35.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:15:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:15:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:15:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:15:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1370126860' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:15:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:15:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:15:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:15:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:35 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1370126860' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:35.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:35.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:35.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:35.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:36.012 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:36.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:36.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:36.146 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:36.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:36.248 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:36.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:36.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:36.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:36.414 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: pgmap v233: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 483 op/s; 2.7 MiB/s, 56 objects/s recovering 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:36.415 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[29565]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T08:15:36.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.416 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:36.416 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:15:36.416 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: pgmap v233: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 483 op/s; 2.7 MiB/s, 56 objects/s recovering 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:36.417 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:36 smithi079 ceph-mon[40649]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T08:15:36.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:36.564 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:36.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:36.666 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:36.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:36.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:15:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:15:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:15:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: pgmap v233: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 483 op/s; 2.7 MiB/s, 56 objects/s recovering 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:36 smithi173 ceph-mon[33187]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T08:15:36.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:36.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:36.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:36.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:36.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:36.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:36.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:37.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:37.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:37.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:37.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:37.199 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:37.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:37.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:37.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.339 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:37.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:37.403 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:15:37.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:37.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:37.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:37.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:37.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1363906949' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:15:37.613 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1363906949' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.614 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:37 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.713 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:15:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:15:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:15:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:15:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1363906949' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:37 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:15:37.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:37.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:37.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:37.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:37.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:37.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:37.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:38.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:38.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:38.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.165 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:38.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:38.304 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:38.404 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:15:38.404 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:238011 7.d 7:b1348f13:::benchmark_data_smithi079_83551_object110611:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:38.385375+0000", 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.018021193000000001, 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018047451999999999, 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238011 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385375+0000", 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385375+0000", 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385373+0000", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385380+0000", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1500000000000002e-06 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385396+0000", 2024-09-17T08:15:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6036999999999999e-05 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385404+0000", 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9480000000000008e-06 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385485+0000", 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0333000000000005e-05 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385600+0000", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011538499999999999 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385730+0000", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013007700000000001 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.389679+0000", 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039484919999999996 2024-09-17T08:15:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.398437+0000", 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0087581759999999995 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:238012 7.d 7:bbc19fa6:::benchmark_data_smithi079_83551_object110612:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:38.385420+0000", 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.017976603000000001, 2024-09-17T08:15:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018096174, 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238012 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385420+0000", 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385420+0000", 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385419+0000", 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385422+0000", 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5790000000000002e-06 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385428+0000", 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9869999999999996e-06 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385432+0000", 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.42e-06 2024-09-17T08:15:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.389791+0000", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043587560000000001 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.390061+0000", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027003500000000001 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.390174+0000", 2024-09-17T08:15:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112834 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.398277+0000", 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081033700000000004 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.398453+0000", 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575699999999999 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:238013 7.d 7:b7f23fd6:::benchmark_data_smithi079_83551_object110613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:38.385436+0000", 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.017960002999999999, 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018166521000000001, 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238013 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385436+0000", 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385436+0000", 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385436+0000", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385438+0000", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.838e-06 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385443+0000", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9980000000000003e-06 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.385447+0000", 2024-09-17T08:15:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0840000000000002e-06 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.393140+0000", 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076936080000000002 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.393243+0000", 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010238699999999999 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.393346+0000", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010345599999999999 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.398406+0000", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597480000000002 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400612+0000", 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022060690000000002 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:238025 7.4 e155/151 7:2d54e589:::benchmark_data_smithi079_83551_object110625:head v 155'14814, mlcod=155'14811)", 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:38.400248+0000", 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0031480100000000001, 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034385879999999998, 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400248+0000", 2024-09-17T08:15:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400248+0000", 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400247+0000", 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400256+0000", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3430000000000002e-06 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400279+0000", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2521e-05 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400286+0000", 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2200000000000003e-06 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400449+0000", 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016287499999999999 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.400471+0000", 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2067999999999999e-05 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:238029 7.0 7:08321534:::benchmark_data_smithi079_83551_object110629:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:38.401286+0000", 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0021099410000000002, 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024599589999999998, 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238029 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401286+0000", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401286+0000", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401285+0000", 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401290+0000", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2279999999999998e-06 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401301+0000", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1338999999999999e-05 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401308+0000", 2024-09-17T08:15:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0670000000000003e-06 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.401908+0000", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00059924100000000003 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.402021+0000", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113124 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:38.402871+0000", 2024-09-17T08:15:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084992200000000003 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 5 2024-09-17T08:15:38.429 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:15:38.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:38.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:38.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:38.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:38.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:38.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:38.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.723 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: pgmap v235: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 507 op/s; 2.3 MiB/s, 49 objects/s recovering 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:15:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T08:15:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:38 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:38.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:38.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:38.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:15:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:15:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: pgmap v235: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 507 op/s; 2.3 MiB/s, 49 objects/s recovering 2024-09-17T08:15:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: pgmap v235: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 1.4 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 507 op/s; 2.3 MiB/s, 49 objects/s recovering 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:15:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T08:15:38.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:38.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:38 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:38.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:38.984 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:38.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:38.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:39.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:39.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:39.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:39.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:39.247 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:39.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:39.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:39.377 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:39.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:39.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:39.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:39.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:39.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:39.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:39.693 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:39 smithi079 ceph-mon[40649]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T08:15:39.693 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:39 smithi079 ceph-mon[40649]: pgmap v238: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:39.694 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:39 smithi079 ceph-mon[29565]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T08:15:39.694 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:39 smithi079 ceph-mon[29565]: pgmap v238: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:39 smithi173 ceph-mon[33187]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T08:15:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:39 smithi173 ceph-mon[33187]: pgmap v238: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:39.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:39.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:39.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:39.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:39.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:39.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:39.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:40.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:40.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:15:40.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.064 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:40.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:40.079 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:40.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:40.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:15:40.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:40.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:40.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:40.364 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:40.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:40.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:40.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:40.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:15:40.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:15:40.583 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:40 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:15:40.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:40.617 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:40.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:40.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:15:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:40 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:15:40.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:40.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:40.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:40.883 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:40.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:40.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:40.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:41.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:41.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:41.042 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:41.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:41.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:41.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:41.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:41.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:41.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:41.419 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:41.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:41.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:41.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:15:41.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[29565]: pgmap v239: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:15:41.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:41 smithi079 ceph-mon[40649]: pgmap v239: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:41.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:41.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:41.698 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:41.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:41 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:15:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:41 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:15:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:41 smithi173 ceph-mon[33187]: pgmap v239: 148 pgs: 148 active+clean; 1.1 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 938 op/s 2024-09-17T08:15:41.800 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:41.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:41.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:41.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:41.970 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:41.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:41.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:42.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:42.114 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:42.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:42.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:42.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:42.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:42.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:42.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:42.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:42.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:42.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:42.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:42 smithi079 ceph-mon[29565]: 2.d deep-scrub starts 2024-09-17T08:15:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:42 smithi079 ceph-mon[29565]: 2.d deep-scrub ok 2024-09-17T08:15:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:42 smithi079 ceph-mon[40649]: 2.d deep-scrub starts 2024-09-17T08:15:42.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:42 smithi079 ceph-mon[40649]: 2.d deep-scrub ok 2024-09-17T08:15:42.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:42.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:42 smithi173 ceph-mon[33187]: 2.d deep-scrub starts 2024-09-17T08:15:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:42 smithi173 ceph-mon[33187]: 2.d deep-scrub ok 2024-09-17T08:15:42.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:42.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:42.786 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:42.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:42.888 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:42.916 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:42.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:42.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:43.018 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:43.043 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:43.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:43.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:43.171 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:43.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:43.273 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:43.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:43.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:43.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:43.537 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:43.562 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:43.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:43.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:43.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:43.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:43 smithi173 ceph-mon[33187]: pgmap v240: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:15:43.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:43.827 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:43.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:15:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:15:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:15:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:15:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[40649]: pgmap v240: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:15:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:43 smithi079 ceph-mon[29565]: pgmap v240: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:15:43.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:43.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:43.962 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:43.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:44.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:44.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:44.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:44.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:44.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:44.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:44.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:44.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:44.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:44.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:44.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:44.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:44.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:44.614 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:44.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:44.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:44 smithi173 ceph-mon[33187]: 6.a deep-scrub starts 2024-09-17T08:15:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:44 smithi173 ceph-mon[33187]: 6.a deep-scrub ok 2024-09-17T08:15:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:44 smithi173 ceph-mon[33187]: 2.1e deep-scrub starts 2024-09-17T08:15:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:44 smithi173 ceph-mon[33187]: 2.1e deep-scrub ok 2024-09-17T08:15:44.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.752 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:44.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:44.783 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[29565]: 6.a deep-scrub starts 2024-09-17T08:15:44.783 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[29565]: 6.a deep-scrub ok 2024-09-17T08:15:44.783 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[29565]: 2.1e deep-scrub starts 2024-09-17T08:15:44.783 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[29565]: 2.1e deep-scrub ok 2024-09-17T08:15:44.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[40649]: 6.a deep-scrub starts 2024-09-17T08:15:44.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[40649]: 6.a deep-scrub ok 2024-09-17T08:15:44.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[40649]: 2.1e deep-scrub starts 2024-09-17T08:15:44.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:44 smithi079 ceph-mon[40649]: 2.1e deep-scrub ok 2024-09-17T08:15:44.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:44.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:44.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:44.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:44.996 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:45.021 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:45.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:45.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:45.146 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:45.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:45.248 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:45.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:45.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:45.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:45.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:45.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:45.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:45.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:45.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:45.617 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T08:15:45.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:45.708 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:45.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 4.2 deep-scrub starts 2024-09-17T08:15:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 4.2 deep-scrub ok 2024-09-17T08:15:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 4.e deep-scrub starts 2024-09-17T08:15:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 4.e deep-scrub ok 2024-09-17T08:15:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:15:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:15:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:15:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:45 smithi173 ceph-mon[33187]: pgmap v241: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:15:45.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:45.847 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:45.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 4.2 deep-scrub starts 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 4.2 deep-scrub ok 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 4.e deep-scrub starts 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 4.e deep-scrub ok 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:15:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[29565]: pgmap v241: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 4.2 deep-scrub starts 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 4.2 deep-scrub ok 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 4.e deep-scrub starts 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 4.e deep-scrub ok 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:15:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:45 smithi079 ceph-mon[40649]: pgmap v241: 148 pgs: 148 active+clean; 747 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:15:45.949 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:45.973 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:45.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:45.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:46.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:46.101 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:46.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:46.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:46.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.241 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:46.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:46.342 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:46.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:46.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:46.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:46.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:46.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:46.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:46.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:46 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:15:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:46 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:15:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:15:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:46 smithi173 ceph-mon[33187]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T08:15:46.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:46.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:46.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:15:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:15:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:15:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[29565]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T08:15:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:15:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:15:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/682087190' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:15:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:46 smithi079 ceph-mon[40649]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T08:15:46.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:46.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:46.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:46.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:47.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:47.061 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:47.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:47.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:47.199 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:47.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:47.205 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:47.301 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:47.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:47.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:47.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:47.463 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:15:47.463 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:15:47.463 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:15:47.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:47.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:47.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:47.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:47.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:47.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-17T08:15:47.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:47.718 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:47 smithi173 ceph-mon[33187]: pgmap v243: 148 pgs: 148 active+clean; 706 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 459 B/s rd, 0 B/s wr, 714 op/s 2024-09-17T08:15:47.730 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:47.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[29565]: pgmap v243: 148 pgs: 148 active+clean; 706 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 459 B/s rd, 0 B/s wr, 714 op/s 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:15:47.737 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:15:47.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:47.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:15:47.738 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:47 smithi079 ceph-mon[40649]: pgmap v243: 148 pgs: 148 active+clean; 706 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 459 B/s rd, 0 B/s wr, 714 op/s 2024-09-17T08:15:47.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:47.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:47.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:47.961 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:47.986 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:47.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:47.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:48.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:48.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:48.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:48.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:48.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.260 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:48.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:48.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:48.395 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:48.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:48.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:48.525 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:48.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:48.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:48.681 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:48.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:48 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:15:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:48 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:15:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:48 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:15:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:48 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:15:48.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:48 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:15:48.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:48.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.827 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:48.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:48.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:48.958 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:48.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:48.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:49.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:49.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:49.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:49.238 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:49.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:49.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:49.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:49.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:49.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:49.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:49.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:49.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:49.639 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:49.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:15:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:49 smithi173 ceph-mon[33187]: pgmap v244: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:15:49.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:49.766 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:49.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:49.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[40649]: pgmap v244: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:15:49.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:15:49.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:15:49.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:49 smithi079 ceph-mon[29565]: pgmap v244: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:15:49.868 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:49.893 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:49.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:49.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:49.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:50.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:50.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:50.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:50.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:50.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:50.298 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:50.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:50.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:50.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:50.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 5.d deep-scrub starts 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: 5.d deep-scrub ok 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:15:50.592 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 5.d deep-scrub starts 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: 5.d deep-scrub ok 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:50.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:50 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:50.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:50.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:50.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:50.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:50.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:50.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:50.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:50.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:50.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:15:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:15:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:15:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:15:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 5.d deep-scrub starts 2024-09-17T08:15:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: 5.d deep-scrub ok 2024-09-17T08:15:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:50 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:51.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:51.095 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:51.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:51.235 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:51.338 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:51.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:51.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:51.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:51.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.501 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:51.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:51.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:51.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:51.779 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.780 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:51.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[40649]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T08:15:51.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:15:51.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:15:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[40649]: pgmap v246: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 669 op/s 2024-09-17T08:15:51.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[29565]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T08:15:51.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:15:51.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:15:51.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:51 smithi079 ceph-mon[29565]: pgmap v246: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 669 op/s 2024-09-17T08:15:51.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:51.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:51.908 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:51.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:51 smithi173 ceph-mon[33187]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T08:15:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:51 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:15:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:51 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:15:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:51 smithi173 ceph-mon[33187]: pgmap v246: 148 pgs: 148 active+clean; 412 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 669 op/s 2024-09-17T08:15:52.009 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:52.036 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:52.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.037 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:52.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:52.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:52.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:52.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:52.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:52.329 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:15:52.329 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:15:52.329 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:15:52.329 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:15:52.329 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:15:52.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:52.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:52.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:52.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:52.590 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:52.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:15:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:15:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[29565]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[40649]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:15:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:52 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:15:52.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.734 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:52.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:52.814 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:52.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:52.871 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:15:52.871 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:15:52.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:52.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:52.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:52.908 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:15:52.908 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:15:52.941 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:15:52.942 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:15:52.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:15:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:15:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:15:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-mon[33187]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T08:15:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:15:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:52 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:15:52.986 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:15:52.986 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:15:52.994 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:53.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.017 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:53.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:53.019 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:15:53.019 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:15:53.049 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:15:53.050 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:15:53.080 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:15:53.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:15:53.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:15:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:15:53.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:53.181 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:53.246 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:15:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:15:53.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:53.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:53.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:53.467 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:53.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:53.593 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:53.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:53.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:53.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:53.822 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Removed 127397 objects 2024-09-17T08:15:53.823 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Clean up completed and total clean up time :125.721 2024-09-17T08:15:53.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:53.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.854 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:15:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:15:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:15:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:15:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.b deep-scrub starts 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: 5.b deep-scrub ok 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/984980501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[29565]: pgmap v248: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.b deep-scrub starts 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: 5.b deep-scrub ok 2024-09-17T08:15:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:53.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/984980501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:53.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:53 smithi079 ceph-mon[40649]: pgmap v248: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:15:53.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:53.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:15:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:15:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:15:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.b deep-scrub starts 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: 5.b deep-scrub ok 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/984980501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:15:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:53 smithi173 ceph-mon[33187]: pgmap v248: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-17T08:15:53.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:53.977 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:53.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:54.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:54.101 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:54.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:54.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:54.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:54.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.349 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:54.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:54.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:54.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:54.595 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:54.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:54.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:54.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:54.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:54.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.844 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:54.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:15:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:15:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:15:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:15:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 2.2 deep-scrub starts 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[40649]: 2.2 deep-scrub ok 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 2.2 deep-scrub starts 2024-09-17T08:15:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:54 smithi079 ceph-mon[29565]: 2.2 deep-scrub ok 2024-09-17T08:15:54.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:54.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:54.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:54.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:15:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:15:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:15:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:15:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 2.2 deep-scrub starts 2024-09-17T08:15:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:54 smithi173 ceph-mon[33187]: 2.2 deep-scrub ok 2024-09-17T08:15:55.067 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:55.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:55.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:55.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:55.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:55.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:55.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:55.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:55.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:55.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:55.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:55.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:55.534 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:15:55.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:55.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:55.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:55.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: 7.1 scrub starts 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: 7.1 scrub ok 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/742208803' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[40649]: pgmap v249: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: 7.1 scrub starts 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: 7.1 scrub ok 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/742208803' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:55 smithi079 ceph-mon[29565]: pgmap v249: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:15:55.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.716 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:55.716 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:55.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:55.839 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:55.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:55.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:55.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:55.962 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:55.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: 7.1 scrub starts 2024-09-17T08:15:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: 7.1 scrub ok 2024-09-17T08:15:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:15:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:15:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/742208803' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:15:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:55 smithi173 ceph-mon[33187]: pgmap v249: 148 pgs: 148 active+clean; 79 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:15:56.063 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:56.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:56.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:56.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.209 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:56.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:56.311 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:56.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.333 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:56.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:56.455 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:56.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:15:56.557 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:56.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:56.680 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:56.701 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:56.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:56.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:15:56.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:15:56.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:56.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:56.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:56.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:15:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:15:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:15:57.050 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:57.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:57.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:57.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:57.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:57.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:57.298 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:57.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:57.367 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:57.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:15:57.444 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:57.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:15:57.542 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T08:15:57.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:57.567 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:57.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:57.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:57.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:57.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:57.779 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-17T08:15:57.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-17T08:15:57.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:57.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:15:57.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:15:57.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[40649]: pgmap v251: 148 pgs: 148 active+clean; 66 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 B/s wr, 812 op/s 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3083461077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[29565]: pgmap v251: 148 pgs: 148 active+clean; 66 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 B/s wr, 812 op/s 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3083461077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:57.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:57.814 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:57.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:57.881 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:15:57.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:57.936 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:57.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:57.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:57 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:15:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:57 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:15:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:57 smithi173 ceph-mon[33187]: pgmap v251: 148 pgs: 148 active+clean; 66 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 B/s wr, 812 op/s 2024-09-17T08:15:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3083461077' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:15:58.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:58.174 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:58.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:58.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:58.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:58.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.300 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:58.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:58.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:58.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:58.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:58.547 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:58.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.548 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:58.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:15:58.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:58.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:15:58.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:15:58.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:58.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:15:58.816 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:58 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:15:58.816 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:58 smithi079 ceph-mon[40649]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T08:15:58.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:58 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:15:58.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:58 smithi079 ceph-mon[29565]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T08:15:58.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-17T08:15:58.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:58.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:58.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:58.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:58 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:15:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:58 smithi173 ceph-mon[33187]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T08:15:59.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:15:59.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:15:59.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:15:59.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:15:59.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:15:59.278 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:59.303 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:59.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:59.428 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.429 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:59.530 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:15:59.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:15:59.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:59.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:59.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.677 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:15:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:15:59 smithi173 ceph-mon[33187]: pgmap v253: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 844 op/s 2024-09-17T08:15:59.778 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:15:59.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:15:59.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:15:59.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:15:59 smithi079 ceph-mon[29565]: pgmap v253: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 844 op/s 2024-09-17T08:15:59.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:15:59 smithi079 ceph-mon[40649]: pgmap v253: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 844 op/s 2024-09-17T08:15:59.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:15:59.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:15:59.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:15:59.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:00.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:00.049 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:00.151 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:00.174 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:00.275 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:00.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:00.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:00.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:00.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:00.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:00.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:00.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:00.674 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:00.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:16:00.679 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:16:00.685 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:00.685 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:00.698 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:16:00.698 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:16:00.716 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:16:00.717 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:16:00.732 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:16:00.733 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:16:00.749 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:16:00.749 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:16:00.765 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:16:00.765 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:16:00.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:00.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:16:00.781 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:16:00.797 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:16:00.797 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:16:00.799 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:00.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:00.924 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:00.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:00.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1549175807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:01.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:01.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:01.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1549175807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:01.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1549175807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:01.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:01.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:01.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:01.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:01.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:01.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:01.321 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:16:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:16:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.14156.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 495.57367551499999, 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.76600744499999995, 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.14156", 2024-09-17T08:16:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3019822778", 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746538+0000", 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746537+0000", 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746543+0000", 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0100000000000001e-06 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746551+0000", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9650000000000008e-06 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746559+0000", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6389999999999998e-06 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746611+0000", 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1743999999999998e-05 2024-09-17T08:16:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:45.746621+0000", 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5219999999999996e-06 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511508+0000", 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.76488780899999997 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511664+0000", 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000155754 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.511784+0000", 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120276 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512293+0000", 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050823700000000003 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512443+0000", 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015041300000000001 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512486+0000", 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2548e-05 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512525+0000", 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9128000000000002e-05 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:07:46.512545+0000", 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-05 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:30840 7.b 7:dda5b3aa:::benchmark_data_smithi079_83551_object30839:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:16:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.280921933, 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14325889999999999, 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30840 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039291+0000", 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039290+0000", 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039314+0000", 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3815000000000002e-05 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039327+0000", 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3742e-05 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039335+0000", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2930000000000003e-06 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039396+0000", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1291000000000007e-05 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039499+0000", 2024-09-17T08:16:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102843 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039584+0000", 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5100999999999997e-05 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040576+0000", 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099186099999999991 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040910+0000", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033465699999999998 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182472+0000", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14156137299999999 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182514+0000", 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2103e-05 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.182550+0000", 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6454e-05 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123272 7.b 7:d9e16f4d:::benchmark_data_smithi079_83551_object123271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.307886+0000", 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.012327691, 2024-09-17T08:16:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103549609, 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123272 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307886+0000", 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307886+0000", 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307884+0000", 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307910+0000", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5613999999999999e-05 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307921+0000", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1462e-05 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307928+0000", 2024-09-17T08:16:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7259999999999996e-06 2024-09-17T08:16:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308215+0000", 2024-09-17T08:16:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028657900000000002 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308315+0000", 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100759 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308487+0000", 2024-09-17T08:16:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017156199999999999 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.314161+0000", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056736649999999996 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.319301+0000", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051406680000000001 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.319333+0000", 2024-09-17T08:16:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1466e-05 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.319396+0000", 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3325999999999996e-05 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411435+0000", 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.092039129999999997 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123274 7.b 7:d7f2cba9:::benchmark_data_smithi079_83551_object123273:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.307942+0000", 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.012270895, 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103714763, 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123274 2024-09-17T08:16:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307942+0000", 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307942+0000", 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307942+0000", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307964+0000", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.228e-05 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307971+0000", 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.489e-06 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307976+0000", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9420000000000003e-06 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311295+0000", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003319537 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311410+0000", 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115151 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311557+0000", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014661399999999999 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.319272+0000", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077146319999999999 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411454+0000", 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.092182511999999994 2024-09-17T08:16:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411528+0000", 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3465000000000002e-05 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411619+0000", 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1237999999999996e-05 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411657+0000", 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8362999999999998e-05 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123275 7.d 7:bf60358e:::benchmark_data_smithi079_83551_object123274:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.307980+0000", 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.01223321699999, 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10790050299999999, 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123275 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307980+0000", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307980+0000", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.307980+0000", 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308001+0000", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1539000000000002e-05 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308007+0000", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9959999999999999e-06 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308011+0000", 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2860000000000001e-06 2024-09-17T08:16:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308072+0000", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0760000000000001e-05 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308206+0000", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000133962 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308337+0000", 2024-09-17T08:16:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013129099999999999 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311427+0000", 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030897810000000002 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311577+0000", 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014997599999999999 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.415798+0000", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10422039900000001 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.415843+0000", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5612999999999998e-05 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.415881+0000", 2024-09-17T08:16:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7299999999999999e-05 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123277 7.b 7:dfa22b5e:::benchmark_data_smithi079_83551_object123276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.308023+0000", 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.012190199, 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103776749, 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123277 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308023+0000", 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308023+0000", 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308023+0000", 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308043+0000", 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0645e-05 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308050+0000", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2840000000000003e-06 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.308054+0000", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7960000000000003e-06 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.314188+0000", 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061339250000000001 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.314299+0000", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011035900000000001 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.314422+0000", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012284899999999999 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.319326+0000", 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049047020000000004 2024-09-17T08:16:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411460+0000", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.092133487 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411698+0000", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023834400000000001 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411764+0000", 2024-09-17T08:16:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6210999999999993e-05 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411800+0000", 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5512999999999997e-05 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123282 7.b 7:d0314f35:::benchmark_data_smithi079_83551_object123281:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.311141+0000", 2024-09-17T08:16:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.00907181599999, 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10081804599999999, 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123282 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311141+0000", 2024-09-17T08:16:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311141+0000", 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311140+0000", 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311195+0000", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4601999999999997e-05 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311206+0000", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.131e-05 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.311212+0000", 2024-09-17T08:16:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5139999999999998e-06 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.316685+0000", 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054723979999999998 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.316813+0000", 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012858600000000001 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.317005+0000", 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019149799999999999 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411667+0000", 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.094661625999999999 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411833+0000", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016622100000000001 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411864+0000", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.099e-05 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411931+0000", 2024-09-17T08:16:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7529999999999999e-05 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.411959+0000", 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8212e-05 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:123283 7.b 7:db98a2a6:::benchmark_data_smithi079_83551_object123282:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:13:44.312494+0000", 2024-09-17T08:16:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.00771892700001, 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.111667208, 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 123283 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312494+0000", 2024-09-17T08:16:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312494+0000", 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312493+0000", 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312539+0000", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6332000000000003e-05 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312551+0000", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1474999999999999e-05 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.312557+0000", 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-17T08:16:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.414716+0000", 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.102158437 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.418210+0000", 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034942300000000001 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.418345+0000", 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134583 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.421225+0000", 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002880142 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.421662+0000", 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00043763199999999997 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.424019+0000", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023565639999999998 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.424123+0000", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010359 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:13:44.424162+0000", 2024-09-17T08:16:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8921000000000002e-05 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:11916 7.0 e133/91 7:05e2e8f4:::benchmark_data_smithi079_83551_object116547:head v 133'11885, mlcod=133'11862)", 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:14:19.804686+0000", 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 101.51552683600001, 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.104677009, 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804686+0000", 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804686+0000", 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804686+0000", 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804690+0000", 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804695+0000", 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0329999999999999e-06 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.804699+0000", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6619999999999998e-06 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.854031+0000", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049332449 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.854049+0000", 2024-09-17T08:16:01.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7448e-05 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.909301+0000", 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055252031 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.909363+0000", 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2634999999999999e-05 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:11917 7.0 e133/91 7:05e3bf8c:::benchmark_data_smithi079_83551_object80510:head v 133'11886, mlcod=133'11862)", 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:14:19.808426+0000", 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 101.511787075, 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.100979531, 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808426+0000", 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808426+0000", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808425+0000", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808433+0000", 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5599999999999996e-06 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808443+0000", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0475000000000001e-05 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.808450+0000", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7309999999999996e-06 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.856878+0000", 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048428064999999999 2024-09-17T08:16:01.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.856898+0000", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885e-05 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.909366+0000", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052468582999999999 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:19.909406+0000", 2024-09-17T08:16:01.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9252999999999998e-05 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:185624 7.b 7:d22b262b:::benchmark_data_smithi079_83551_object58224:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:14:44.541403+0000", 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.778809851999995, 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107741009, 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185624 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541403+0000", 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541403+0000", 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541402+0000", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541407+0000", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7219999999999999e-06 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541415+0000", 2024-09-17T08:16:01.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5099999999999998e-06 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541421+0000", 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2700000000000004e-06 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.631360+0000", 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.089939211000000005 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.631469+0000", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108837 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.631589+0000", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119913 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.637384+0000", 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057951000000000001 2024-09-17T08:16:01.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.637421+0000", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7515999999999998e-05 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649055+0000", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011633426000000001 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649104+0000", 2024-09-17T08:16:01.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9842999999999997e-05 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649144+0000", 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9978999999999998e-05 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:185626 7.b 7:d02f898e:::benchmark_data_smithi079_83551_object58226:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:14:44.541428+0000", 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.778785400000004, 2024-09-17T08:16:01.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10779343500000001, 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185626 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541428+0000", 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541428+0000", 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541427+0000", 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541429+0000", 2024-09-17T08:16:01.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9989999999999998e-06 2024-09-17T08:16:01.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541435+0000", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2800000000000003e-06 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.541439+0000", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1949999999999996e-06 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.634481+0000", 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.093041946 2024-09-17T08:16:01.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.634598+0000", 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011731 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.634723+0000", 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012511600000000001 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.645133+0000", 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010409379999999999 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.645162+0000", 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9604e-05 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649152+0000", 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003989406 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649188+0000", 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6625999999999998e-05 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:14:44.649221+0000", 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2937e-05 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218570 7.b 7:d2ff73cd:::benchmark_data_smithi079_83551_object91170:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:16.239471+0000", 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.080742035999997, 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1116008740000001, 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218570 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239471+0000", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239471+0000", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239469+0000", 2024-09-17T08:16:01.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239477+0000", 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0870000000000001e-06 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239490+0000", 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3441999999999999e-05 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239501+0000", 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1117999999999999e-05 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239985+0000", 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048351699999999999 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.239996+0000", 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0966e-05 2024-09-17T08:16:01.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.347364+0000", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.107368329 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.347509+0000", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145019 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5,6", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.347654+0000", 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014545000000000001 2024-09-17T08:16:01.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.350402+0000", 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027472180000000001 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.350529+0000", 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012723899999999999 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.350539+0000", 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302999999999999e-05 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.350762+0000", 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000222585 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.350851+0000", 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9353999999999999e-05 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.351072+0000", 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022101600000000001 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:243145 7.d 7:b6bfbd3c:::benchmark_data_smithi079_83551_object115745:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:42.924223+0000", 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.395990427000001, 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.099840737999999998, 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 243145 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924223+0000", 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924223+0000", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924221+0000", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924227+0000", 2024-09-17T08:16:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8089999999999998e-06 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924240+0000", 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2527999999999999e-05 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924247+0000", 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4830000000000002e-06 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924358+0000", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111121 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924464+0000", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105782 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.924572+0000", 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107645 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.927080+0000", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025080459999999999 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.927151+0000", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1569999999999994e-05 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.023902+0000", 2024-09-17T08:16:01.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.096750742000000001 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.024026+0000", 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012347899999999999 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.024064+0000", 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7973999999999999e-05 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:252508 7.0 7:0d42f66e:::benchmark_data_smithi079_83551_object125108:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.630929+0000", 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.6892844680000003, 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.125738508, 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 252508 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630929+0000", 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630929+0000", 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630927+0000", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630934+0000", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.55e-06 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630947+0000", 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2707999999999999e-05 2024-09-17T08:16:01.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.630954+0000", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4780000000000002e-06 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.640283+0000", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093285720000000003 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.640401+0000", 2024-09-17T08:16:01.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118658 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.640880+0000", 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047892900000000001 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644466+0000", 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035855090000000002 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644682+0000", 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021609999999999999 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756547+0000", 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.111865248 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756596+0000", 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9224999999999997e-05 2024-09-17T08:16:01.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756667+0000", 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0988999999999994e-05 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:252514 7.4 e159/151 7:28255f1e:::benchmark_data_smithi079_83551_object125114:head v 159'15702, mlcod=159'15700)", 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.641046+0000", 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.679166833, 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114762173, 2024-09-17T08:16:01.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641046+0000", 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641046+0000", 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641046+0000", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641051+0000", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6930000000000004e-06 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641058+0000", 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.939e-06 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.641063+0000", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5680000000000001e-06 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.642922+0000", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018592319999999999 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.642941+0000", 2024-09-17T08:16:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9128999999999999e-05 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.755766+0000", 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.112824518 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.427 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.755809+0000", 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2805000000000001e-05 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:252515 7.4 e159/151 7:2eebab8f:::benchmark_data_smithi079_83551_object125115:head v 159'15703, mlcod=159'15700)", 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.644200+0000", 2024-09-17T08:16:01.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.6760133209999992, 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.111631543, 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644200+0000", 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644200+0000", 2024-09-17T08:16:01.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644199+0000", 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644206+0000", 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.294e-06 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644218+0000", 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1352000000000001e-05 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644225+0000", 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9190000000000002e-06 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.742799+0000", 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.098573895999999994 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.742820+0000", 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1311999999999999e-05 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.755812+0000", 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012992034 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.755831+0000", 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9511e-05 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:252523 7.0 7:059cffef:::benchmark_data_smithi079_83551_object125123:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.644922+0000", 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.6752909200000001, 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.117208249, 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 252523 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644922+0000", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644922+0000", 2024-09-17T08:16:01.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644921+0000", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644927+0000", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5330000000000002e-06 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644964+0000", 2024-09-17T08:16:01.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7663000000000002e-05 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.644972+0000", 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2530000000000004e-06 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645017+0000", 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4134999999999997e-05 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645197+0000", 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018024600000000001 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645441+0000", 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024454899999999999 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756515+0000", 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11107325799999999 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756797+0000", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028207999999999998 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.761889+0000", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050924849999999999 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.762090+0000", 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020101299999999999 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.762131+0000", 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0368999999999999e-05 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:252525 7.0 7:0e1d2072:::benchmark_data_smithi079_83551_object125125:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.645219+0000", 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.6749947580000004, 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119844956, 2024-09-17T08:16:01.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 252525 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645219+0000", 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645219+0000", 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645218+0000", 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645223+0000", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645234+0000", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0908e-05 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.645240+0000", 2024-09-17T08:16:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4609999999999996e-06 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.753425+0000", 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10818483800000001 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.753589+0000", 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016378200000000001 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.753749+0000", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016038099999999999 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.760304+0000", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065545550000000001 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.762139+0000", 2024-09-17T08:16:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018359380000000001 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.764695+0000", 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025559369999999999 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.764927+0000", 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000231766 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.765063+0000", 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013628499999999999 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:252523 7.0 e159/152 ondisk, result = 0)", 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:51.647674+0000", 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.6725395879999994, 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108843656, 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:16:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.647674+0000", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.647674+0000", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.647673+0000", 2024-09-17T08:16:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.647678+0000", 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7729999999999996e-06 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.741220+0000", 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.093541799999999994 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.741235+0000", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4605e-05 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756500+0000", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015264801 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756508+0000", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0390000000000005e-06 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:51.756517+0000", 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7229999999999999e-06 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:01.450 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:16:01.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:01.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:01.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:01.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:01.598 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:01.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:01.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:01.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.721 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:01.822 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:01.848 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:01.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:01.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:01.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:01.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:01.972 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:02.073 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:02.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:02.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:02.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:02.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:02.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:02.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:02.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.345 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:02.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:02.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:02.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:02.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:02.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[40649]: pgmap v254: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:02.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:02.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:02.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[29565]: pgmap v254: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:02.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:02.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:02.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:02 smithi173 ceph-mon[33187]: pgmap v254: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:02.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:02.539 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:02.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:02.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:02.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:02.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:02.716 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:02.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:02.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:02.844 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:02.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:02.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:02.971 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:02.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:02.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:02.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:03.073 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:03.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:03.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:03.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:03.226 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:03.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:16:03.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:03.349 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:03.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.350 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:03.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:03.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:03.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:03.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:03.705 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:03.729 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:03.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:03.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.852 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:03.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:03.953 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:03.978 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:03.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:03.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:04.080 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:04.106 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:04.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:04.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:04.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:04.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:04.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:04.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:04.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:04.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:04.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:04.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:04.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:04.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:04.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:04.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:04 smithi079 ceph-mon[40649]: pgmap v255: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:04 smithi079 ceph-mon[29565]: pgmap v255: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:04.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:04 smithi173 ceph-mon[33187]: pgmap v255: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 158 op/s 2024-09-17T08:16:04.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:04.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:04.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:04.861 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:04.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:04.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:04.983 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:04.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:04.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:05.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:05.106 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:05.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:05.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.230 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:05.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:05.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:05.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:05.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:05.480 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.481 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:05.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:05.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:05.624 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:05.624 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:05.624 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[40649]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T08:16:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:05.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:05.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:05 smithi079 ceph-mon[29565]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T08:16:05.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:05 smithi173 ceph-mon[33187]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T08:16:05.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:05.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:16:05.829 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:05.855 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:05.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:05.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:05.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:05.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:05.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:06.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:06.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:06.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:06.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:06.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:06.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.354 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.354 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:06.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:06.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:06.599 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:06.653 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:06 smithi079 ceph-mon[40649]: pgmap v257: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 132 op/s 2024-09-17T08:16:06.653 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:06 smithi079 ceph-mon[29565]: pgmap v257: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 132 op/s 2024-09-17T08:16:06.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:06.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:06 smithi173 ceph-mon[33187]: pgmap v257: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 132 op/s 2024-09-17T08:16:06.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:06.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:06.851 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:06.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:06.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:06.963 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:06.975 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:06.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:06.976 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:07.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:07.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:07.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:07.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:07.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:07.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:07.327 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:07.353 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:07.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:07.399 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_0' removed 2024-09-17T08:16:07.399 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:16:07.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:07.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:07.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:07.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:07.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:07.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:07.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:07.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:16:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:07.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:16:07.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:16:07.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:07.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:07.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:07.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:07.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:07.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:07.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:07.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:07.979 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:08.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:08.100 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:08.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:08.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:08.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:08.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:08.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:08.388 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-17T08:16:08.389 INFO:tasks.radosbench:Beginning radosbench... 2024-09-17T08:16:08.389 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-17T08:16:08.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_1 16 2024-09-17T08:16:08.401 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T08:16:08.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:08.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.480 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:08.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:08.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: pgmap v258: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 523 B/s rd, 0 B/s wr, 79 op/s 2024-09-17T08:16:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: pgmap v258: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 523 B/s rd, 0 B/s wr, 79 op/s 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T08:16:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:16:08.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: pgmap v258: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 523 B/s rd, 0 B/s wr, 79 op/s 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4284255328' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3667221148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T08:16:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:16:08.729 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:08.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:08.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:08.852 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:08.954 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:08.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:08.977 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:08.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:09.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:09.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:09.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:09.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:09.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:09.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:09.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:09.347 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:09.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:09.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:09.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.471 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:09.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:09.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:09.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:09.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:09.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:09.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:09.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:09.719 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:16:09.719 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:09 smithi079 ceph-mon[40649]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T08:16:09.720 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:16:09.720 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:09 smithi079 ceph-mon[29565]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T08:16:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2450372053' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:16:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:09 smithi173 ceph-mon[33187]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T08:16:09.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:09.841 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:09.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:09.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:09.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:09.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:10.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:10.091 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:10.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-17T08:16:10.145 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:10.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.215 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:10.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:10.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:10.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:10.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.463 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:10.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:10.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:10.593 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:10.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:10.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.710 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:10.714 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:10 smithi079 ceph-mon[40649]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:16:10.714 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:10 smithi079 ceph-mon[29565]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:16:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:10 smithi173 ceph-mon[33187]: pgmap v261: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:16:10.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:10.833 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:10.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:10.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:10.961 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:10.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:10.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:11.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:11.088 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:11.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:11.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:16:11.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:11.212 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:11.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:11.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:11.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:11.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:11.458 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:11.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:11.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:11.582 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:11.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.583 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:11.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:11.705 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:11.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:11.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:11.833 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:11.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:11.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:11.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:11.961 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:11.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:12.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:12.085 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:12.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:12.186 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:12.213 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:12.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:12.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:12.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:12.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:12.359 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:12 smithi079 ceph-mon[29565]: pgmap v262: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 589 B/s rd, 0 op/s 2024-09-17T08:16:12.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:12 smithi079 ceph-mon[40649]: pgmap v262: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 589 B/s rd, 0 op/s 2024-09-17T08:16:12.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:12.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.466 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:12.550 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:12 smithi173 ceph-mon[33187]: pgmap v262: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 589 B/s rd, 0 op/s 2024-09-17T08:16:12.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:12.592 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:12.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:12.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:12.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:12.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:12.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:12.844 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:12.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:12.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:12.970 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:12.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:12.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:12.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:13.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:13.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:13.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:13.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:13.224 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:13.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:13.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:13.352 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:13.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:13.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:13.480 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:13.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.481 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:13.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:13.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:13.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:13.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:13.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:13.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:13.825 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:16:13.828 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:16:13.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:13.838 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:16:13.839 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:16:13.854 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:16:13.855 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:16:13.860 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:13.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:13.870 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:16:13.870 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:16:13.887 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:16:13.887 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:16:13.904 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:16:13.904 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:16:13.920 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:16:13.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:16:13.937 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:16:13.938 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:16:13.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:13.987 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:13.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:13.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:14.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:14.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:14.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:14.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:14.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:14.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:14.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:14.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:14.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:14.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:14.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:14.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:14.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:14.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[40649]: pgmap v263: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:14.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1740974387' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:14.493 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:14.493 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[29565]: pgmap v263: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:14.493 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:14 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1740974387' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:14.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:14.614 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:14.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:14 smithi173 ceph-mon[33187]: pgmap v263: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:14 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1740974387' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:14.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:14.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:14.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:14.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:14.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:14.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:14.984 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:14.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:14.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:15.006 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:15.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:15.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.108 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:15.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:15.235 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:15.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:15.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:15.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:15.394 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_1' created 2024-09-17T08:16:15.394 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-17T08:16:15.395 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-17T08:16:15.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:15.485 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:15.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:15.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:15.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-17T08:16:15.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:15.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:15.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:15.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:15.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:15.982 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:15.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:15.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:16.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:16.109 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:16.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:16.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-17T08:16:16.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:16.232 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:16.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:16.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:16.360 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:16.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.485 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:16.485 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:16.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:16.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:16.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:16.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:16 smithi173 ceph-mon[33187]: pgmap v264: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-17T08:16:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:16 smithi173 ceph-mon[33187]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T08:16:16.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:16.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:16.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[40649]: pgmap v264: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:16.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-17T08:16:16.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[40649]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T08:16:16.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[29565]: pgmap v264: 132 pgs: 132 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:16:16.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1565678525' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-17T08:16:16.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:16 smithi079 ceph-mon[29565]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T08:16:16.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:16.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:16.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:16.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:16.988 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:16.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:16.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:17.044 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:16:17.044 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:16:17.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:17.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:17.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:17.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:17.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:17.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:17.243 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:17.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:17.366 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:17.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:17.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:17.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:17.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:17.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:17 smithi079 ceph-mon[40649]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T08:16:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:17.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:17 smithi079 ceph-mon[29565]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T08:16:17.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:17.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.620 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:17.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:17.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:17.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:17 smithi173 ceph-mon[33187]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T08:16:17.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:17.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:17.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:17.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:17.871 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:17.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:17.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:17.995 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:17.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:17.996 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:18.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:18.117 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:16:18.119 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:18.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:18.241 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:18.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:18.363 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:18.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:18.487 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:18.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:18.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:18.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:18.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[29565]: pgmap v267: 148 pgs: 12 unknown, 136 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[29565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[29565]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[40649]: pgmap v267: 148 pgs: 12 unknown, 136 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[40649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:18 smithi079 ceph-mon[40649]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:18 smithi173 ceph-mon[33187]: pgmap v267: 148 pgs: 12 unknown, 136 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:16:18.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:18 smithi173 ceph-mon[33187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:16:18.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:18 smithi173 ceph-mon[33187]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T08:16:18.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:18.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:18.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.865 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:18.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:18.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:18.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:18.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:19.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:19.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:19.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:19.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:19.344 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:19.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:19.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:19.492 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:19.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:19.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:19.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.620 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:19.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:19.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:19.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:19.871 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:19.973 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:19.997 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:19.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:19.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:20.039 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:20.099 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:20.123 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:20.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:20.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:20.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:20.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:20.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:20.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:20.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.373 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:20.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:20.482 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:20 smithi079 ceph-mon[40649]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:20.482 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:20 smithi079 ceph-mon[29565]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:20.497 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:20.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:20.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:20.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:20.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:20.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:20 smithi173 ceph-mon[33187]: pgmap v269: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:20.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:20.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:20.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:20.850 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:20.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:20.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:20.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:21.003 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:21.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:21.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:21.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:21.255 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:21.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:21.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:21.378 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:21.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:21.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:21.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:21.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:16:21.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:21.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.630 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:21.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:21.752 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:21.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:21.876 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:21.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:21.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:21.999 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:21.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:22.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:22.127 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:22.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:22.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:22.251 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:16:22.253 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:22.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:22.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:22.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:22.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:22.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:22.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.505 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:22.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:22.556 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:22 smithi173 ceph-mon[33187]: pgmap v270: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:22.556 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/996944896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.557 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:22 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[40649]: pgmap v270: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/996944896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[29565]: pgmap v270: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/996944896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:22 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:16:22.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.629 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:22.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:22.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:22.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:22.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:22.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:22.879 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:22.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:22.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:22.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:22.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:23.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.002 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:23.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:23.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:23.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:23.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:23.193 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:23.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:23.247 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:23.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:23.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:23.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:23.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.379 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:23.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:23.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:23.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:23.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:23.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:23.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:16:23.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[29565]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T08:16:23.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:23.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:16:23.509 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:23 smithi079 ceph-mon[40649]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T08:16:23.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:23.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:23.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:23 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:16:23.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:16:23.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:23 smithi173 ceph-mon[33187]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T08:16:23.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:23.807 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:23.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:23.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:23.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:23.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:23.936 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:24.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:24.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:24.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:24.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:24.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.186 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:24.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:24.256 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T08:16:24.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:24.308 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:24.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.309 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:24.410 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:24.436 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:24.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:24.538 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:24.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:24.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:24.571 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3644732567' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:16:24.572 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3644732567' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.573 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:24.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:24.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:16:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:16:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: pgmap v272: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:16:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3644732567' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:16:24.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:24.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.816 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:24.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:24.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:24.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:24.939 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:24.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:24.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:16:24.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-17T08:16:25.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:25.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.085 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:25.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:25.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:25.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:25.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:25.263 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-17T08:16:25.310 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:25.337 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:25.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:25.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:25.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:25.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:25.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:16:25.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:16:25.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:16:25.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T08:16:25.562 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1820421578' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1820421578' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:25.590 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:25.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:25.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:25.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:25.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:16:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:16:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:16:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T08:16:25.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1820421578' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:16:25.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:25.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:25.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:25.970 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:25.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:25.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:26.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:26.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:26.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:26.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:26.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:26.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:26.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:26.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:26.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:26.375 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:26.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:26.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:26.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:26.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:16:26.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:16:26.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:16:26.533 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[40649]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[40649]: pgmap v275: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[29565]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T08:16:26.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:26 smithi079 ceph-mon[29565]: pgmap v275: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:26.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:26.595 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:26.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:26.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:26.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:26 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:16:26.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:26 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:16:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:26 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:16:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:26 smithi173 ceph-mon[33187]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T08:16:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:26 smithi173 ceph-mon[33187]: pgmap v275: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:26.724 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:26.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:26.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:26.847 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:26.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:26.949 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:26.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:26.976 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:26.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:27.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:27.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:27.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:27.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:27.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:27.327 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:27.353 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:27.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:27.481 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:16:27.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[40649]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:16:27.583 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:27 smithi079 ceph-mon[29565]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T08:16:27.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:27.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:27.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:27.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:27.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:16:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:27 smithi173 ceph-mon[33187]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T08:16:27.737 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:27.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:27.864 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:27.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:27.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:27.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:27.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:28.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:28.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:28.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:28.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:28.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:28.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:28.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:28.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:28.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:28.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:16:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:16:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[29565]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:16:28.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:28 smithi079 ceph-mon[40649]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-17T08:16:28.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:28.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:28 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:16:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:28 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:16:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:28 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:16:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:28 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:16:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:28 smithi173 ceph-mon[33187]: pgmap v277: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-17T08:16:28.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:28.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:28.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:28.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:28.952 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:16:28.952 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:16:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:16:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:16:28.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:28.967 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:16:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:16:28.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:28.985 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:28.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:28.986 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:16:28.986 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:16:28.999 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:16:29.000 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:16:29.016 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:16:29.017 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:16:29.033 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:16:29.033 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:16:29.052 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:16:29.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:16:29.086 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:29.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:29.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:29.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:29.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:29.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:29.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:29.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.360 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:29.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:29.370 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:16:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:16:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:16:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:16:29.370 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:16:29.460 DEBUG:teuthology.orchestra.run.smithi079:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-17T08:16:29.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:29.463 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-17T08:16:29.464 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-17T08:16:29.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.483 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:29.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:29.513 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:hints = 1 2024-09-17T08:16:29.513 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-17T08:16:29.513 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object prefix: benchmark_data_smithi079_115040 2024-09-17T08:16:29.513 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:16:29.513 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 0 0 0 0 0 0 - 0 2024-09-17T08:16:29.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:29.616 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:29.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:16:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:16:29.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1412674748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:29.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:16:29.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:16:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1412674748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:29.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:29 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:16:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:29 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:16:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1412674748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:29.749 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:29.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:29.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:29.890 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:29.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:29.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:29.992 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:30.023 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:30.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:30.125 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:30.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:30.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:30.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:30.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:30.400 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:30.448 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:30.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:30.514 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 1 16 3704 3688 230.285 230.5 0.0106143 0.00432677 2024-09-17T08:16:30.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:30.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:30 smithi079 ceph-mon[40649]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:30.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:30 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:16:30.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:30 smithi079 ceph-mon[29565]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:30.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:30 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:16:30.592 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:30.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:30.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:30.695 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:30 smithi173 ceph-mon[33187]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:16:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:30 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:16:30.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:16:30.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:30.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:30.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:30.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:30.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:30.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:30.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:31.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:31.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:31.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:31.309 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:31.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:31.448 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:31.476 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:16:31.476 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:16:31.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:16:31.477 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:16:31.477 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:16:31.477 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:31 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:16:31.514 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 2 16 6918 6902 215.578 200.875 0.00247753 0.00462679 2024-09-17T08:16:31.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:31.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.588 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.588 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:31.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:31.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:31.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:31.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:31 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:16:31.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:31 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:16:31.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:31 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:16:31.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:31.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.845 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:31.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:31.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:31.989 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:31.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:31.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:32.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:32.124 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:32.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:32.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:32.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:32.316 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T08:16:32.364 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:32.394 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:32.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:32.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:32.514 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 3 15 9830 9815 204.39 182.062 0.00395761 0.00488189 2024-09-17T08:16:32.530 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:32.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:32.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:16:32.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1938358354' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1938358354' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1938358354' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:32 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:16:32.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:32.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:32.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:32.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:32.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:32.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:32.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:32.936 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:32.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:32.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:32.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:33.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:33.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:33.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:16:33.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:33.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:33.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:33.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:33.325 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:33.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:33.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:33.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:33.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 4 16 12898 12882 201.211 191.688 0.00416277 0.00496368 2024-09-17T08:16:33.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-17T08:16:33.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-17T08:16:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]': finished 2024-09-17T08:16:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:16:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:16:33.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[29565]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T08:16:33.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-17T08:16:33.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]': finished 2024-09-17T08:16:33.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:16:33.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:16:33.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:33 smithi079 ceph-mon[40649]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T08:16:33.663 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-17T08:16:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 5]}]': finished 2024-09-17T08:16:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-17T08:16:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:16:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:33 smithi173 ceph-mon[33187]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T08:16:33.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:33.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:33.944 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:33.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:33.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:34.046 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:34.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:34.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:34.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:34.209 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:34.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:34.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:34.333 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:34.334 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:34.334 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:16:34.334 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:16:34.334 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[29565]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T08:16:34.334 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[29565]: pgmap v282: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-17T08:16:34.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.349 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:34.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:34.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:34.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:34.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:34.514 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 5 16 15654 15638 195.417 172.25 0.0191513 0.0051001 2024-09-17T08:16:34.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:34.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:34.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:16:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:16:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[40649]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T08:16:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:34 smithi079 ceph-mon[40649]: pgmap v282: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-17T08:16:34.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:34 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:16:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:34 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:16:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:34 smithi173 ceph-mon[33187]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T08:16:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:34 smithi173 ceph-mon[33187]: pgmap v282: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-17T08:16:34.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.744 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:34.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:34.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:34.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:34.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:34.880 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:34.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:35.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:35.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:35.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:35.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:35.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:35.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:35.313 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:35.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:35.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:35.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:35.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:35.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 6 16 18282 18266 190.221 164.25 0.00964666 0.00525044 2024-09-17T08:16:35.551 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:35.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:35.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.584 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:16:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:16:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:16:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:16:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:16:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:16:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:16:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:35 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:16:35.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:35.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:35.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:35 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:16:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:35 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:16:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:35 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:16:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:35 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:16:35.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:35.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:35.867 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:35.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:35.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:36.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:36.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:36.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:36.142 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:36.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:36.239 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:36.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:36.303 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:36.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:36.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:36.434 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:36.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:36.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 7 16 20934 20918 186.723 165.75 0.00374157 0.00534864 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: pgmap v283: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-17T08:16:36.530 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: pgmap v283: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-17T08:16:36.531 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:36 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:16:36.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:36.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:36.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:36.696 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:36.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:16:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:16:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:16:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:16:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:16:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: pgmap v283: 148 pgs: 2 peering, 146 active+clean; 634 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-17T08:16:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:36 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:16:36.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:36.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:36.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:36.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:36.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:36.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:36.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:37.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:37.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:37.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:37.258 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:37.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.259 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:37.287 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:29220 7.e 7:74d84a1d:::benchmark_data_smithi079_83551_object29219:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.249287+0000", 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.035025819, 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.089173584, 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29220 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249287+0000", 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249287+0000", 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249286+0000", 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249316+0000", 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490999999999999e-05 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249327+0000", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1585e-05 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249334+0000", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1640000000000004e-06 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249450+0000", 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115876 2024-09-17T08:16:37.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.249912+0000", 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000462261 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250045+0000", 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013212200000000001 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251766+0000", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017212480000000001 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338158+0000", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.086391701000000001 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338204+0000", 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6394999999999999e-05 2024-09-17T08:16:37.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338414+0000", 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209698 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338461+0000", 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7346999999999999e-05 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:29215 7.3 e98/91 7:ce5a44d1:::benchmark_data_smithi079_83551_object29214:head v 98'1912, mlcod=98'1909)", 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.250601+0000", 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.03371203099999, 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085999080000000006, 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250601+0000", 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250601+0000", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250601+0000", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250673+0000", 2024-09-17T08:16:37.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2867000000000006e-05 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250685+0000", 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1952e-05 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250691+0000", 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.015e-06 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250951+0000", 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026007100000000001 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.250969+0000", 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7122999999999999e-05 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253536+0000", 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025674209999999999 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336600+0000", 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.083064267999999997 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:29219 7.5 e98/91 7:ac17fa26:::benchmark_data_smithi079_83551_object29218:head v 98'1835, mlcod=98'1832)", 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.251056+0000", 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.033256835, 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085333035000000002, 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.299 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251056+0000", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251056+0000", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251056+0000", 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251213+0000", 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015773100000000001 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251225+0000", 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1260999999999999e-05 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251230+0000", 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.772e-06 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251277+0000", 2024-09-17T08:16:37.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6490999999999999e-05 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251293+0000", 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6050999999999999e-05 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336352+0000", 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085058580999999994 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336389+0000", 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7879000000000001e-05 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:29223 7.e 7:70f1e10e:::benchmark_data_smithi079_83551_object29222:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.251103+0000", 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.03321023000001, 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091330588000000004, 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29223 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251103+0000", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251103+0000", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251101+0000", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251121+0000", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9556e-05 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251137+0000", 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6251000000000001e-05 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251144+0000", 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1470000000000004e-06 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253113+0000", 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019685430000000001 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253235+0000", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122216 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253402+0000", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000167601 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338503+0000", 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085100800000000004 2024-09-17T08:16:37.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338544+0000", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0797999999999998e-05 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.342127+0000", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035827200000000002 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.342171+0000", 2024-09-17T08:16:37.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4419000000000003e-05 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.342434+0000", 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026235400000000002 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:29225 7.8 7:18112b61:::benchmark_data_smithi079_83551_object29224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.251737+0000", 2024-09-17T08:16:37.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.032576154, 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085137365000000007, 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29225 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251737+0000", 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251737+0000", 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251736+0000", 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251757+0000", 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1297999999999998e-05 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251767+0000", 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.825e-06 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251774+0000", 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4439999999999996e-06 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253160+0000", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013868489999999999 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253233+0000", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2835000000000003e-05 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.253312+0000", 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9102000000000003e-05 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336700+0000", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.083387524000000005 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336775+0000", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5354999999999999e-05 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336792+0000", 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6614999999999999e-05 2024-09-17T08:16:37.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336837+0000", 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5336000000000001e-05 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336874+0000", 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7147999999999997e-05 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:29220 7.e e98/91 ondisk, result = 0)", 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.251805+0000", 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.03250774700001, 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.086359293000000004, 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251805+0000", 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251805+0000", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251805+0000", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251808+0000", 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4589999999999998e-06 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251818+0000", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7289999999999995e-06 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.251823+0000", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338143+0000", 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.086319850000000004 2024-09-17T08:16:37.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338150+0000", 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4989999999999997e-06 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338165+0000", 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4319e-05 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.24698.0:29222 7.5 e98/91 7:aad82960:::benchmark_data_smithi079_83551_object29221:head v 98'1836, mlcod=98'1832)", 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.252271+0000", 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.03204208599999, 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084140946999999994, 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252271+0000", 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252271+0000", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252271+0000", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252389+0000", 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118517 2024-09-17T08:16:37.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252401+0000", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2261e-05 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252408+0000", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4509999999999997e-06 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252746+0000", 2024-09-17T08:16:37.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033790699999999998 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252764+0000", 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8117999999999998e-05 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336395+0000", 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.083631269999999994 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.336412+0000", 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6994999999999999e-05 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.24698.0:29220 7.e e98/91 ondisk, result = 0)", 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:29.252563+0000", 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 248.031750018, 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085907998999999999, 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:16:37.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252563+0000", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252563+0000", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252563+0000", 2024-09-17T08:16:37.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252566+0000", 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8569999999999997e-06 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252574+0000", 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4619999999999999e-06 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.252579+0000", 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7890000000000002e-06 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338193+0000", 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085614286999999997 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338199+0000", 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1099999999999999e-06 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:29.338471+0000", 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027206700000000002 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:30839 7.8 7:1cf19e7e:::benchmark_data_smithi079_83551_object30838:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:30.038924+0000", 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.24538963699999, 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13802040400000001, 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30839 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038924+0000", 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038924+0000", 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038922+0000", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038944+0000", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1973000000000001e-05 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038959+0000", 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5489999999999999e-05 2024-09-17T08:16:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.038968+0000", 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2539999999999992e-06 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039009+0000", 2024-09-17T08:16:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1715999999999998e-05 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039128+0000", 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118479 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.039269+0000", 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141519 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040289+0000", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001019245 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.040574+0000", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028505700000000002 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.176859+0000", 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13628554200000001 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.176908+0000", 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8922999999999999e-05 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:30.176944+0000", 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5843999999999998e-05 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:67123 7.e 7:77f58e2f:::benchmark_data_smithi079_83551_object67122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:12:53.543201+0000", 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 223.74111216899999, 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091944039000000005, 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 67123 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543201+0000", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543201+0000", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543200+0000", 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543224+0000", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4287999999999999e-05 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543237+0000", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.314e-05 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543245+0000", 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3819999999999998e-06 2024-09-17T08:16:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543289+0000", 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4539999999999997e-05 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543397+0000", 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108278 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.543523+0000", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012555300000000001 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.545934+0000", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024113160000000002 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.546451+0000", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051661199999999995 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.635067+0000", 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088616120000000007 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.635112+0000", 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5105999999999998e-05 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:12:53.635145+0000", 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2925000000000001e-05 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:212049 7.e 7:7459d8fe:::benchmark_data_smithi079_83551_object84649:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:09.606453+0000", 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.677860365000001, 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10947533299999999, 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 212049 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606453+0000", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606453+0000", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606451+0000", 2024-09-17T08:16:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606458+0000", 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4280000000000001e-06 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606476+0000", 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8241e-05 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606484+0000", 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8869999999999992e-06 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606535+0000", 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1422000000000003e-05 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606639+0000", 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103108 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.606755+0000", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116786 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.612799+0000", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060434850000000003 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.614349+0000", 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015497849999999999 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.715844+0000", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.10149591299999999 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.715891+0000", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6856e-05 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:09.715928+0000", 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6819000000000001e-05 2024-09-17T08:16:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:937 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94434125129984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.872433227000002, 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85191269000000003, 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:16:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 937 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411870+0000", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999895 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411884+0000", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.434e-05 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411897+0000", 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3069e-05 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411904+0000", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1919999999999999e-06 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411954+0000", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9398999999999999e-05 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411960+0000", 2024-09-17T08:16:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9059999999999999e-06 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.430585+0000", 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018625889 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.430595+0000", 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0023e-05 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.254928+0000", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.82433245600000005 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for degraded object", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.255372+0000", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044362999999999998 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.257806+0000", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024343569999999998 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.257909+0000", 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103237 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.263793+0000", 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058835199999999997 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218570 7.b 7.b3ceff4b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.451463+0000", 2024-09-17T08:16:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.832849713000002, 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.78587292500000006, 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218570 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451463+0000", 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451463+0000", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451462+0000", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451469+0000", 2024-09-17T08:16:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1969999999999999e-06 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451480+0000", 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1259e-05 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451488+0000", 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5320000000000001e-06 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451682+0000", 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019427600000000001 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.451690+0000", 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.237326+0000", 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.78563625500000001 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.237336+0000", 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173999999999999e-05 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24418.0:308 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:16.081508+0000", 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.202805448999996, 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17516472599999999, 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24418", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.173:0/3469022300", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 308 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081508+0000", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081508+0000", 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081506+0000", 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081514+0000", 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3370000000000002e-06 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081522+0000", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4649999999999994e-06 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081530+0000", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7579999999999992e-06 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081604+0000", 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4431000000000006e-05 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.081612+0000", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5970000000000003e-06 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.256317+0000", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17470539099999999 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.256477+0000", 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001594 2024-09-17T08:16:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.256672+0000", 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019564100000000001 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:221788 7.e 7:73227b75:::benchmark_data_smithi079_83551_object94388:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:20.568572+0000", 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.715741249000004, 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.097769889999999998, 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 221788 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568572+0000", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568572+0000", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568571+0000", 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568576+0000", 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.728e-06 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568687+0000", 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011027999999999999 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568698+0000", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1858999999999999e-05 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568747+0000", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8915999999999997e-05 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568852+0000", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104373 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.568917+0000", 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5513999999999996e-05 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.571597+0000", 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002680174 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.571723+0000", 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012599700000000001 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.666255+0000", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.094531720999999999 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.666305+0000", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9706000000000001e-05 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:20.666342+0000", 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7054999999999997e-05 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:234185 7.e 7:7595df1f:::benchmark_data_smithi079_83551_object106785:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:35.061048+0000", 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.223265441999999, 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.083543603999999994, 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 234185 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061048+0000", 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061048+0000", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061046+0000", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061054+0000", 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4460000000000001e-06 2024-09-17T08:16:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061068+0000", 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4001e-05 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061077+0000", 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7569999999999993e-06 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061125+0000", 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7916e-05 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061232+0000", 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107229 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.061330+0000", 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8270999999999995e-05 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.064739+0000", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034088920000000002 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.077250+0000", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012510999 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.144461+0000", 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067210927000000004 2024-09-17T08:16:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.144553+0000", 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2187000000000004e-05 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:35.144591+0000", 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8002e-05 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:242883 7.e 7:7b23504c:::benchmark_data_smithi079_83551_object115483:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:42.643018+0000", 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.641295595000003, 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095320600000000005, 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 242883 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643018+0000", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643018+0000", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643016+0000", 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643023+0000", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3040000000000001e-06 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643032+0000", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8849999999999994e-06 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643039+0000", 2024-09-17T08:16:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3829999999999997e-06 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643083+0000", 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4058999999999999e-05 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643198+0000", 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115355 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.643306+0000", 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107488 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.646358+0000", 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003052763 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.646437+0000", 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8993999999999996e-05 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.738062+0000", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091624321999999994 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.738217+0000", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000155582 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.738338+0000", 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012082 2024-09-17T08:16:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:243151 7.e 7:7fcf28ff:::benchmark_data_smithi079_83551_object115751:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:42.929200+0000", 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.355113689, 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.092903966000000004, 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 243151 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929200+0000", 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929200+0000", 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929198+0000", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929204+0000", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0410000000000003e-06 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929218+0000", 2024-09-17T08:16:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3647e-05 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929223+0000", 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3249999999999998e-06 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.929255+0000", 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1689000000000001e-05 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.931127+0000", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018717370000000001 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.931265+0000", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013898799999999999 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.934649+0000", 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00338317 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.992257+0000", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057608466999999997 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.021913+0000", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029655457999999999 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.021972+0000", 2024-09-17T08:16:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9015999999999999e-05 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.022103+0000", 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013186099999999999 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:243161 7.e 7:7c4eb8c7:::benchmark_data_smithi079_83551_object115761:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:42.935144+0000", 2024-09-17T08:16:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.349169304999997, 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.087520835000000005, 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 243161 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935144+0000", 2024-09-17T08:16:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935144+0000", 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935142+0000", 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935148+0000", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935160+0000", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1432e-05 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935167+0000", 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7279999999999998e-06 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935204+0000", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6949000000000001e-05 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935306+0000", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102698 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.935414+0000", 2024-09-17T08:16:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107366 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.992291+0000", 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056877664000000001 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:42.992303+0000", 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1983e-05 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.022581+0000", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.030277598999999999 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.022634+0000", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2873e-05 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:43.022665+0000", 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0954000000000002e-05 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:245261 7.e 7:76d73053:::benchmark_data_smithi079_83551_object117861:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:44.893734+0000", 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.390579572999997, 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.089999062000000005, 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 245261 2024-09-17T08:16:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893734+0000", 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893734+0000", 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:16:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893731+0000", 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893739+0000", 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7929999999999998e-06 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893752+0000", 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2897e-05 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.893759+0000", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9759999999999998e-06 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.894032+0000", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000272448 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.894141+0000", 2024-09-17T08:16:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010968600000000001 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.894238+0000", 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6797000000000002e-05 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.897580+0000", 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033415300000000001 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.899210+0000", 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001630521 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.983633+0000", 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084422412000000002 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.983689+0000", 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6014000000000003e-05 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:16:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:44.983733+0000", 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4165000000000002e-05 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:16:37.405 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:16:37.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:37.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:37.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:37.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 8 16 23350 23334 182.256 151 0.0121984 0.00548124 2024-09-17T08:16:37.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.539 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:37.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:37.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:16:37.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:16:37.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:16:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:16:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:16:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:37.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:37 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:37.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:37.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:37.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:16:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:37 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:37.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:37.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:37.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:37.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:37.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:37.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:37.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:38.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:38.081 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:38.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:38.214 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:38.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:38.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:38.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:38.472 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:38.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 9 16 25795 25779 178.985 152.812 0.00720878 0.00558124 2024-09-17T08:16:38.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:38.598 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:38.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:38 smithi173 ceph-mon[33187]: pgmap v284: 148 pgs: 2 peering, 146 active+clean; 849 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:16:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:38 smithi173 ceph-mon[33187]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T08:16:38.732 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:38.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:38.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:38 smithi079 ceph-mon[29565]: pgmap v284: 148 pgs: 2 peering, 146 active+clean; 849 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:16:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:38 smithi079 ceph-mon[29565]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T08:16:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:38 smithi079 ceph-mon[40649]: pgmap v284: 148 pgs: 2 peering, 146 active+clean; 849 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:16:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:38 smithi079 ceph-mon[40649]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T08:16:38.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:38.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:38.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:38.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:39.011 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:39.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:39.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:39.143 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:39.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:39.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:39.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.279 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:39.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:39.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:39.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:39.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:39.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 10 16 27963 27947 174.635 135.5 0.00356872 0.00572083 2024-09-17T08:16:39.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:39.562 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:39.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:39.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:39.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:16:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:16:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:16:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:16:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:39 smithi173 ceph-mon[33187]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T08:16:39.797 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:39.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:39.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[40649]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:16:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:16:39.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:16:39.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:39 smithi079 ceph-mon[29565]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T08:16:39.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:39.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:39.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:39.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:40.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:40.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.107 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:40.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:40.167 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:40.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:40.249 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:40.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:40.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:40.382 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:40.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.383 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:40.410 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:16:40.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:40.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 11 16 30026 30010 170.481 128.938 0.00571188 0.00586047 2024-09-17T08:16:40.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:40.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:40.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:40.650 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:40.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:40 smithi173 ceph-mon[33187]: pgmap v287: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 850 B/s rd, 140 MiB/s wr, 2.24k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:40 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:16:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:40 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:16:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:40 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2824921274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:40 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:40.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:40.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[29565]: pgmap v287: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 850 B/s rd, 140 MiB/s wr, 2.24k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:16:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:16:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2824921274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[40649]: pgmap v287: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 850 B/s rd, 140 MiB/s wr, 2.24k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2824921274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:40 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:16:40.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:40.908 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:40.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:40.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:41.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:41.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:41.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:41.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:41.183 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:41.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:41.285 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:41.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:41.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:41.412 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:41.434 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:41.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:41.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 12 16 31757 31741 165.289 108.188 0.00468942 0.00604481 2024-09-17T08:16:41.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:41.573 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:41.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:41.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:41.711 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:41.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:41.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T08:16:41.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:41 smithi173 ceph-mon[33187]: pgmap v289: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 140 MiB/s wr, 2.25k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:41.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:41.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:41.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T08:16:41.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[40649]: pgmap v289: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 140 MiB/s wr, 2.25k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T08:16:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:41 smithi079 ceph-mon[29565]: pgmap v289: 148 pgs: 148 active+clean; 1.4 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 140 MiB/s wr, 2.25k op/s; 36 B/s, 2 objects/s recovering 2024-09-17T08:16:41.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:41.964 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:41.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:41.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:42.065 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:42.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:42.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:42.219 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:42.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:42.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:42.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:42.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:42.491 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:42.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 13 16 33646 33630 161.656 118.062 0.0178583 0.00617715 2024-09-17T08:16:42.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:42.629 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:42 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:16:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:42 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:16:42.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:42.730 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:42.757 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:42.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:42.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:42 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:16:42.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:42 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:16:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:42 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:16:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:42 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:16:42.891 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:42.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:42.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:42.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:43.025 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:43.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:43.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:43.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:43.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:43.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:43.287 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:43.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:43.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:43.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:43.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:43.512 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:43.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 14 16 35248 35232 157.261 100.125 0.00659589 0.00635354 2024-09-17T08:16:43.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:43.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:43.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:43.682 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:43.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:16:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:16:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:16:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:16:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 2.1f deep-scrub starts 2024-09-17T08:16:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: 2.1f deep-scrub ok 2024-09-17T08:16:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:43 smithi173 ceph-mon[33187]: pgmap v290: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 212 MiB/s wr, 3.39k op/s; 18 B/s, 1 objects/s recovering 2024-09-17T08:16:43.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:43.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:16:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:16:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 2.1f deep-scrub starts 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: 2.1f deep-scrub ok 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[40649]: pgmap v290: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 212 MiB/s wr, 3.39k op/s; 18 B/s, 1 objects/s recovering 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 2.1f deep-scrub starts 2024-09-17T08:16:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: 2.1f deep-scrub ok 2024-09-17T08:16:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:43 smithi079 ceph-mon[29565]: pgmap v290: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 212 MiB/s wr, 3.39k op/s; 18 B/s, 1 objects/s recovering 2024-09-17T08:16:43.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:43.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:43.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:43.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:44.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:44.094 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:44.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:44.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:44.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.217 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:44.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:44.318 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:44.350 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:44.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:44.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:44.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.478 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:44.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:44.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 15 16 36887 36871 153.605 102.438 0.00195605 0.00648142 2024-09-17T08:16:44.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:44.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:44.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:44.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:44 smithi173 ceph-mon[33187]: 2.7 deep-scrub starts 2024-09-17T08:16:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:44 smithi173 ceph-mon[33187]: 2.7 deep-scrub ok 2024-09-17T08:16:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:44 smithi173 ceph-mon[33187]: 2.3 deep-scrub starts 2024-09-17T08:16:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:44 smithi173 ceph-mon[33187]: 2.3 deep-scrub ok 2024-09-17T08:16:44.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:44.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:44.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[29565]: 2.7 deep-scrub starts 2024-09-17T08:16:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[29565]: 2.7 deep-scrub ok 2024-09-17T08:16:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[29565]: 2.3 deep-scrub starts 2024-09-17T08:16:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[29565]: 2.3 deep-scrub ok 2024-09-17T08:16:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[40649]: 2.7 deep-scrub starts 2024-09-17T08:16:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[40649]: 2.7 deep-scrub ok 2024-09-17T08:16:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[40649]: 2.3 deep-scrub starts 2024-09-17T08:16:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:44 smithi079 ceph-mon[40649]: 2.3 deep-scrub ok 2024-09-17T08:16:44.871 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:44.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:44.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:44.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:45.011 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:45.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:45.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:45.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.146 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:45.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:45.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:45.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:45.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:45.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:45.410 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:45.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.411 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:45.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:45.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 16 16 38794 38778 151.454 119.188 0.00969201 0.00659765 2024-09-17T08:16:45.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:45.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:45.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:45.684 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:45.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:45 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:16:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:45 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:16:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:45 smithi173 ceph-mon[33187]: pgmap v291: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 160 MiB/s wr, 2.56k op/s; 13 B/s, 1 objects/s recovering 2024-09-17T08:16:45.786 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:45.816 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:45.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:16:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:16:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[40649]: pgmap v291: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 160 MiB/s wr, 2.56k op/s; 13 B/s, 1 objects/s recovering 2024-09-17T08:16:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:16:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:16:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:45 smithi079 ceph-mon[29565]: pgmap v291: 148 pgs: 148 active+clean; 2.1 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 160 MiB/s wr, 2.56k op/s; 13 B/s, 1 objects/s recovering 2024-09-17T08:16:45.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:45.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:45.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:45.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:46.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:46.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:46.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:46.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:16:46.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:46.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:46.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:46.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:46.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:46.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:46.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:46.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 17 16 40718 40702 149.618 120.25 0.00480863 0.00667826 2024-09-17T08:16:46.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.518 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:46.519 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:16:46.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:16:46.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:46 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:16:46.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:46.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.666 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:46.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:16:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:16:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:16:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:16:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:16:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:46 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:16:46.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:46.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:46.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:46.940 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:46.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:46.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:47.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:47.075 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:47.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:47.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:47.218 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:47.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:47.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:47.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:47.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:47.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:47.499 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:47.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:47.515 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 18 16 42646 42630 148 120.5 0.00805631 0.0067517 2024-09-17T08:16:47.598 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:16:47.599 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:16:47.601 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:47.628 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:47.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:47.638 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:16:47.638 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:16:47.669 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:16:47.670 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:16:47.704 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:16:47.704 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:16:47.730 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:47.739 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:16:47.739 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:16:47.761 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:47.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:47.770 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:16:47.771 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:16:47.801 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:16:47.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:16:47.834 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:16:47.834 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:16:47.864 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:47.893 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:47.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:47.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:47.960 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: pgmap v292: 148 pgs: 148 active+clean; 2.2 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 571 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:16:47.961 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3762887906' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: pgmap v292: 148 pgs: 148 active+clean; 2.2 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 571 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:16:47.969 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3762887906' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: pgmap v292: 148 pgs: 148 active+clean; 2.2 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 571 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:16:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3762887906' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:16:47.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:48.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:48.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:48.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:48.170 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:48.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:48.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:48.309 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:48.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:48.410 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:48.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:48.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:48.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 19 16 44548 44532 146.466 118.875 0.00755434 0.00682302 2024-09-17T08:16:48.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:48.584 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:48.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:48.685 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:48.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:48.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:48.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:48.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.849 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:48.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:48.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 6.3 deep-scrub starts 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 6.3 deep-scrub ok 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 6.3 deep-scrub starts 2024-09-17T08:16:48.954 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 6.3 deep-scrub ok 2024-09-17T08:16:48.955 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:16:48.955 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:48 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:16:48.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:48.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:48.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:49.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:49.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.123 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:49.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:49.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 6.3 deep-scrub starts 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 6.3 deep-scrub ok 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:16:49.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:48 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:16:49.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:49.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:49.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:49.389 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:49.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:49.491 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:49.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:16:49.516738+0000 min lat: 0.00121752 max lat: 0.157719 avg lat: 0.0069007 2024-09-17T08:16:49.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:16:49.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 20 16 46362 46346 144.811 113.375 0.00577849 0.0069007 2024-09-17T08:16:49.520 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:49.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:49.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:49.655 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:49.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:49.757 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:49.790 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:49.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:16:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:16:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 5.0 deep-scrub starts 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 5.0 deep-scrub ok 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 6.1a deep-scrub starts 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 6.1a deep-scrub ok 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 5.1a deep-scrub starts 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 5.1a deep-scrub ok 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:16:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:16:49.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[29565]: pgmap v293: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-17T08:16:49.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:16:49.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:16:49.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 5.0 deep-scrub starts 2024-09-17T08:16:49.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 5.0 deep-scrub ok 2024-09-17T08:16:49.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 6.1a deep-scrub starts 2024-09-17T08:16:49.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 6.1a deep-scrub ok 2024-09-17T08:16:49.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 5.1a deep-scrub starts 2024-09-17T08:16:49.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 5.1a deep-scrub ok 2024-09-17T08:16:49.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:16:49.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:16:49.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:49 smithi079 ceph-mon[40649]: pgmap v293: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-17T08:16:49.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:49.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:49.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:49.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:50.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:16:50.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:50.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.069 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:50.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:50.170 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:50.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:50.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 5.0 deep-scrub starts 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 5.0 deep-scrub ok 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 6.1a deep-scrub starts 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 6.1a deep-scrub ok 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 5.1a deep-scrub starts 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 5.1a deep-scrub ok 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:16:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:49 smithi173 ceph-mon[33187]: pgmap v293: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-17T08:16:50.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:50.331 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:50.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:50.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:50.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:50.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 21 15 47878 47863 142.43 94.8125 0.00537801 0.00701619 2024-09-17T08:16:50.579 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:50.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:50.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:50.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:50.746 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:50.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:50.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:50.875 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:50.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:50.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:50.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:51.007 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:51.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:51.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:51.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:50 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:16:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:50 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:16:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:50 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:16:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:50 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:16:51.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:51.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:50 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:16:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:50 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:16:51.227 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:51.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:51.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:51.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:51.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:51.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.412 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:51.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:51.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:51.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 22 16 49523 49507 140.626 102.75 0.0184542 0.00710615 2024-09-17T08:16:51.563 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:51.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:51.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:16:51.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:51.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:51.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.721 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:51.762 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T08:16:51.787 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: 3.1 deep-scrub starts 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: 3.1 deep-scrub ok 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: pgmap v294: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 936 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1173385857' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:51.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: 3.1 deep-scrub starts 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: 3.1 deep-scrub ok 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: pgmap v294: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 936 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1173385857' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:51.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:51.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:51.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:51.865 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:51.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:51.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:51.999 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:52.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.001 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:52.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:52.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.142 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:52.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:52.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: 3.1 deep-scrub starts 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: 3.1 deep-scrub ok 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: pgmap v294: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 936 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1173385857' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:16:52.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:52.277 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:52.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:52.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:52.414 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:52.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:52.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:52.517 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 23 16 51230 51214 139.15 106.688 0.00627899 0.00718119 2024-09-17T08:16:52.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.552 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:52.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:52.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:52.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:52.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:52.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:52.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:52.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:52.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:16:52.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:16:52.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:16:52.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T08:16:52.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:16:52.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:16:52.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 5.6 deep-scrub starts 2024-09-17T08:16:52.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[40649]: 5.6 deep-scrub ok 2024-09-17T08:16:52.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:16:52.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:16:52.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:16:52.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T08:16:52.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:16:52.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:16:52.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 5.6 deep-scrub starts 2024-09-17T08:16:52.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:52 smithi079 ceph-mon[29565]: 5.6 deep-scrub ok 2024-09-17T08:16:52.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:52.957 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:52.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:52.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:52.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:16:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 5.6 deep-scrub starts 2024-09-17T08:16:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:52 smithi173 ceph-mon[33187]: 5.6 deep-scrub ok 2024-09-17T08:16:53.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:53.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:53.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:53.131 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:53.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:53.243 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:53.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:53.281 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:16:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:16:53.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:53.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:53.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:53.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:53.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 24 15 52921 52906 137.758 105.75 0.00779734 0.00725456 2024-09-17T08:16:53.523 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:53.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:53.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:53.665 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:53.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:53.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-17T08:16:53.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:53.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:53.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:53.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:53.930 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:53.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:53.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:54.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:54.033 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:54.033 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:16:54.033 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:16:54.033 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[40649]: pgmap v296: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:16:54.034 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:53 smithi079 ceph-mon[29565]: pgmap v296: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:54.101 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:54.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:54.103 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:16:54.103 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:16:54.204 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:54.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:16:54.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:16:54.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:16:54.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:16:54.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:16:54.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:53 smithi173 ceph-mon[33187]: pgmap v296: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:54.235 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:54.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:54.337 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:54.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:54.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:54.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 25 16 54540 54524 136.293 101.125 0.00479056 0.00733207 2024-09-17T08:16:54.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:54.605 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:54.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:54.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:54.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:54.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:54.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:54.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:54.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:54.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:54.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:55.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:16:55.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:55.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.137 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:55.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:55.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.274 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:55.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:55 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:16:55.280 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:55 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:16:55.280 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:55 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:16:55.280 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:55 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:16:55.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:55.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:55.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:55.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:55 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:16:55.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:55 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:16:55.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 26 16 56175 56159 134.981 102.188 0.00747307 0.00740373 2024-09-17T08:16:55.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:55.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:55.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:55.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:55.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:16:55.799 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:55.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:55.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:55.962 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:55.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:55.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:56.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:56.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.097 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:56.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:56.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:56.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:56.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:56.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:56 smithi079 ceph-mon[29565]: pgmap v297: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:56.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:56 smithi079 ceph-mon[40649]: pgmap v297: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:56.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:56.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:56 smithi173 ceph-mon[33187]: pgmap v297: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-17T08:16:56.496 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:56.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:56.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 27 16 57730 57714 133.581 97.1875 0.0117649 0.00748122 2024-09-17T08:16:56.598 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:56.631 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:56.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:56.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:56.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:16:56.897 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:56.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:56.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:16:56.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:16:57.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:16:57.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:57.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:57.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:57.292 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:57.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:57.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:57.419 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:57.516 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 28 16 59094 59078 131.854 85.25 0.0148949 0.00757992 2024-09-17T08:16:57.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:57.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:57.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:16:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:16:57.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:16:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:16:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:16:57.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:57 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:16:57.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:57.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:57 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:16:57.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:57 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:16:57.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:57 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:16:57.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:57.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:57.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:57.952 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:57.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:57.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:58.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:58.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.086 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:58.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:58.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:58.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.222 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:58.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:58.323 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:58.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:58.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:58.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:58.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:58.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:58.517 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 29 16 60619 60603 130.594 95.3125 0.0166524 0.00765221 2024-09-17T08:16:58.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:16:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:16:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:16:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:16:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[40649]: pgmap v298: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:16:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:16:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:16:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:16:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:16:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:58 smithi079 ceph-mon[29565]: pgmap v298: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:16:58.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.623 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:58.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:58 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:16:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:58 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:16:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:58 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:16:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:58 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:16:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:58 smithi173 ceph-mon[33187]: pgmap v298: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.72k op/s 2024-09-17T08:16:58.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:58.760 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:58.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:58.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:16:58.889 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:58.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:58.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:16:58.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:59.025 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:59.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:59.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:59.157 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:59.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:59.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:59.294 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:59.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:16:59.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:16:59.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:59.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:16:59.517 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 30 16 62206 62190 129.544 99.1875 0.0100392 0.00771449 2024-09-17T08:16:59.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:16:59.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:16:59.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:16:59.576 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:16:59.576 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:16:59.576 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:16:59.576 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:16:59.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:16:59.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:16:59.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:16:59.577 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:16:59 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:16:59.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:16:59.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:59.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:16:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:59 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:16:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:59 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:16:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:59 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:16:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:16:59 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:16:59.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:16:59.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.844 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:59.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:16:59.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:16:59.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:16:59.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:16:59.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:00.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:00.114 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:00.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:00.244 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:00.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:00.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:00.407 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[29565]: pgmap v299: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:17:00.408 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[40649]: pgmap v299: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:00.409 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:17:00.409 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:00 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:17:00.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:00.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 31 16 63704 63688 128.385 93.625 0.0126711 0.00778437 2024-09-17T08:17:00.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:00.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:00.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:00.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.650 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:00 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:17:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:00 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:17:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:00 smithi173 ceph-mon[33187]: pgmap v299: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:00 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:17:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:00 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:17:00.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:00.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:00.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:00.901 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:00.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:00.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:01.002 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:01.039 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:01.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:01.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:01.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:01.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:01.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:01.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:17:01.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:17:01.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:17:01.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:01 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:01.409 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:01.440 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:01.517 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 32 16 65089 65073 127.078 86.5625 0.00835673 0.00786386 2024-09-17T08:17:01.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:01.573 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:01.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:01.706 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:17:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:01 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:01.807 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:01.832 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:01.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:01.963 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:01.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:01.964 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:02.065 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:02.092 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:02.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:02.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:02.219 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:02.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:02.321 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:02.329 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:17:02.330 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:17:02.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:02.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:02.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:17:02.372 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:17:02.410 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:02.427 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:17:02.427 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: 4.15 deep-scrub starts 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: 4.15 deep-scrub ok 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: pgmap v300: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:02.428 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/796535837' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: 4.15 deep-scrub starts 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: 4.15 deep-scrub ok 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: pgmap v300: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:02.429 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/796535837' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:02.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:02.461 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:17:02.461 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:17:02.492 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:17:02.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:17:02.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.508 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:02.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:02.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 33 16 66442 66426 125.787 84.5625 0.00462432 0.00794451 2024-09-17T08:17:02.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:17:02.529 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:17:02.557 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:17:02.557 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:17:02.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:17:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: 4.15 deep-scrub starts 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: 4.15 deep-scrub ok 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: pgmap v300: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:02.597 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/796535837' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:02.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:02.650 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:02.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:02.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:02.773 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:02.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:02.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:02.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:02.913 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:02.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:02.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:03.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:03.054 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:03.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:03.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:03.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.190 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:03.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:03.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:03.306 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:03.314 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:17:03.336 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:03.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:03.438 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:03.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:03.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:03.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 34 16 67798 67782 124.58 84.75 0.00789011 0.00802171 2024-09-17T08:17:03.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:03.614 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:03.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 2.1c deep-scrub starts 2024-09-17T08:17:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 2.1c deep-scrub ok 2024-09-17T08:17:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:17:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:17:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2985396865' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 2.1c deep-scrub starts 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 2.1c deep-scrub ok 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2985396865' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:17:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:03 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:17:03.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 2.1c deep-scrub starts 2024-09-17T08:17:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 2.1c deep-scrub ok 2024-09-17T08:17:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:17:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:17:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2985396865' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:17:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:03 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:17:03.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:03.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:03.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:03.889 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:03.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:03.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:03.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:04.032 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:04.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:04.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:04.164 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:04.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:04.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:04.291 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:04.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:04.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:04.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:04.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:04.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[40649]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[40649]: pgmap v302: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[29565]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T08:17:04.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:04 smithi079 ceph-mon[29565]: pgmap v302: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:17:04.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 35 16 69245 69229 123.604 90.4375 0.0124756 0.00808534 2024-09-17T08:17:04.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:04.566 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:04.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:04.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:04.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:04.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:04 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:04 smithi173 ceph-mon[33187]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T08:17:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:04 smithi173 ceph-mon[33187]: pgmap v302: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:17:04.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:04.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.833 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:04.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:04.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:04.974 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:04.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:04.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:05.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:05.114 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:05.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:05.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:05.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.250 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:05.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:05.341 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T08:17:05.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:05.392 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:05.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:05.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:05.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 36 16 70659 70643 122.625 88.375 0.00562247 0.00815042 2024-09-17T08:17:05.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:05.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:05.592 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:17:05.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:17:05.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:05.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:05.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:17:05.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:05.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:05.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:05.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:17:05.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:05.937 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:05.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:05.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:06.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:06.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:06.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:06.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:06.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.242 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:06.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:06.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:06.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:06.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:06.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:06.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:06.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:06.518 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 37 16 72039 72023 121.642 86.25 0.00744286 0.00821601 2024-09-17T08:17:06.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:17:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[40649]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T08:17:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[40649]: pgmap v304: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:17:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[40649]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T08:17:06.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:17:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[29565]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T08:17:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[29565]: pgmap v304: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:17:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:06 smithi079 ceph-mon[29565]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T08:17:06.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:06.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3941962664' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:17:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:06 smithi173 ceph-mon[33187]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T08:17:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:06 smithi173 ceph-mon[33187]: pgmap v304: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:17:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:06 smithi173 ceph-mon[33187]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T08:17:06.757 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:06.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:06.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:06.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:06.913 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:06.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:06.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:07.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:07.057 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:07.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:07.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:07.190 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:07.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:07.292 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:07.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:07.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:07.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:07.452 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:07.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:07.519 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 38 15 73189 73174 120.334 71.9375 0.0209969 0.00830513 2024-09-17T08:17:07.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:07.589 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:07.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:07.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:17:07.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:07.721 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:07.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:07.863 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:07.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:07.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:07.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:07.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:07.998 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:08.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:08.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.133 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:08.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:08.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:08.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:08.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:08.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:08.400 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:08.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:08.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:08.519 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 39 16 74362 74346 119.127 73.25 0.00512839 0.00838946 2024-09-17T08:17:08.529 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:08.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:08 smithi079 ceph-mon[40649]: pgmap v306: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-09-17T08:17:08.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:08 smithi079 ceph-mon[29565]: pgmap v306: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-09-17T08:17:08.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:08.664 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:08.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:08 smithi173 ceph-mon[33187]: pgmap v306: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 94 MiB/s wr, 1.51k op/s 2024-09-17T08:17:08.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:08.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:08.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:08.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:08.916 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:08.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:08.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:09.017 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:09.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:09.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:09.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:09.188 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:09.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:09.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:09.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-17T08:17:09.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:09.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:09.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:09.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:09.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:09.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:17:09.522105+0000 min lat: 0.00121752 max lat: 0.193327 avg lat: 0.00847442 2024-09-17T08:17:09.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:17:09.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 40 16 75510 75494 117.935 71.75 0.0138625 0.00847442 2024-09-17T08:17:09.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:09.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.599 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:09.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:09.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:17:09.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:17:09.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[29565]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T08:17:09.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:17:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:17:09.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:09 smithi079 ceph-mon[40649]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T08:17:09.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:09 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:17:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:09 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:17:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:09 smithi173 ceph-mon[33187]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T08:17:09.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:09.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:09.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:09.878 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:09.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:09.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:09.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:09.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:10.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:10.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:10.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:10.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:10.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:10.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:10.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:10.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:10.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[29565]: pgmap v308: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[40649]: pgmap v308: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:17:10.422 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:10 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:17:10.466 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:10.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:10.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 41 16 76811 76795 117.042 81.3125 0.0112226 0.0085389 2024-09-17T08:17:10.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:10.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:10.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:10.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:10 smithi173 ceph-mon[33187]: pgmap v308: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-17T08:17:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:10 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:17:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:10 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:17:10.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:10.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:10.844 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:10.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:10.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:10.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:10.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:11.016 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:11.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:11.158 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:11.260 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:11.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.288 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:11.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:11.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:11.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:11.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 42 16 78133 78117 116.223 82.625 0.0211706 0.00859931 2024-09-17T08:17:11.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:11.567 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:11.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:11 smithi079 ceph-mon[40649]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T08:17:11.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:11 smithi079 ceph-mon[40649]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T08:17:11.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:11 smithi079 ceph-mon[29565]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T08:17:11.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:11 smithi079 ceph-mon[29565]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T08:17:11.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:11.670 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:17:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:17:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:17:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:17:11.671 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:17:11.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:11 smithi173 ceph-mon[33187]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T08:17:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:11 smithi173 ceph-mon[33187]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T08:17:11.804 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:11.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:11.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:11.902 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:11.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:11.967 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:11.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:11.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:12.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:12.105 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:12.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:12.242 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.243 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:12.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:12.379 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:12.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:17:12.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:12.522 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 43 16 79463 79447 115.453 83.125 0.0282298 0.0086571 2024-09-17T08:17:12.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:12.553 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:12.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:12.664 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:12 smithi079 ceph-mon[40649]: pgmap v311: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1006 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:17:12.665 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:12 smithi079 ceph-mon[29565]: pgmap v311: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1006 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:17:12.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.682 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:12 smithi173 ceph-mon[33187]: pgmap v311: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1006 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:17:12.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:12.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:12.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:12.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:12.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:12.957 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:12.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:12.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:13.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:13.101 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:13.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:13.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:13.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.233 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:13.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:13.296 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:13.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:13.361 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:13.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:13.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:13.508 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:13.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:13.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 44 16 80705 80689 114.593 77.625 0.0086531 0.00872167 2024-09-17T08:17:13.610 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:13.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:13.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:13.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:13.776 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:13.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:13.878 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:13.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:13.908 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:13.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:14.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:14.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:14.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:14.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:14.187 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:14.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.188 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:14.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:14.330 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:14.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:14.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:14.466 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:14.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:14.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 45 16 81985 81969 113.824 80 0.0116525 0.00878076 2024-09-17T08:17:14.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:14.601 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:14.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:14.689 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[40649]: pgmap v312: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 118 MiB/s wr, 1.88k op/s 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[29565]: pgmap v312: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 118 MiB/s wr, 1.88k op/s 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:17:14.690 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:14 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:17:14.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:14 smithi173 ceph-mon[33187]: pgmap v312: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 118 MiB/s wr, 1.88k op/s 2024-09-17T08:17:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:14 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:17:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:14 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:17:14.732 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:14.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:14.786 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:17:14.787 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:17:14.824 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:17:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:17:14.940 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:17:14.940 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:17:14.940 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:17:14.941 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:17:14.941 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:17:14.941 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:17:14.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:17:14.965 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:17:14.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:14.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:14.988 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:14.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:17:14.995 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:17:15.027 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:17:15.028 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:17:15.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:15.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:15.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:15.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:15.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.265 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:15.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:15.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:15.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:15.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:15.512 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:15.521 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 46 16 83242 83226 113.058 78.5625 0.0103365 0.00884021 2024-09-17T08:17:15.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:15.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:15.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:15 smithi079 ceph-mon[29565]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T08:17:15.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/819595056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:15.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:15 smithi079 ceph-mon[40649]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T08:17:15.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/819595056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:15.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:15.681 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:15.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:15 smithi173 ceph-mon[33187]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T08:17:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/819595056' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:15.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:15.818 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:15.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:15.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:15.957 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:15.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:15.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:16.059 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:16.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:16.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:16.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:16.222 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:16.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:16.324 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:16.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:16.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:16.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:16.489 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:16.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:16.522 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 47 16 84560 84544 112.405 82.375 0.00812046 0.00889169 2024-09-17T08:17:16.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:16.630 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:16.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:16 smithi173 ceph-mon[33187]: pgmap v314: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:16 smithi173 ceph-mon[33187]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T08:17:16.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:16.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:16.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:16.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:16 smithi079 ceph-mon[29565]: pgmap v314: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:16.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:16 smithi079 ceph-mon[29565]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T08:17:16.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:16 smithi079 ceph-mon[40649]: pgmap v314: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:16.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:16 smithi079 ceph-mon[40649]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T08:17:16.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:16.911 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:16.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:16.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:17.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:17.049 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:17.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:17.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:17.194 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:17.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:17.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:17.337 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:17.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.338 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:17.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:17.471 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:17.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:17.522 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 48 16 85773 85757 111.642 75.8125 0.00843134 0.00895247 2024-09-17T08:17:17.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:17.616 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:17.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:17.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:17.746 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:17.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:17.753 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:17.775 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:17:17.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:17.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:17.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:17.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:17.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:18.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.016 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:18.016 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:18.117 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[29565]: pgmap v316: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 841 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:17:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1335982697' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[40649]: pgmap v316: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 841 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1335982697' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:17 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.150 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:18.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.151 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:18.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:18.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:18.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:17 smithi173 ceph-mon[33187]: pgmap v316: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 841 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:17:18.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:17 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1335982697' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:17 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:18.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:18.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:18.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:18.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:18.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:18.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:18.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:18.522 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 49 16 87016 87000 110.949 77.6875 0.00883336 0.00900837 2024-09-17T08:17:18.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:18.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:18.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:18.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:18.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:18.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:18.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:18.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:18.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:18.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:18.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[29565]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T08:17:18.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:18.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:18.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:18 smithi079 ceph-mon[40649]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T08:17:18.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:18.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:18.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:18.969 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:19.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:17:19.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:19.101 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:19.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:18 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:18 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:18 smithi173 ceph-mon[33187]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T08:17:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:19.330 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:19.361 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:19.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:19.490 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:19.522 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 50 16 88250 88234 110.273 77.125 0.0145805 0.00906371 2024-09-17T08:17:19.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:19.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:19.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:19.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:19.889 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:19.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:19.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:19.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:20.025 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:19 smithi079 ceph-mon[40649]: pgmap v318: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:19 smithi079 ceph-mon[29565]: pgmap v318: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:20.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:20.162 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:19 smithi173 ceph-mon[33187]: pgmap v318: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:20.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:20.285 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:20.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:20.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:20.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:20.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:20.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 51 16 89375 89359 109.487 70.3125 0.0112623 0.00912864 2024-09-17T08:17:20.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:20.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:17:20.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:20.683 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:20.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:20.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:20.809 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:20.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:20.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:20.962 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:21.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:21.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:21.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:21.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.233 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:21.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:21.333 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:21.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:21.467 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:21.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:21.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:21.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 52 16 90337 90321 108.538 60.125 0.0217975 0.00920831 2024-09-17T08:17:21.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:21.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:21.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.645 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:21.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:21.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:21.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:21.874 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:21.901 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:21.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:21.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:22.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:22.032 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:22.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:22.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:17:22.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:17:22.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[40649]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T08:17:22.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[40649]: pgmap v320: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:17:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:17:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[29565]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T08:17:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:21 smithi079 ceph-mon[29565]: pgmap v320: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:22.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:22.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:22.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:21 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:17:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:21 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:17:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:21 smithi173 ceph-mon[33187]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T08:17:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:21 smithi173 ceph-mon[33187]: pgmap v320: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:17:22.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:22.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:22.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:22.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:22.460 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:22.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:22.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 53 16 91544 91528 107.913 75.4375 0.00838359 0.00926135 2024-09-17T08:17:22.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:22.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:22.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:22.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:22.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.735 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:22.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:22.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:22.870 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:22.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:22.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:22.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:22.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:23.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.006 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:23.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:23.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:23.146 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:23.183 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:23.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:23.283 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:23.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:23.312 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T08:17:23.332 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:23.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:23.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:23.477 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:23.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:23.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 54 16 92686 92670 107.237 71.375 0.00771335 0.00931993 2024-09-17T08:17:23.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:23.614 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:23.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:23.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3986205614' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3986205614' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:23.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3986205614' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:17:23.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:23.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:23.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:23.881 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:23.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:23.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:23.915 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:23.939 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:17:23.939 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:17:23.983 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:24.013 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:24.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:24.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:24.155 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:24.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:24.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:24.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:24.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:24.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:24.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.430 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:24.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:24.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 55 16 93929 93913 106.699 77.6875 0.00765926 0.00936752 2024-09-17T08:17:24.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:24.563 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:24.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:17:24.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[40649]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T08:17:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[40649]: pgmap v322: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:17:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:17:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[29565]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T08:17:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:24 smithi079 ceph-mon[29565]: pgmap v322: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:17:24.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:24.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:24.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:17:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:24 smithi173 ceph-mon[33187]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T08:17:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:24 smithi173 ceph-mon[33187]: pgmap v322: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:17:24.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:24.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:24.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:24.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:24.976 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:24.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:24.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:25.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:25.108 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:25.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:25.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:17:25.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:25.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:25.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:25.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:25.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.394 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:25.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:25.461 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:25.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:25.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 56 16 95182 95166 106.192 78.3125 0.00483435 0.00941249 2024-09-17T08:17:25.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:25.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:25.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:25.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:25.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:25.819 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:25.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:25.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:25.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:25.962 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:25.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:26.063 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:26.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:26.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:26.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:26.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:26.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:26.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:26.380 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:26.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:26.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:26.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:26.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:26.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 57 16 96465 96449 105.736 80.1875 0.0133686 0.00945284 2024-09-17T08:17:26.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:26.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.661 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:26.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:26.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:26 smithi173 ceph-mon[33187]: pgmap v323: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 675 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:26.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:26.806 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:26.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:26 smithi079 ceph-mon[29565]: pgmap v323: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 675 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:26 smithi079 ceph-mon[40649]: pgmap v323: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 675 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:26.908 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:26.940 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:26.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:26.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:26.959 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T08:17:26.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:17:27.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:27.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:27.185 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:27.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:27.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:27.329 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:27.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:27.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:27.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:27.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:27.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:27.523 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 58 16 97565 97549 105.098 68.75 0.0106944 0.00950995 2024-09-17T08:17:27.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:27.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.662 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:27.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:27.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:27.803 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:27.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:27.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:27.939 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:27.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:27.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:28.041 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:28.079 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:28.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:28.105 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:17:28.105 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:17:28.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:28.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:28.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:28.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:28.368 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:28.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:28.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:28.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:28.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 59 16 98843 98827 104.671 79.875 0.0139341 0.00954925 2024-09-17T08:17:28.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:28.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:28.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:28 smithi173 ceph-mon[33187]: pgmap v324: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 913 op/s 2024-09-17T08:17:28.726 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:28 smithi079 ceph-mon[40649]: pgmap v324: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 913 op/s 2024-09-17T08:17:28.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:28 smithi079 ceph-mon[29565]: pgmap v324: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 913 op/s 2024-09-17T08:17:28.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:28.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:28.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:28.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:17:28.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:28.926 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:28.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:28.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:29.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:29.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.063 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:29.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:29.144 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:29.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:29.200 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:29.302 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:29.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:29.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:29.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:29.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.478 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:29.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:29.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:17:29.524720+0000 min lat: 0.00121752 max lat: 0.193327 avg lat: 0.00959307 2024-09-17T08:17:29.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:17:29.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 60 16 100059 100043 104.193 76 0.0112478 0.00959307 2024-09-17T08:17:29.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:29.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:29.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:29.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:29.716 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:17:29.716 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:17:29.725 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:29.727 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:17:29.727 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:17:29.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:29.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:29.772 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:17:29.773 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:17:29.811 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:17:29.811 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:17:29.846 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:29.848 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:17:29.848 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:17:29.874 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:29.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:29.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:29.887 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:17:29.888 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:17:29.931 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:17:29.932 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:17:29.963 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:17:29.963 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:17:29.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:30.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.015 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:30.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:30.116 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:30.153 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:30.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:30.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:30.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:30.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:30.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:30.429 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:17:30.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.450 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:30.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:30.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 61 16 101156 101140 103.609 68.5625 0.0125026 0.00964699 2024-09-17T08:17:30.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:30.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.590 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:30.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:30.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:30.722 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:30.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:30 smithi173 ceph-mon[33187]: pgmap v325: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-17T08:17:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:30 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3129168562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:30 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1956949170' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:30 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[29565]: pgmap v325: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3129168562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1956949170' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[40649]: pgmap v325: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3129168562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1956949170' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.810 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:30 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:17:30.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:30.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:30.870 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:30.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:30.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:31.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:31.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:31.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:31.131 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:31.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.132 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:31.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:31.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:31.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:31.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:31.389 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:31.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:31.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:31.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 62 16 102298 102282 103.089 71.375 0.01988 0.00969617 2024-09-17T08:17:31.530 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:31.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:31.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:31.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:31.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:31.710 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:31 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:31.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:31 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:17:31.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:31 smithi173 ceph-mon[33187]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T08:17:31.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:31.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:31.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:17:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[29565]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T08:17:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:17:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:31 smithi079 ceph-mon[40649]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T08:17:31.902 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:31.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:31.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:31.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:32.029 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:32.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:17:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:17:32.030 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:17:32.030 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:32.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:32.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:32.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:32.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:32.204 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:32.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:32.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:32.349 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:32.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:32.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:32.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:32.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:32.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 63 16 103575 103559 102.719 79.8125 0.00904776 0.00973052 2024-09-17T08:17:32.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 636 B/s rd, 47 MiB/s wr, 753 op/s 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:17:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 636 B/s rd, 47 MiB/s wr, 753 op/s 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:17:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:17:32.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:17:32.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:32.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.623 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:32.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:32.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 636 B/s rd, 47 MiB/s wr, 753 op/s 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:17:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:17:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:17:32.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:32.725 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:32.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:32.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:32.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:32.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:32.878 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:32.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:32.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:33.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:33.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:33.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:33.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:33.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:33.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:33.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.285 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:33.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:33.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:33.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:33.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:33.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:33.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 64 15 104642 104627 102.157 66.75 0.0133608 0.00978463 2024-09-17T08:17:33.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:33.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:33.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:33.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:33.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:33.717 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:17:33.717 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:17:33.717 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:33.717 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[29565]: pgmap v328: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:33.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:17:33.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:17:33.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:33.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:33 smithi079 ceph-mon[40649]: pgmap v328: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:33 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:17:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:33 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:17:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:33 smithi173 ceph-mon[33187]: pgmap v328: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:33.754 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:17:33.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:33.838 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:33.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:33.940 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:33.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:33.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:33.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:34.078 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:34.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:34.118 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:34.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:34.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:34.263 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:34.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:34.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:17:34.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:34.401 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:34.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:34.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:34 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:17:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:34 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:17:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:34 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:17:34.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:34 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:17:34.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:34.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 65 16 105845 105829 101.741 75.125 0.00783315 0.0098244 2024-09-17T08:17:34.541 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:34.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:17:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:17:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:17:34.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:17:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:17:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:17:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:17:34.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:34 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:17:34.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:34.681 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:34.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:34.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:34.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:34.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:34.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:34.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:34.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:34.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:35.059 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:35.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:35.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:35.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:35.227 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:35.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:35.329 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:35.359 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:35.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:35.361 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:35.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:35.498 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:17:35.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:17:35.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:17:35.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:17:35.499 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[29565]: pgmap v329: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:17:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:17:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:17:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:17:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:35 smithi079 ceph-mon[40649]: pgmap v329: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:35.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:35.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:35.525 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 66 16 107011 106995 101.303 72.875 0.00826979 0.00986649 2024-09-17T08:17:35.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:35.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:17:35.640 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:35.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:35.712 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:35 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:17:35.712 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:35 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:17:35.712 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:35 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:17:35.712 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:35 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:17:35.712 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:35 smithi173 ceph-mon[33187]: pgmap v329: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:17:35.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:35.748 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:17:35.781 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:35.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:35.883 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:35.921 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:35.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:35.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:36.000 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:36.000 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:17:36.000 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30828 7.1 7:814ad74a:::benchmark_data_smithi079_83551_object30827:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 305.96763787899999, 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14898619199999999, 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30828 2024-09-17T08:17:36.001 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.002 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029666+0000", 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.003 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029758+0000", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1730000000000004e-05 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029772+0000", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3888e-05 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029779+0000", 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0060000000000003e-06 2024-09-17T08:17:36.004 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029822+0000", 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2682999999999998e-05 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029949+0000", 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127482 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.005 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.030066+0000", 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117027 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.032345+0000", 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022790940000000002 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.006 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035973+0000", 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003627791 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178565+0000", 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14259147699999999 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2923999999999997e-05 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.007 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178655+0000", 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7166000000000003e-05 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30837 7.1 7:84e0e382:::benchmark_data_smithi079_83551_object30836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 305.96010955499997, 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14170434200000001, 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.008 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30837 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:17:36.009 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037195+0000", 2024-09-17T08:17:36.010 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037307+0000", 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111345 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037318+0000", 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1612e-05 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.011 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037326+0000", 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4959999999999999e-06 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037356+0000", 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0233e-05 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037437+0000", 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0385999999999999e-05 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.012 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037519+0000", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2231000000000006e-05 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.038523+0000", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001004158 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.039311+0000", 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078787300000000002 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.013 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178816+0000", 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13950497000000001 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178860+0000", 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4440000000000001e-05 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.014 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178901+0000", 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0961999999999998e-05 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67130 7.1 7:868286e5:::benchmark_data_smithi079_83551_object67129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 282.45064985900001, 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099432551999999993, 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.015 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.016 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.016 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.016 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67130 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:17:36.017 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546657+0000", 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.018 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546655+0000", 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.019 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546716+0000", 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1740000000000002e-05 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.020 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546732+0000", 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6005999999999999e-05 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546741+0000", 2024-09-17T08:17:36.021 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0229999999999992e-06 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546792+0000", 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1205000000000002e-05 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.022 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546929+0000", 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137167 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.547027+0000", 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8185999999999998e-05 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.023 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.548900+0000", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018727170000000001 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.549110+0000", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021050600000000001 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.645997+0000", 2024-09-17T08:17:36.024 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096886776999999993 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.646051+0000", 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3665000000000002e-05 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.646089+0000", 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8324000000000001e-05 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.025 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67361 7.1 7:8095d638:::benchmark_data_smithi079_83551_object67360:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 282.20131952899999, 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.084037299999999995, 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67361 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.026 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795987+0000", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.795986+0000", 2024-09-17T08:17:36.027 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796036+0000", 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0630999999999997e-05 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796046+0000", 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0249e-05 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.028 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796052+0000", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2689999999999999e-06 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796097+0000", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5562999999999997e-05 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796237+0000", 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000139307 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.029 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796396+0000", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000159145 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800418+0000", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0040222030000000002 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800445+0000", 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7274e-05 2024-09-17T08:17:36.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.879940+0000", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.079494422999999995 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.879985+0000", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5290999999999997e-05 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880024+0000", 2024-09-17T08:17:36.031 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9240999999999999e-05 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67364 7.1 7:8e7f7c18:::benchmark_data_smithi079_83551_object67363:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 282.20100904200001, 2024-09-17T08:17:36.032 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.083829862000000005, 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67364 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.033 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796297+0000", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796366+0000", 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9509000000000001e-05 2024-09-17T08:17:36.034 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796372+0000", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0480000000000004e-06 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.796376+0000", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1250000000000003e-06 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798382+0000", 2024-09-17T08:17:36.035 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002005976 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798502+0000", 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119765 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.798645+0000", 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014252899999999999 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.036 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800474+0000", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018297389999999999 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.800512+0000", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7613000000000001e-05 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880060+0000", 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.079547962 2024-09-17T08:17:36.037 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880098+0000", 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7691999999999998e-05 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.880127+0000", 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9541999999999999e-05 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68437 7.1 7:8794a2e1:::benchmark_data_smithi079_83551_object68436:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 281.23528354199999, 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11749947199999999, 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68437 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.039 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762022+0000", 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762153+0000", 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131729 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762168+0000", 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4851999999999999e-05 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762174+0000", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6439999999999997e-06 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762209+0000", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5030999999999997e-05 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762299+0000", 2024-09-17T08:17:36.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0492e-05 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762395+0000", 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5126999999999999e-05 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764253+0000", 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018581089999999999 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764600+0000", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034766899999999999 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879354+0000", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11475331499999999 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879444+0000", 2024-09-17T08:17:36.044 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0832e-05 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879522+0000", 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7851999999999999e-05 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68447 7.1 7:81e8cead:::benchmark_data_smithi079_83551_object68446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:17:36.045 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 281.22912358899998, 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.116018159, 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68447 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768181+0000", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768276+0000", 2024-09-17T08:17:36.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5210999999999994e-05 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768289+0000", 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2994e-05 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768296+0000", 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8510000000000001e-06 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768336+0000", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9638999999999998e-05 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768469+0000", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132881 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768636+0000", 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016691399999999999 2024-09-17T08:17:36.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771127+0000", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002491643 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771774+0000", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064712199999999998 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884106+0000", 2024-09-17T08:17:36.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11233185 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884160+0000", 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3644999999999999e-05 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884201+0000", 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1152000000000002e-05 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.051 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68453 7.1 7:847278dd:::benchmark_data_smithi079_83551_object68452:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 281.225374923, 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.115527932, 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68453 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771930+0000", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772008+0000", 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7849999999999995e-05 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772019+0000", 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1488000000000001e-05 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772025+0000", 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1270000000000002e-06 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772067+0000", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2552000000000002e-05 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772178+0000", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110838 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772272+0000", 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3548999999999995e-05 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774206+0000", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001934727 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774634+0000", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00042775400000000002 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887387+0000", 2024-09-17T08:17:36.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.112753373 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887423+0000", 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5494999999999998e-05 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887459+0000", 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6498999999999997e-05 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:138557 7.1 7:8b1ebd74:::benchmark_data_smithi079_83551_object11157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 217.803366447, 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.074466521999999993, 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 138557 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193940+0000", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193938+0000", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193946+0000", 2024-09-17T08:17:36.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8720000000000002e-06 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193959+0000", 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2288e-05 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.193966+0000", 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6899999999999992e-06 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198079+0000", 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041130380000000003 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198163+0000", 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4127000000000003e-05 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.198250+0000", 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6661000000000003e-05 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:36.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.200929+0000", 2024-09-17T08:17:36.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026791829999999999 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.201219+0000", 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028933099999999998 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268334+0000", 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067115887999999999 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268372+0000", 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7580999999999998e-05 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:58.268406+0000", 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4388999999999999e-05 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183015 7.1 7:881aa8e5:::benchmark_data_smithi079_83551_object55615:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 174.24735889600001, 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110799543, 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183015 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749946+0000", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749953+0000", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749968+0000", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4815e-05 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749975+0000", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7070000000000003e-06 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750019+0000", 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4474000000000001e-05 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750110+0000", 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0523e-05 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750189+0000", 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9081999999999999e-05 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752660+0000", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024708320000000001 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752849+0000", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018942199999999999 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860669+0000", 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10781995900000001 2024-09-17T08:17:36.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860711+0000", 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2150000000000001e-05 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860747+0000", 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5818999999999997e-05 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183025 7.1 7:819884a4:::benchmark_data_smithi079_83551_object55625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 174.23827764399999, 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10716666699999999, 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183025 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759027+0000", 2024-09-17T08:17:36.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759036+0000", 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9169999999999995e-06 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759053+0000", 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6585000000000001e-05 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759062+0000", 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0010000000000007e-06 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759107+0000", 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5754000000000003e-05 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759217+0000", 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110092 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759305+0000", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7967999999999995e-05 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761460+0000", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002154885 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761625+0000", 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000164461 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866070+0000", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.104445353 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866155+0000", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5018000000000005e-05 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866195+0000", 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0274000000000001e-05 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.077 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183184 7.1 7:8813662c:::benchmark_data_smithi079_83551_object55784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:42.011601+0000", 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.98570556300001, 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.080255926000000005, 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183184 2024-09-17T08:17:36.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011601+0000", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011601+0000", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011599+0000", 2024-09-17T08:17:36.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011606+0000", 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9870000000000002e-06 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011619+0000", 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2860999999999999e-05 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011625+0000", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9220000000000002e-06 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011658+0000", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3220999999999999e-05 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011779+0000", 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121463 2024-09-17T08:17:36.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.011917+0000", 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137157 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.016229+0000", 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004312008 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.016386+0000", 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015764800000000001 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.091799+0000", 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.075412224 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.091830+0000", 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1915000000000002e-05 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.091857+0000", 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6282999999999999e-05 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183189 7.1 7:805f3fd2:::benchmark_data_smithi079_83551_object55789:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:42.012494+0000", 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 173.98481234299999, 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.082695517999999996, 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.084 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183189 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012494+0000", 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012494+0000", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012493+0000", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012498+0000", 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0209999999999999e-06 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012511+0000", 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3088000000000001e-05 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.012518+0000", 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8959999999999997e-06 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.014092+0000", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00157383 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.014197+0000", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104638 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.014290+0000", 2024-09-17T08:17:36.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3115000000000005e-05 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.016460+0000", 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021701749999999999 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.016718+0000", 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025861500000000002 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.095117+0000", 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.078398429000000006 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.095152+0000", 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5574999999999997e-05 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:42.095190+0000", 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7085999999999997e-05 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:185617 7.1 7:8d4bcb84:::benchmark_data_smithi079_83551_object58217:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:44.534578+0000", 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 171.46272865500001, 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.072593065999999998, 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.091 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 185617 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534578+0000", 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534578+0000", 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534576+0000", 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534583+0000", 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7460000000000003e-06 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534597+0000", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4487e-05 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534604+0000", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1250000000000004e-06 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534759+0000", 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000154695 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.534892+0000", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013315700000000001 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.535013+0000", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012119400000000001 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.537405+0000", 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00239156 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.537630+0000", 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000224775 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.607079+0000", 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069449216999999994 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.607132+0000", 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3167000000000001e-05 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:44.607171+0000", 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8640999999999998e-05 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:187752 7.1 7:843ca67d:::benchmark_data_smithi079_83551_object60352:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-17T08:17:36.097 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:46.582635+0000", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 169.414671822, 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.075828443999999995, 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 187752 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582635+0000", 2024-09-17T08:17:36.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582635+0000", 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582633+0000", 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582640+0000", 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6290000000000004e-06 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582654+0000", 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4961e-05 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582662+0000", 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2300000000000002e-06 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582703+0000", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1641000000000003e-05 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582814+0000", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111148 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.582935+0000", 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012060799999999999 2024-09-17T08:17:36.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.585452+0000", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025173259999999999 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.585666+0000", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000213205 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.658376+0000", 2024-09-17T08:17:36.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.072709941 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.658426+0000", 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9985000000000003e-05 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:46.658463+0000", 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7419999999999997e-05 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:245266 7.1 7:8c9c773e:::benchmark_data_smithi079_83551_object117866:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:44.899343+0000", 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 111.097963202, 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.090465552000000005, 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 245266 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899343+0000", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899343+0000", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899342+0000", 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899348+0000", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2489999999999998e-06 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899357+0000", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5900000000000008e-06 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899362+0000", 2024-09-17T08:17:36.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9779999999999996e-06 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899399+0000", 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7580000000000003e-05 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899483+0000", 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3677000000000005e-05 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:17:36.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899556+0000", 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2594000000000003e-05 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.901757+0000", 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022018179999999999 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.902009+0000", 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025115400000000001 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.989749+0000", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.087739981999999994 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.989781+0000", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2555999999999998e-05 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.989809+0000", 2024-09-17T08:17:36.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7611999999999999e-05 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:31639 8.2 8:4c490710:::benchmark_data_smithi079_115040_object31638:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:41.354079+0000", 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 54.643227046, 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.080935978000000006, 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.110 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 31639 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.354079+0000", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.354079+0000", 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.354078+0000", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.354111+0000", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3040000000000002e-05 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.397004+0000", 2024-09-17T08:17:36.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042893222000000002 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.397014+0000", 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6649999999999998e-06 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.397251+0000", 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00023670200000000001 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.397481+0000", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000230106 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.397617+0000", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000135678 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.434916+0000", 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037299019000000003 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.434924+0000", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4570000000000005e-06 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.434950+0000", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5996e-05 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.434980+0000", 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9903000000000002e-05 2024-09-17T08:17:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:41.435015+0000", 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.536e-05 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35137 8.2 8:468fa297:::benchmark_data_smithi079_115040_object35136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.654059513999997, 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.131430191, 2024-09-17T08:17:36.116 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35137 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343245+0000", 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343349+0000", 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103587 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343366+0000", 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7232999999999999e-05 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343373+0000", 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3030000000000002e-06 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343441+0000", 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7742000000000003e-05 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343563+0000", 2024-09-17T08:17:36.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012195499999999999 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343696+0000", 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132664 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345071+0000", 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013757260000000001 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345717+0000", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064516599999999997 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474616+0000", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12889890800000001 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474651+0000", 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5364000000000002e-05 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474677+0000", 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6166e-05 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35143 8.2 8:48b7532d:::benchmark_data_smithi079_115040_object35142:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 52.651017553999999, 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13410802799999999, 2024-09-17T08:17:36.123 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35143 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346288+0000", 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346342+0000", 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3986000000000001e-05 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346353+0000", 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1019000000000001e-05 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346361+0000", 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7889999999999995e-06 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346403+0000", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2738999999999999e-05 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346519+0000", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115482 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346671+0000", 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015182600000000001 2024-09-17T08:17:36.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.347889+0000", 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012181220000000001 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.349481+0000", 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001591631 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480320+0000", 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.130839172 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480362+0000", 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1878000000000001e-05 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480397+0000", 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5224e-05 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.129 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:36507 8.2 8:46403afa:::benchmark_data_smithi079_115040_object36506:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 51.848772381000003, 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14247363399999999, 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 36507 2024-09-17T08:17:36.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:36.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148533+0000", 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148580+0000", 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7367999999999998e-05 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148593+0000", 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2622e-05 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148598+0000", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6359999999999999e-06 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148634+0000", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5598e-05 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148725+0000", 2024-09-17T08:17:36.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0792999999999996e-05 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148820+0000", 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5533999999999997e-05 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.149972+0000", 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001151727 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.150108+0000", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013644899999999999 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290944+0000", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14083580900000001 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290980+0000", 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5593000000000003e-05 2024-09-17T08:17:36.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.291008+0000", 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7813e-05 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:36.136 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:36.151 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:36.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:36.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:36.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:36.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:36.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:36.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:36.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:36.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:36.524 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 67 16 108197 108181 100.897 74.125 0.0157478 0.00990654 2024-09-17T08:17:36.560 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:36.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:36.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:36.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:36.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:17:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:17:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:17:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:17:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:36 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:17:36.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:36.831 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:36.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:17:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:17:36.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:17:36.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:36.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:36 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:17:36.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:17:36.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:36.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:36.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:36.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:37.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:37.126 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:37.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:37.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:37.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.281 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:37.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:37.290 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:37.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:37.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:37.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:37.507 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-17T08:17:35.353938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6798579990,"num_objects":103925,"num_object_clones":0,"num_object_copies":311775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103925,"num_whiteouts":0,"num_read":1043,"num_read_kb":889,"num_write":104338,"num_write_kb":6641115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":1776389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84499,"ondisk_log_size":84499,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21496400,"kb_used_data":19761028,"kb_used_omap":12,"kb_used_meta":1735283,"kb_avail":728464816,"statfs":{"total":767960285184,"available":745947971584,"internally_reserved":0,"allocated":20235292672,"data_stored":20232958890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":1776930360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,9,4,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786038784,"num_objects":11994,"num_object_clones":0,"num_object_copies":35982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11994,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11994,"num_write_kb":767616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003183"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":153,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.438318+0000","last_change":"2024-09-17T08:14:03.327276+0000","last_active":"2024-09-17T08:17:30.438318+0000","last_peered":"2024-09-17T08:17:30.438318+0000","last_clean":"2024-09-17T08:17:30.438318+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:17:30.438318+0000","last_undegraded":"2024-09-17T08:17:30.438318+0000","last_fullsized":"2024-09-17T08:17:30.438318+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:03.327169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:03.327169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013886001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":181,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.308113+0000","last_change":"2024-09-17T08:16:42.337270+0000","last_active":"2024-09-17T08:17:23.308113+0000","last_peered":"2024-09-17T08:17:23.308113+0000","last_clean":"2024-09-17T08:17:23.308113+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:17:23.308113+0000","last_undegraded":"2024-09-17T08:17:23.308113+0000","last_fullsized":"2024-09-17T08:17:23.308113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.337157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:08.710909+0000","scrub_duration":0.001642129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":182,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312531+0000","last_change":"2024-09-17T08:14:56.372401+0000","last_active":"2024-09-17T08:17:23.312531+0000","last_peered":"2024-09-17T08:17:23.312531+0000","last_clean":"2024-09-17T08:17:23.312531+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:17:23.312531+0000","last_undegraded":"2024-09-17T08:17:23.312531+0000","last_fullsized":"2024-09-17T08:17:23.312531+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:14:56.372270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:56.372270+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010005975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":184,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301541+0000","last_change":"2024-09-17T08:15:52.984443+0000","last_active":"2024-09-17T08:17:23.301541+0000","last_peered":"2024-09-17T08:17:23.301541+0000","last_clean":"2024-09-17T08:17:23.301541+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:17:23.301541+0000","last_undegraded":"2024-09-17T08:17:23.301541+0000","last_fullsized":"2024-09-17T08:17:23.301541+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.984337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.984337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030030090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":159,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313926+0000","last_change":"2024-09-17T08:15:43.592053+0000","last_active":"2024-09-17T08:17:23.313926+0000","last_peered":"2024-09-17T08:17:23.313926+0000","last_clean":"2024-09-17T08:17:23.313926+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:17:23.313926+0000","last_undegraded":"2024-09-17T08:17:23.313926+0000","last_fullsized":"2024-09-17T08:17:23.313926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:15:43.591667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013131347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":264,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301600+0000","last_change":"2024-09-17T08:16:35.827557+0000","last_active":"2024-09-17T08:17:23.301600+0000","last_peered":"2024-09-17T08:17:23.301600+0000","last_clean":"2024-09-17T08:17:23.301600+0000","last_became_active":"2024-09-17T08:16:33.326106+0000","last_became_peered":"2024-09-17T08:16:33.326106+0000","last_unstale":"2024-09-17T08:17:23.301600+0000","last_undegraded":"2024-09-17T08:17:23.301600+0000","last_fullsized":"2024-09-17T08:17:23.301600+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:16:35.827443+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.827443+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:39.548050+0000","scrub_duration":0.0033149960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314793+0000","last_change":"2024-09-17T08:16:44.335176+0000","last_active":"2024-09-17T08:17:23.314793+0000","last_peered":"2024-09-17T08:17:23.314793+0000","last_clean":"2024-09-17T08:17:23.314793+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:17:23.314793+0000","last_undegraded":"2024-09-17T08:17:23.314793+0000","last_fullsized":"2024-09-17T08:17:23.314793+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.335057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.335057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:49.556092+0000","scrub_duration":0.0065690419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":193,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.326180+0000","last_change":"2024-09-17T08:16:48.900627+0000","last_active":"2024-09-17T08:17:27.326180+0000","last_peered":"2024-09-17T08:17:27.326180+0000","last_clean":"2024-09-17T08:17:27.326180+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:17:27.326180+0000","last_undegraded":"2024-09-17T08:17:27.326180+0000","last_fullsized":"2024-09-17T08:17:27.326180+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.900525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.943918+0000","scrub_duration":0.0033962240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":170,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.308926+0000","last_change":"2024-09-17T08:16:33.826157+0000","last_active":"2024-09-17T08:17:27.308926+0000","last_peered":"2024-09-17T08:17:27.308926+0000","last_clean":"2024-09-17T08:17:27.308926+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:17:27.308926+0000","last_undegraded":"2024-09-17T08:17:27.308926+0000","last_fullsized":"2024-09-17T08:17:27.308926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.826006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.826006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.282923+0000","scrub_duration":0.0026989420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":210,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.326233+0000","last_change":"2024-09-17T08:15:31.963456+0000","last_active":"2024-09-17T08:17:27.326233+0000","last_peered":"2024-09-17T08:17:27.326233+0000","last_clean":"2024-09-17T08:17:27.326233+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:17:27.326233+0000","last_undegraded":"2024-09-17T08:17:27.326233+0000","last_fullsized":"2024-09-17T08:17:27.326233+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:15:31.963328+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:15:31.963328+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040865249999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":195,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.308798+0000","last_change":"2024-09-17T08:16:49.012348+0000","last_active":"2024-09-17T08:17:27.308798+0000","last_peered":"2024-09-17T08:17:27.308798+0000","last_clean":"2024-09-17T08:17:27.308798+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:17:27.308798+0000","last_undegraded":"2024-09-17T08:17:27.308798+0000","last_fullsized":"2024-09-17T08:17:27.308798+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.012248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.718200+0000","scrub_duration":0.0070975750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314571+0000","last_change":"2024-09-17T08:16:23.065070+0000","last_active":"2024-09-17T08:17:23.314571+0000","last_peered":"2024-09-17T08:17:23.314571+0000","last_clean":"2024-09-17T08:17:23.314571+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:17:23.314571+0000","last_undegraded":"2024-09-17T08:17:23.314571+0000","last_fullsized":"2024-09-17T08:17:23.314571+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.064993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.064993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015683839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":181,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.308870+0000","last_change":"2024-09-17T08:17:01.908852+0000","last_active":"2024-09-17T08:17:27.308870+0000","last_peered":"2024-09-17T08:17:27.308870+0000","last_clean":"2024-09-17T08:17:27.308870+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:17:27.308870+0000","last_undegraded":"2024-09-17T08:17:27.308870+0000","last_fullsized":"2024-09-17T08:17:27.308870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.908732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.507561+0000","scrub_duration":0.0085101169999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":195,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.326205+0000","last_change":"2024-09-17T08:16:46.894268+0000","last_active":"2024-09-17T08:17:27.326205+0000","last_peered":"2024-09-17T08:17:27.326205+0000","last_clean":"2024-09-17T08:17:27.326205+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:17:27.326205+0000","last_undegraded":"2024-09-17T08:17:27.326205+0000","last_fullsized":"2024-09-17T08:17:27.326205+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:16:46.894134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.894134+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.434020+0000","scrub_duration":0.0053467080000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":69,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429140+0000","last_change":"2024-09-17T08:15:54.069982+0000","last_active":"2024-09-17T08:17:30.429140+0000","last_peered":"2024-09-17T08:17:30.429140+0000","last_clean":"2024-09-17T08:17:30.429140+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:17:30.429140+0000","last_undegraded":"2024-09-17T08:17:30.429140+0000","last_fullsized":"2024-09-17T08:17:30.429140+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.069886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.069886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001445294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":193,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.420954+0000","last_change":"2024-09-17T08:17:31.421060+0000","last_active":"2024-09-17T08:17:31.420954+0000","last_peered":"2024-09-17T08:17:31.420954+0000","last_clean":"2024-09-17T08:17:31.420954+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:17:31.420954+0000","last_undegraded":"2024-09-17T08:17:31.420954+0000","last_fullsized":"2024-09-17T08:17:31.420954+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.420934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.420934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.381148+0000","scrub_duration":0.0086368369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"189'6490","reported_seq":6523,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.575696+0000","last_change":"2024-09-17T08:16:16.645652+0000","last_active":"2024-09-17T08:17:32.575696+0000","last_peered":"2024-09-17T08:17:32.575696+0000","last_clean":"2024-09-17T08:17:32.575696+0000","last_became_active":"2024-09-17T08:16:16.645277+0000","last_became_peered":"2024-09-17T08:16:16.645277+0000","last_unstale":"2024-09-17T08:17:32.575696+0000","last_undegraded":"2024-09-17T08:17:32.575696+0000","last_fullsized":"2024-09-17T08:17:32.575696+0000","mapping_epoch":165,"log_start":"173'1400","ondisk_log_start":"173'1400","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5090,"log_dups_size":0,"ondisk_log_size":5090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425328640,"num_objects":6490,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6490,"num_write_kb":415360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":250,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302353+0000","last_change":"2024-09-17T08:16:26.873516+0000","last_active":"2024-09-17T08:17:23.302353+0000","last_peered":"2024-09-17T08:17:23.302353+0000","last_clean":"2024-09-17T08:17:23.302353+0000","last_became_active":"2024-09-17T08:15:29.362201+0000","last_became_peered":"2024-09-17T08:15:29.362201+0000","last_unstale":"2024-09-17T08:17:23.302353+0000","last_undegraded":"2024-09-17T08:17:23.302353+0000","last_fullsized":"2024-09-17T08:17:23.302353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:16:26.873402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.873402+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:39.799533+0000","scrub_duration":0.002741856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":170,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325113+0000","last_change":"2024-09-17T08:15:45.009419+0000","last_active":"2024-09-17T08:17:23.325113+0000","last_peered":"2024-09-17T08:17:23.325113+0000","last_clean":"2024-09-17T08:17:23.325113+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:17:23.325113+0000","last_undegraded":"2024-09-17T08:17:23.325113+0000","last_fullsized":"2024-09-17T08:17:23.325113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:45.009304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:45.009304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015103079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314565+0000","last_change":"2024-09-17T08:15:48.924108+0000","last_active":"2024-09-17T08:17:23.314565+0000","last_peered":"2024-09-17T08:17:23.314565+0000","last_clean":"2024-09-17T08:17:23.314565+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:17:23.314565+0000","last_undegraded":"2024-09-17T08:17:23.314565+0000","last_fullsized":"2024-09-17T08:17:23.314565+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.923947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.923947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039645160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324540+0000","last_change":"2024-09-17T08:16:26.262129+0000","last_active":"2024-09-17T08:17:23.324540+0000","last_peered":"2024-09-17T08:17:23.324540+0000","last_clean":"2024-09-17T08:17:23.324540+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:17:23.324540+0000","last_undegraded":"2024-09-17T08:17:23.324540+0000","last_fullsized":"2024-09-17T08:17:23.324540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:26.262049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.262049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:36.985195+0000","scrub_duration":0.001547122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"189'6468","reported_seq":6501,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.767940+0000","last_change":"2024-09-17T08:16:16.648796+0000","last_active":"2024-09-17T08:17:32.767940+0000","last_peered":"2024-09-17T08:17:32.767940+0000","last_clean":"2024-09-17T08:17:32.767940+0000","last_became_active":"2024-09-17T08:16:16.648526+0000","last_became_peered":"2024-09-17T08:16:16.648526+0000","last_unstale":"2024-09-17T08:17:32.767940+0000","last_undegraded":"2024-09-17T08:17:32.767940+0000","last_fullsized":"2024-09-17T08:17:32.767940+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":423886848,"num_objects":6468,"num_object_clones":0,"num_object_copies":19404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6468,"num_write_kb":413952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":194,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302314+0000","last_change":"2024-09-17T08:16:56.688647+0000","last_active":"2024-09-17T08:17:23.302314+0000","last_peered":"2024-09-17T08:17:23.302314+0000","last_clean":"2024-09-17T08:17:23.302314+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:17:23.302314+0000","last_undegraded":"2024-09-17T08:17:23.302314+0000","last_fullsized":"2024-09-17T08:17:23.302314+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:56.688373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.688373+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.791670+0000","scrub_duration":0.0062131820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.320141+0000","last_change":"2024-09-17T08:15:29.377784+0000","last_active":"2024-09-17T08:17:23.320141+0000","last_peered":"2024-09-17T08:17:23.320141+0000","last_clean":"2024-09-17T08:17:23.320141+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:17:23.320141+0000","last_undegraded":"2024-09-17T08:17:23.320141+0000","last_fullsized":"2024-09-17T08:17:23.320141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:51.813043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:51.813043+0000","last_clean_scrub_stamp":"2024-09-17T08:14:51.813043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":277,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.915875+0000","last_change":"2024-09-17T08:15:49.311596+0000","last_active":"2024-09-17T08:17:27.915875+0000","last_peered":"2024-09-17T08:17:27.915875+0000","last_clean":"2024-09-17T08:17:27.915875+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:17:27.915875+0000","last_undegraded":"2024-09-17T08:17:27.915875+0000","last_fullsized":"2024-09-17T08:17:27.915875+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:15:49.311199+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018142607000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":182,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302483+0000","last_change":"2024-09-17T08:16:30.856483+0000","last_active":"2024-09-17T08:17:23.302483+0000","last_peered":"2024-09-17T08:17:23.302483+0000","last_clean":"2024-09-17T08:17:23.302483+0000","last_became_active":"2024-09-17T08:15:29.358693+0000","last_became_peered":"2024-09-17T08:15:29.358693+0000","last_unstale":"2024-09-17T08:17:23.302483+0000","last_undegraded":"2024-09-17T08:17:23.302483+0000","last_fullsized":"2024-09-17T08:17:23.302483+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:30.856368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.856368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.067757+0000","scrub_duration":0.0044042839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"189'6407","reported_seq":6440,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.545183+0000","last_change":"2024-09-17T08:16:16.648125+0000","last_active":"2024-09-17T08:17:32.545183+0000","last_peered":"2024-09-17T08:17:32.545183+0000","last_clean":"2024-09-17T08:17:32.545183+0000","last_became_active":"2024-09-17T08:16:16.647816+0000","last_became_peered":"2024-09-17T08:16:16.647816+0000","last_unstale":"2024-09-17T08:17:32.545183+0000","last_undegraded":"2024-09-17T08:17:32.545183+0000","last_fullsized":"2024-09-17T08:17:32.545183+0000","mapping_epoch":165,"log_start":"173'1400","ondisk_log_start":"173'1400","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419889152,"num_objects":6407,"num_object_clones":0,"num_object_copies":19221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":410048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":54,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324384+0000","last_change":"2024-09-17T08:16:53.116193+0000","last_active":"2024-09-17T08:17:23.324384+0000","last_peered":"2024-09-17T08:17:23.324384+0000","last_clean":"2024-09-17T08:17:23.324384+0000","last_became_active":"2024-09-17T08:16:33.328511+0000","last_became_peered":"2024-09-17T08:16:33.328511+0000","last_unstale":"2024-09-17T08:17:23.324384+0000","last_undegraded":"2024-09-17T08:17:23.324384+0000","last_fullsized":"2024-09-17T08:17:23.324384+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:16:53.115891+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:16:53.115891+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.168215+0000","scrub_duration":0.0043675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.316355+0000","last_change":"2024-09-17T08:16:40.302468+0000","last_active":"2024-09-17T08:17:23.316355+0000","last_peered":"2024-09-17T08:17:23.316355+0000","last_clean":"2024-09-17T08:17:23.316355+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:17:23.316355+0000","last_undegraded":"2024-09-17T08:17:23.316355+0000","last_fullsized":"2024-09-17T08:17:23.316355+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.302235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.302235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.381029+0000","scrub_duration":0.0071342690000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":265,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.913044+0000","last_change":"2024-09-17T08:15:40.042590+0000","last_active":"2024-09-17T08:17:27.913044+0000","last_peered":"2024-09-17T08:17:27.913044+0000","last_clean":"2024-09-17T08:17:27.913044+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:17:27.913044+0000","last_undegraded":"2024-09-17T08:17:27.913044+0000","last_fullsized":"2024-09-17T08:17:27.913044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:15:40.042507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:40.042507+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048815059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315501+0000","last_change":"2024-09-17T08:16:56.421870+0000","last_active":"2024-09-17T08:17:23.315501+0000","last_peered":"2024-09-17T08:17:23.315501+0000","last_clean":"2024-09-17T08:17:23.315501+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:17:23.315501+0000","last_undegraded":"2024-09-17T08:17:23.315501+0000","last_fullsized":"2024-09-17T08:17:23.315501+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:56.421736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.421736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.576854+0000","scrub_duration":0.0048071169999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"189'6574","reported_seq":6607,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.317816+0000","last_change":"2024-09-17T08:16:16.649421+0000","last_active":"2024-09-17T08:17:32.317816+0000","last_peered":"2024-09-17T08:17:32.317816+0000","last_clean":"2024-09-17T08:17:32.317816+0000","last_became_active":"2024-09-17T08:16:16.648969+0000","last_became_peered":"2024-09-17T08:16:16.648969+0000","last_unstale":"2024-09-17T08:17:32.317816+0000","last_undegraded":"2024-09-17T08:17:32.317816+0000","last_fullsized":"2024-09-17T08:17:32.317816+0000","mapping_epoch":165,"log_start":"173'1300","ondisk_log_start":"173'1300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5274,"log_dups_size":0,"ondisk_log_size":5274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430833664,"num_objects":6574,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6574,"num_write_kb":420736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"156'16","reported_seq":234,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302707+0000","last_change":"2024-09-17T08:15:44.029509+0000","last_active":"2024-09-17T08:17:23.302707+0000","last_peered":"2024-09-17T08:17:23.302707+0000","last_clean":"2024-09-17T08:17:23.302707+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:17:23.302707+0000","last_undegraded":"2024-09-17T08:17:23.302707+0000","last_fullsized":"2024-09-17T08:17:23.302707+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'16","last_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.029355+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011380917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":188,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324332+0000","last_change":"2024-09-17T08:16:37.745942+0000","last_active":"2024-09-17T08:17:23.324332+0000","last_peered":"2024-09-17T08:17:23.324332+0000","last_clean":"2024-09-17T08:17:23.324332+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:17:23.324332+0000","last_undegraded":"2024-09-17T08:17:23.324332+0000","last_fullsized":"2024-09-17T08:17:23.324332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.745837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.563608+0000","scrub_duration":0.0044495799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":70,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301988+0000","last_change":"2024-09-17T08:16:25.867376+0000","last_active":"2024-09-17T08:17:23.301988+0000","last_peered":"2024-09-17T08:17:23.301988+0000","last_clean":"2024-09-17T08:17:23.301988+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:17:23.301988+0000","last_undegraded":"2024-09-17T08:17:23.301988+0000","last_fullsized":"2024-09-17T08:17:23.301988+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:25.867267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:25.867267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.318499+0000","scrub_duration":0.0015224139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312470+0000","last_change":"2024-09-17T08:14:48.431225+0000","last_active":"2024-09-17T08:17:23.312470+0000","last_peered":"2024-09-17T08:17:23.312470+0000","last_clean":"2024-09-17T08:17:23.312470+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:17:23.312470+0000","last_undegraded":"2024-09-17T08:17:23.312470+0000","last_fullsized":"2024-09-17T08:17:23.312470+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:48.429206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:14:48.429206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080393310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"189'6505","reported_seq":6538,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.298288+0000","last_change":"2024-09-17T08:16:16.649228+0000","last_active":"2024-09-17T08:17:32.298288+0000","last_peered":"2024-09-17T08:17:32.298288+0000","last_clean":"2024-09-17T08:17:32.298288+0000","last_became_active":"2024-09-17T08:16:16.648937+0000","last_became_peered":"2024-09-17T08:16:16.648937+0000","last_unstale":"2024-09-17T08:17:32.298288+0000","last_undegraded":"2024-09-17T08:17:32.298288+0000","last_fullsized":"2024-09-17T08:17:32.298288+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5005,"log_dups_size":0,"ondisk_log_size":5005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426311680,"num_objects":6505,"num_object_clones":0,"num_object_copies":19515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6505,"num_write_kb":416320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":189,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301754+0000","last_change":"2024-09-17T08:16:27.874195+0000","last_active":"2024-09-17T08:17:23.301754+0000","last_peered":"2024-09-17T08:17:23.301754+0000","last_clean":"2024-09-17T08:17:23.301754+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:17:23.301754+0000","last_undegraded":"2024-09-17T08:17:23.301754+0000","last_fullsized":"2024-09-17T08:17:23.301754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:16:27.874083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.874083+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:45.441731+0000","scrub_duration":0.0022658510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325268+0000","last_change":"2024-09-17T08:16:42.793996+0000","last_active":"2024-09-17T08:17:23.325268+0000","last_peered":"2024-09-17T08:17:23.325268+0000","last_clean":"2024-09-17T08:17:23.325268+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:17:23.325268+0000","last_undegraded":"2024-09-17T08:17:23.325268+0000","last_fullsized":"2024-09-17T08:17:23.325268+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.793906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.082126+0000","scrub_duration":0.0013736460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":297,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.913629+0000","last_change":"2024-09-17T08:16:48.678053+0000","last_active":"2024-09-17T08:17:27.913629+0000","last_peered":"2024-09-17T08:17:27.913629+0000","last_clean":"2024-09-17T08:17:27.913629+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:17:27.913629+0000","last_undegraded":"2024-09-17T08:17:27.913629+0000","last_fullsized":"2024-09-17T08:17:27.913629+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.677913+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.360519+0000","scrub_duration":0.032798792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324907+0000","last_change":"2024-09-17T08:16:47.964627+0000","last_active":"2024-09-17T08:17:23.324907+0000","last_peered":"2024-09-17T08:17:23.324907+0000","last_clean":"2024-09-17T08:17:23.324907+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:17:23.324907+0000","last_undegraded":"2024-09-17T08:17:23.324907+0000","last_fullsized":"2024-09-17T08:17:23.324907+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.964546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.603179+0000","scrub_duration":0.0015627130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"189'6393","reported_seq":6426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.060005+0000","last_change":"2024-09-17T08:16:16.404499+0000","last_active":"2024-09-17T08:17:33.060005+0000","last_peered":"2024-09-17T08:17:33.060005+0000","last_clean":"2024-09-17T08:17:33.060005+0000","last_became_active":"2024-09-17T08:16:16.404227+0000","last_became_peered":"2024-09-17T08:16:16.404227+0000","last_unstale":"2024-09-17T08:17:33.060005+0000","last_undegraded":"2024-09-17T08:17:33.060005+0000","last_fullsized":"2024-09-17T08:17:33.060005+0000","mapping_epoch":165,"log_start":"172'700","ondisk_log_start":"172'700","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418971648,"num_objects":6393,"num_object_clones":0,"num_object_copies":19179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6393,"num_write_kb":409152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":200,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313074+0000","last_change":"2024-09-17T08:15:52.770015+0000","last_active":"2024-09-17T08:17:23.313074+0000","last_peered":"2024-09-17T08:17:23.313074+0000","last_clean":"2024-09-17T08:17:23.313074+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:17:23.313074+0000","last_undegraded":"2024-09-17T08:17:23.313074+0000","last_fullsized":"2024-09-17T08:17:23.313074+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:15:52.769743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.769743+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.023040108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.317292+0000","last_change":"2024-09-17T08:17:02.482908+0000","last_active":"2024-09-17T08:17:23.317292+0000","last_peered":"2024-09-17T08:17:23.317292+0000","last_clean":"2024-09-17T08:17:23.317292+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:17:23.317292+0000","last_undegraded":"2024-09-17T08:17:23.317292+0000","last_fullsized":"2024-09-17T08:17:23.317292+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:02.482796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.482796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:24.312024+0000","scrub_duration":0.006488972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":166,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301677+0000","last_change":"2024-09-17T08:15:48.938396+0000","last_active":"2024-09-17T08:17:23.301677+0000","last_peered":"2024-09-17T08:17:23.301677+0000","last_clean":"2024-09-17T08:17:23.301677+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:17:23.301677+0000","last_undegraded":"2024-09-17T08:17:23.301677+0000","last_fullsized":"2024-09-17T08:17:23.301677+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.938247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.938247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00524202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301970+0000","last_change":"2024-09-17T08:15:35.024756+0000","last_active":"2024-09-17T08:17:23.301970+0000","last_peered":"2024-09-17T08:17:23.301970+0000","last_clean":"2024-09-17T08:17:23.301970+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:17:23.301970+0000","last_undegraded":"2024-09-17T08:17:23.301970+0000","last_fullsized":"2024-09-17T08:17:23.301970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:35.024623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:35.024623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010817126999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"189'6425","reported_seq":6458,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.761623+0000","last_change":"2024-09-17T08:16:16.647250+0000","last_active":"2024-09-17T08:17:32.761623+0000","last_peered":"2024-09-17T08:17:32.761623+0000","last_clean":"2024-09-17T08:17:32.761623+0000","last_became_active":"2024-09-17T08:16:16.646899+0000","last_became_peered":"2024-09-17T08:16:16.646899+0000","last_unstale":"2024-09-17T08:17:32.761623+0000","last_undegraded":"2024-09-17T08:17:32.761623+0000","last_fullsized":"2024-09-17T08:17:32.761623+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4925,"log_dups_size":0,"ondisk_log_size":4925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421068800,"num_objects":6425,"num_object_clones":0,"num_object_copies":19275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":411200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":215,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.308403+0000","last_change":"2024-09-17T08:16:48.235582+0000","last_active":"2024-09-17T08:17:23.308403+0000","last_peered":"2024-09-17T08:17:23.308403+0000","last_clean":"2024-09-17T08:17:23.308403+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:17:23.308403+0000","last_undegraded":"2024-09-17T08:17:23.308403+0000","last_fullsized":"2024-09-17T08:17:23.308403+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:16:48.235424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.235424+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.405506+0000","scrub_duration":0.0061761419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325828+0000","last_change":"2024-09-17T08:15:35.039655+0000","last_active":"2024-09-17T08:17:23.325828+0000","last_peered":"2024-09-17T08:17:23.325828+0000","last_clean":"2024-09-17T08:17:23.325828+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:17:23.325828+0000","last_undegraded":"2024-09-17T08:17:23.325828+0000","last_fullsized":"2024-09-17T08:17:23.325828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:15:35.039547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:35.039547+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049775519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":122,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.307105+0000","last_change":"2024-09-17T08:15:46.091387+0000","last_active":"2024-09-17T08:17:23.307105+0000","last_peered":"2024-09-17T08:17:23.307105+0000","last_clean":"2024-09-17T08:17:23.307105+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:17:23.307105+0000","last_undegraded":"2024-09-17T08:17:23.307105+0000","last_fullsized":"2024-09-17T08:17:23.307105+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:46.091278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:46.091278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020828919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315066+0000","last_change":"2024-09-17T08:15:32.989425+0000","last_active":"2024-09-17T08:17:23.315066+0000","last_peered":"2024-09-17T08:17:23.315066+0000","last_clean":"2024-09-17T08:17:23.315066+0000","last_became_active":"2024-09-17T08:15:29.372791+0000","last_became_peered":"2024-09-17T08:15:29.372791+0000","last_unstale":"2024-09-17T08:17:23.315066+0000","last_undegraded":"2024-09-17T08:17:23.315066+0000","last_fullsized":"2024-09-17T08:17:23.315066+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:32.986201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:32.986201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0116349,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"189'6434","reported_seq":6467,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.065250+0000","last_change":"2024-09-17T08:16:16.645635+0000","last_active":"2024-09-17T08:17:33.065250+0000","last_peered":"2024-09-17T08:17:33.065250+0000","last_clean":"2024-09-17T08:17:33.065250+0000","last_became_active":"2024-09-17T08:16:16.645315+0000","last_became_peered":"2024-09-17T08:16:16.645315+0000","last_unstale":"2024-09-17T08:17:33.065250+0000","last_undegraded":"2024-09-17T08:17:33.065250+0000","last_fullsized":"2024-09-17T08:17:33.065250+0000","mapping_epoch":165,"log_start":"172'700","ondisk_log_start":"172'700","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5734,"log_dups_size":0,"ondisk_log_size":5734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421658624,"num_objects":6434,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":411776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"156'15","reported_seq":76,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.319883+0000","last_change":"2024-09-17T08:15:44.010651+0000","last_active":"2024-09-17T08:17:23.319883+0000","last_peered":"2024-09-17T08:17:23.319883+0000","last_clean":"2024-09-17T08:17:23.319883+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:17:23.319883+0000","last_undegraded":"2024-09-17T08:17:23.319883+0000","last_fullsized":"2024-09-17T08:17:23.319883+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'15","last_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.010126+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012291415999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315005+0000","last_change":"2024-09-17T08:15:29.396166+0000","last_active":"2024-09-17T08:17:23.315005+0000","last_peered":"2024-09-17T08:17:23.315005+0000","last_clean":"2024-09-17T08:17:23.315005+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:17:23.315005+0000","last_undegraded":"2024-09-17T08:17:23.315005+0000","last_fullsized":"2024-09-17T08:17:23.315005+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:48.820434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:48.820434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429007+0000","last_change":"2024-09-17T08:17:00.237348+0000","last_active":"2024-09-17T08:17:30.429007+0000","last_peered":"2024-09-17T08:17:30.429007+0000","last_clean":"2024-09-17T08:17:30.429007+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:17:30.429007+0000","last_undegraded":"2024-09-17T08:17:30.429007+0000","last_fullsized":"2024-09-17T08:17:30.429007+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:17:00.237229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.237229+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:10.239107+0000","scrub_duration":0.004406877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.306906+0000","last_change":"2024-09-17T08:16:35.335671+0000","last_active":"2024-09-17T08:17:23.306906+0000","last_peered":"2024-09-17T08:17:23.306906+0000","last_clean":"2024-09-17T08:17:23.306906+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:17:23.306906+0000","last_undegraded":"2024-09-17T08:17:23.306906+0000","last_fullsized":"2024-09-17T08:17:23.306906+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.335551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.335551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.895726+0000","scrub_duration":0.007005336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"189'6521","reported_seq":6554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.716216+0000","last_change":"2024-09-17T08:16:16.648773+0000","last_active":"2024-09-17T08:17:32.716216+0000","last_peered":"2024-09-17T08:17:32.716216+0000","last_clean":"2024-09-17T08:17:32.716216+0000","last_became_active":"2024-09-17T08:16:16.648491+0000","last_became_peered":"2024-09-17T08:16:16.648491+0000","last_unstale":"2024-09-17T08:17:32.716216+0000","last_undegraded":"2024-09-17T08:17:32.716216+0000","last_fullsized":"2024-09-17T08:17:32.716216+0000","mapping_epoch":165,"log_start":"174'1600","ondisk_log_start":"174'1600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:44.077296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427360256,"num_objects":6521,"num_object_clones":0,"num_object_copies":19563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6521,"num_write_kb":417344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"152'30","reported_seq":95,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.912871+0000","last_change":"2024-09-17T08:15:29.437316+0000","last_active":"2024-09-17T08:17:27.912871+0000","last_peered":"2024-09-17T08:17:27.912871+0000","last_clean":"2024-09-17T08:17:27.912871+0000","last_became_active":"2024-09-17T08:15:29.385461+0000","last_became_peered":"2024-09-17T08:15:29.385461+0000","last_unstale":"2024-09-17T08:17:27.912871+0000","last_undegraded":"2024-09-17T08:17:27.912871+0000","last_fullsized":"2024-09-17T08:17:27.912871+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'28","last_scrub_stamp":"2024-09-17T08:14:49.845016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:49.845016+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.320016+0000","last_change":"2024-09-17T08:15:53.067473+0000","last_active":"2024-09-17T08:17:23.320016+0000","last_peered":"2024-09-17T08:17:23.320016+0000","last_clean":"2024-09-17T08:17:23.320016+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:17:23.320016+0000","last_undegraded":"2024-09-17T08:17:23.320016+0000","last_fullsized":"2024-09-17T08:17:23.320016+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:15:53.067191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081478090000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":184,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.307694+0000","last_change":"2024-09-17T08:15:52.083346+0000","last_active":"2024-09-17T08:17:23.307694+0000","last_peered":"2024-09-17T08:17:23.307694+0000","last_clean":"2024-09-17T08:17:23.307694+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:17:23.307694+0000","last_undegraded":"2024-09-17T08:17:23.307694+0000","last_fullsized":"2024-09-17T08:17:23.307694+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.083080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.083080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011225087999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313849+0000","last_change":"2024-09-17T08:15:44.574330+0000","last_active":"2024-09-17T08:17:23.313849+0000","last_peered":"2024-09-17T08:17:23.313849+0000","last_clean":"2024-09-17T08:17:23.313849+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:17:23.313849+0000","last_undegraded":"2024-09-17T08:17:23.313849+0000","last_fullsized":"2024-09-17T08:17:23.313849+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:44.574209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.574209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050670539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"189'6448","reported_seq":6481,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.732420+0000","last_change":"2024-09-17T08:16:16.402968+0000","last_active":"2024-09-17T08:17:32.732420+0000","last_peered":"2024-09-17T08:17:32.732420+0000","last_clean":"2024-09-17T08:17:32.732420+0000","last_became_active":"2024-09-17T08:16:16.402701+0000","last_became_peered":"2024-09-17T08:16:16.402701+0000","last_unstale":"2024-09-17T08:17:32.732420+0000","last_undegraded":"2024-09-17T08:17:32.732420+0000","last_fullsized":"2024-09-17T08:17:32.732420+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4948,"log_dups_size":0,"ondisk_log_size":4948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422576128,"num_objects":6448,"num_object_clones":0,"num_object_copies":19344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":412672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":71,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314681+0000","last_change":"2024-09-17T08:16:33.206426+0000","last_active":"2024-09-17T08:17:23.314681+0000","last_peered":"2024-09-17T08:17:23.314681+0000","last_clean":"2024-09-17T08:17:23.314681+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:17:23.314681+0000","last_undegraded":"2024-09-17T08:17:23.314681+0000","last_fullsized":"2024-09-17T08:17:23.314681+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:16:33.206032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.206032+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:49.087945+0000","scrub_duration":0.0092541819999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":187,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.756517+0000","last_change":"2024-09-17T08:15:39.964221+0000","last_active":"2024-09-17T08:17:31.756517+0000","last_peered":"2024-09-17T08:17:31.756517+0000","last_clean":"2024-09-17T08:17:31.756517+0000","last_became_active":"2024-09-17T08:15:29.370817+0000","last_became_peered":"2024-09-17T08:15:29.370817+0000","last_unstale":"2024-09-17T08:17:31.756517+0000","last_undegraded":"2024-09-17T08:17:31.756517+0000","last_fullsized":"2024-09-17T08:17:31.756517+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:15:39.964058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:39.964058+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015935661,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":129,"num_read_kb":124,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325204+0000","last_change":"2024-09-17T08:16:35.307716+0000","last_active":"2024-09-17T08:17:23.325204+0000","last_peered":"2024-09-17T08:17:23.325204+0000","last_clean":"2024-09-17T08:17:23.325204+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:17:23.325204+0000","last_undegraded":"2024-09-17T08:17:23.325204+0000","last_fullsized":"2024-09-17T08:17:23.325204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.307594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.307594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:36.845467+0000","scrub_duration":0.0090619600000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":177,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.327036+0000","last_change":"2024-09-17T08:16:52.152636+0000","last_active":"2024-09-17T08:17:23.327036+0000","last_peered":"2024-09-17T08:17:23.327036+0000","last_clean":"2024-09-17T08:17:23.327036+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:17:23.327036+0000","last_undegraded":"2024-09-17T08:17:23.327036+0000","last_fullsized":"2024-09-17T08:17:23.327036+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.152367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.124044+0000","scrub_duration":0.0020852959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325440+0000","last_change":"2024-09-17T08:16:44.868243+0000","last_active":"2024-09-17T08:17:23.325440+0000","last_peered":"2024-09-17T08:17:23.325440+0000","last_clean":"2024-09-17T08:17:23.325440+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:17:23.325440+0000","last_undegraded":"2024-09-17T08:17:23.325440+0000","last_fullsized":"2024-09-17T08:17:23.325440+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.868012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.868012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:06.438510+0000","scrub_duration":0.0025635359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"189'6567","reported_seq":6599,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.583678+0000","last_change":"2024-09-17T08:16:16.648374+0000","last_active":"2024-09-17T08:17:32.583678+0000","last_peered":"2024-09-17T08:17:32.583678+0000","last_clean":"2024-09-17T08:17:32.583678+0000","last_became_active":"2024-09-17T08:16:16.647832+0000","last_became_peered":"2024-09-17T08:16:16.647832+0000","last_unstale":"2024-09-17T08:17:32.583678+0000","last_undegraded":"2024-09-17T08:17:32.583678+0000","last_fullsized":"2024-09-17T08:17:32.583678+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430374912,"num_objects":6567,"num_object_clones":0,"num_object_copies":19701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6567,"num_write_kb":420288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":249,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311386+0000","last_change":"2024-09-17T08:17:02.332268+0000","last_active":"2024-09-17T08:17:23.311386+0000","last_peered":"2024-09-17T08:17:23.311386+0000","last_clean":"2024-09-17T08:17:23.311386+0000","last_became_active":"2024-09-17T08:15:29.379763+0000","last_became_peered":"2024-09-17T08:15:29.379763+0000","last_unstale":"2024-09-17T08:17:23.311386+0000","last_undegraded":"2024-09-17T08:17:23.311386+0000","last_fullsized":"2024-09-17T08:17:23.311386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:17:02.331862+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.331862+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.727563+0000","scrub_duration":0.0094874130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":188,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311893+0000","last_change":"2024-09-17T08:16:50.181218+0000","last_active":"2024-09-17T08:17:23.311893+0000","last_peered":"2024-09-17T08:17:23.311893+0000","last_clean":"2024-09-17T08:17:23.311893+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:17:23.311893+0000","last_undegraded":"2024-09-17T08:17:23.311893+0000","last_fullsized":"2024-09-17T08:17:23.311893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:16:50.181067+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.169610+0000","scrub_duration":0.0098447959999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324393+0000","last_change":"2024-09-17T08:16:35.781222+0000","last_active":"2024-09-17T08:17:23.324393+0000","last_peered":"2024-09-17T08:17:23.324393+0000","last_clean":"2024-09-17T08:17:23.324393+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:17:23.324393+0000","last_undegraded":"2024-09-17T08:17:23.324393+0000","last_fullsized":"2024-09-17T08:17:23.324393+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.781112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.781112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.512658+0000","scrub_duration":0.0040414090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314840+0000","last_change":"2024-09-17T08:15:47.973188+0000","last_active":"2024-09-17T08:17:23.314840+0000","last_peered":"2024-09-17T08:17:23.314840+0000","last_clean":"2024-09-17T08:17:23.314840+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:17:23.314840+0000","last_undegraded":"2024-09-17T08:17:23.314840+0000","last_fullsized":"2024-09-17T08:17:23.314840+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:47.973067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:47.973067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046832360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.320927+0000","last_change":"2024-09-17T08:16:42.309390+0000","last_active":"2024-09-17T08:17:23.320927+0000","last_peered":"2024-09-17T08:17:23.320927+0000","last_clean":"2024-09-17T08:17:23.320927+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:17:23.320927+0000","last_undegraded":"2024-09-17T08:17:23.320927+0000","last_fullsized":"2024-09-17T08:17:23.320927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.309103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.309103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:45.195168+0000","scrub_duration":0.0057864980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"189'6537","reported_seq":6569,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.068572+0000","last_change":"2024-09-17T08:16:16.648023+0000","last_active":"2024-09-17T08:17:33.068572+0000","last_peered":"2024-09-17T08:17:33.068572+0000","last_clean":"2024-09-17T08:17:33.068572+0000","last_became_active":"2024-09-17T08:16:16.647748+0000","last_became_peered":"2024-09-17T08:16:16.647748+0000","last_unstale":"2024-09-17T08:17:33.068572+0000","last_undegraded":"2024-09-17T08:17:33.068572+0000","last_fullsized":"2024-09-17T08:17:33.068572+0000","mapping_epoch":165,"log_start":"173'800","ondisk_log_start":"173'800","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428408832,"num_objects":6537,"num_object_clones":0,"num_object_copies":19611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6537,"num_write_kb":418368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"139'21","reported_seq":152,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302223+0000","last_change":"2024-09-17T08:14:46.102116+0000","last_active":"2024-09-17T08:17:23.302223+0000","last_peered":"2024-09-17T08:17:23.302223+0000","last_clean":"2024-09-17T08:17:23.302223+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:17:23.302223+0000","last_undegraded":"2024-09-17T08:17:23.302223+0000","last_fullsized":"2024-09-17T08:17:23.302223+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'21","last_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:14:46.101949+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011657381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":166,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313823+0000","last_change":"2024-09-17T08:14:33.367441+0000","last_active":"2024-09-17T08:17:23.313823+0000","last_peered":"2024-09-17T08:17:23.313823+0000","last_clean":"2024-09-17T08:17:23.313823+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:17:23.313823+0000","last_undegraded":"2024-09-17T08:17:23.313823+0000","last_fullsized":"2024-09-17T08:17:23.313823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:14:33.367113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:33.367113+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052554730000000001,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.323795+0000","last_change":"2024-09-17T08:16:43.234496+0000","last_active":"2024-09-17T08:17:23.323795+0000","last_peered":"2024-09-17T08:17:23.323795+0000","last_clean":"2024-09-17T08:17:23.323795+0000","last_became_active":"2024-09-17T08:14:58.717890+0000","last_became_peered":"2024-09-17T08:14:58.717890+0000","last_unstale":"2024-09-17T08:17:23.323795+0000","last_undegraded":"2024-09-17T08:17:23.323795+0000","last_fullsized":"2024-09-17T08:17:23.323795+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:16:43.234376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.329605+0000","scrub_duration":0.0026197199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":431,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324532+0000","last_change":"2024-09-17T08:16:41.854709+0000","last_active":"2024-09-17T08:17:23.324532+0000","last_peered":"2024-09-17T08:17:23.324532+0000","last_clean":"2024-09-17T08:17:23.324532+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:17:23.324532+0000","last_undegraded":"2024-09-17T08:17:23.324532+0000","last_fullsized":"2024-09-17T08:17:23.324532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:16:41.854569+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:16:41.854569+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:43.215244+0000","scrub_duration":0.077111972000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":166,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325106+0000","last_change":"2024-09-17T08:16:34.817029+0000","last_active":"2024-09-17T08:17:23.325106+0000","last_peered":"2024-09-17T08:17:23.325106+0000","last_clean":"2024-09-17T08:17:23.325106+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:17:23.325106+0000","last_undegraded":"2024-09-17T08:17:23.325106+0000","last_fullsized":"2024-09-17T08:17:23.325106+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:34.816846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:16:34.816846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:44.180844+0000","scrub_duration":0.001439311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":69,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314288+0000","last_change":"2024-09-17T08:16:33.884267+0000","last_active":"2024-09-17T08:17:23.314288+0000","last_peered":"2024-09-17T08:17:23.314288+0000","last_clean":"2024-09-17T08:17:23.314288+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:17:23.314288+0000","last_undegraded":"2024-09-17T08:17:23.314288+0000","last_fullsized":"2024-09-17T08:17:23.314288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.884139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.884139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.670082+0000","scrub_duration":0.0075210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"189'6453","reported_seq":6486,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.062058+0000","last_change":"2024-09-17T08:16:16.646700+0000","last_active":"2024-09-17T08:17:33.062058+0000","last_peered":"2024-09-17T08:17:33.062058+0000","last_clean":"2024-09-17T08:17:33.062058+0000","last_became_active":"2024-09-17T08:16:16.646368+0000","last_became_peered":"2024-09-17T08:16:16.646368+0000","last_unstale":"2024-09-17T08:17:33.062058+0000","last_undegraded":"2024-09-17T08:17:33.062058+0000","last_fullsized":"2024-09-17T08:17:33.062058+0000","mapping_epoch":165,"log_start":"172'700","ondisk_log_start":"172'700","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5753,"log_dups_size":0,"ondisk_log_size":5753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422903808,"num_objects":6453,"num_object_clones":0,"num_object_copies":19359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6453,"num_write_kb":412992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":79,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.320081+0000","last_change":"2024-09-17T08:15:51.007614+0000","last_active":"2024-09-17T08:17:23.320081+0000","last_peered":"2024-09-17T08:17:23.320081+0000","last_clean":"2024-09-17T08:17:23.320081+0000","last_became_active":"2024-09-17T08:15:29.360821+0000","last_became_peered":"2024-09-17T08:15:29.360821+0000","last_unstale":"2024-09-17T08:17:23.320081+0000","last_undegraded":"2024-09-17T08:17:23.320081+0000","last_fullsized":"2024-09-17T08:17:23.320081+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:15:51.004953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:51.004953+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048347169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":170,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.328844+0000","last_change":"2024-09-17T08:15:36.077784+0000","last_active":"2024-09-17T08:17:23.328844+0000","last_peered":"2024-09-17T08:17:23.328844+0000","last_clean":"2024-09-17T08:17:23.328844+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:17:23.328844+0000","last_undegraded":"2024-09-17T08:17:23.328844+0000","last_fullsized":"2024-09-17T08:17:23.328844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:36.077647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:36.077647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013458461999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":177,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.323719+0000","last_change":"2024-09-17T08:15:52.380080+0000","last_active":"2024-09-17T08:17:23.323719+0000","last_peered":"2024-09-17T08:17:23.323719+0000","last_clean":"2024-09-17T08:17:23.323719+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:17:23.323719+0000","last_undegraded":"2024-09-17T08:17:23.323719+0000","last_fullsized":"2024-09-17T08:17:23.323719+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.379921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071947460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324461+0000","last_change":"2024-09-17T08:15:39.002957+0000","last_active":"2024-09-17T08:17:23.324461+0000","last_peered":"2024-09-17T08:17:23.324461+0000","last_clean":"2024-09-17T08:17:23.324461+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:17:23.324461+0000","last_undegraded":"2024-09-17T08:17:23.324461+0000","last_fullsized":"2024-09-17T08:17:23.324461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:39.002815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:39.002815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023086830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"189'6571","reported_seq":6604,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.536552+0000","last_change":"2024-09-17T08:16:16.650552+0000","last_active":"2024-09-17T08:17:32.536552+0000","last_peered":"2024-09-17T08:17:32.536552+0000","last_clean":"2024-09-17T08:17:32.536552+0000","last_became_active":"2024-09-17T08:16:16.650307+0000","last_became_peered":"2024-09-17T08:16:16.650307+0000","last_unstale":"2024-09-17T08:17:32.536552+0000","last_undegraded":"2024-09-17T08:17:32.536552+0000","last_fullsized":"2024-09-17T08:17:32.536552+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5071,"log_dups_size":0,"ondisk_log_size":5071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430637056,"num_objects":6571,"num_object_clones":0,"num_object_copies":19713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6571,"num_write_kb":420544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":222,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.308329+0000","last_change":"2024-09-17T08:15:55.110906+0000","last_active":"2024-09-17T08:17:23.308329+0000","last_peered":"2024-09-17T08:17:23.308329+0000","last_clean":"2024-09-17T08:17:23.308329+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:17:23.308329+0000","last_undegraded":"2024-09-17T08:17:23.308329+0000","last_fullsized":"2024-09-17T08:17:23.308329+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:15:55.110811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:55.110811+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001860472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315353+0000","last_change":"2024-09-17T08:15:40.995249+0000","last_active":"2024-09-17T08:17:23.315353+0000","last_peered":"2024-09-17T08:17:23.315353+0000","last_clean":"2024-09-17T08:17:23.315353+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:17:23.315353+0000","last_undegraded":"2024-09-17T08:17:23.315353+0000","last_fullsized":"2024-09-17T08:17:23.315353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:15:40.994687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042524099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":177,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.297258+0000","last_change":"2024-09-17T08:17:31.297792+0000","last_active":"2024-09-17T08:17:31.297258+0000","last_peered":"2024-09-17T08:17:31.297258+0000","last_clean":"2024-09-17T08:17:31.297258+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:17:31.297258+0000","last_undegraded":"2024-09-17T08:17:31.297258+0000","last_fullsized":"2024-09-17T08:17:31.297258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.297237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.297237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.099318+0000","scrub_duration":0.0044744609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311822+0000","last_change":"2024-09-17T08:16:54.185896+0000","last_active":"2024-09-17T08:17:23.311822+0000","last_peered":"2024-09-17T08:17:23.311822+0000","last_clean":"2024-09-17T08:17:23.311822+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:17:23.311822+0000","last_undegraded":"2024-09-17T08:17:23.311822+0000","last_fullsized":"2024-09-17T08:17:23.311822+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:54.185311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:54.185311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:16.656330+0000","scrub_duration":0.0041021820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"189'6512","reported_seq":6545,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.298623+0000","last_change":"2024-09-17T08:16:16.649082+0000","last_active":"2024-09-17T08:17:32.298623+0000","last_peered":"2024-09-17T08:17:32.298623+0000","last_clean":"2024-09-17T08:17:32.298623+0000","last_became_active":"2024-09-17T08:16:16.648709+0000","last_became_peered":"2024-09-17T08:16:16.648709+0000","last_unstale":"2024-09-17T08:17:32.298623+0000","last_undegraded":"2024-09-17T08:17:32.298623+0000","last_fullsized":"2024-09-17T08:17:32.298623+0000","mapping_epoch":165,"log_start":"173'1200","ondisk_log_start":"173'1200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5312,"log_dups_size":0,"ondisk_log_size":5312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426770432,"num_objects":6512,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6512,"num_write_kb":416768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"140'22","reported_seq":206,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324171+0000","last_change":"2024-09-17T08:16:57.000722+0000","last_active":"2024-09-17T08:17:23.324171+0000","last_peered":"2024-09-17T08:17:23.324171+0000","last_clean":"2024-09-17T08:17:23.324171+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:17:23.324171+0000","last_undegraded":"2024-09-17T08:17:23.324171+0000","last_fullsized":"2024-09-17T08:17:23.324171+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'22","last_scrub_stamp":"2024-09-17T08:16:57.000450+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:14:50.155143+0000","last_clean_scrub_stamp":"2024-09-17T08:16:57.000450+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.273416+0000","scrub_duration":0.0060780779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":181,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.327206+0000","last_change":"2024-09-17T08:16:40.246174+0000","last_active":"2024-09-17T08:17:23.327206+0000","last_peered":"2024-09-17T08:17:23.327206+0000","last_clean":"2024-09-17T08:17:23.327206+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:17:23.327206+0000","last_undegraded":"2024-09-17T08:17:23.327206+0000","last_fullsized":"2024-09-17T08:17:23.327206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.246057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.246057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.996076+0000","scrub_duration":0.0035572160000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":265,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.912930+0000","last_change":"2024-09-17T08:16:24.936605+0000","last_active":"2024-09-17T08:17:27.912930+0000","last_peered":"2024-09-17T08:17:27.912930+0000","last_clean":"2024-09-17T08:17:27.912930+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:17:27.912930+0000","last_undegraded":"2024-09-17T08:17:27.912930+0000","last_fullsized":"2024-09-17T08:17:27.912930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:16:24.936485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:24.936485+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.936007+0000","scrub_duration":0.0033469289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":80,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.307007+0000","last_change":"2024-09-17T08:16:58.243174+0000","last_active":"2024-09-17T08:17:23.307007+0000","last_peered":"2024-09-17T08:17:23.307007+0000","last_clean":"2024-09-17T08:17:23.307007+0000","last_became_active":"2024-09-17T08:14:58.717856+0000","last_became_peered":"2024-09-17T08:14:58.717856+0000","last_unstale":"2024-09-17T08:17:23.307007+0000","last_undegraded":"2024-09-17T08:17:23.307007+0000","last_fullsized":"2024-09-17T08:17:23.307007+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.242877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.242877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.573195+0000","scrub_duration":0.0056167969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"189'6424","reported_seq":6457,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.275972+0000","last_change":"2024-09-17T08:16:16.647722+0000","last_active":"2024-09-17T08:17:32.275972+0000","last_peered":"2024-09-17T08:17:32.275972+0000","last_clean":"2024-09-17T08:17:32.275972+0000","last_became_active":"2024-09-17T08:16:16.647324+0000","last_became_peered":"2024-09-17T08:16:16.647324+0000","last_unstale":"2024-09-17T08:17:32.275972+0000","last_undegraded":"2024-09-17T08:17:32.275972+0000","last_fullsized":"2024-09-17T08:17:32.275972+0000","mapping_epoch":165,"log_start":"173'1100","ondisk_log_start":"173'1100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.814258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421003264,"num_objects":6424,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6424,"num_write_kb":411136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":244,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301721+0000","last_change":"2024-09-17T08:17:00.600357+0000","last_active":"2024-09-17T08:17:23.301721+0000","last_peered":"2024-09-17T08:17:23.301721+0000","last_clean":"2024-09-17T08:17:23.301721+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:17:23.301721+0000","last_undegraded":"2024-09-17T08:17:23.301721+0000","last_fullsized":"2024-09-17T08:17:23.301721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:17:00.600225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.600225+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.375848+0000","scrub_duration":0.0071635179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":188,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302272+0000","last_change":"2024-09-17T08:16:39.838818+0000","last_active":"2024-09-17T08:17:23.302272+0000","last_peered":"2024-09-17T08:17:23.302272+0000","last_clean":"2024-09-17T08:17:23.302272+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:17:23.302272+0000","last_undegraded":"2024-09-17T08:17:23.302272+0000","last_fullsized":"2024-09-17T08:17:23.302272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:16:39.838667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.838667+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:42.922775+0000","scrub_duration":0.0050598739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":195,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.322892+0000","last_change":"2024-09-17T08:15:46.312539+0000","last_active":"2024-09-17T08:17:23.322892+0000","last_peered":"2024-09-17T08:17:23.322892+0000","last_clean":"2024-09-17T08:17:23.322892+0000","last_became_active":"2024-09-17T08:15:29.374092+0000","last_became_peered":"2024-09-17T08:15:29.374092+0000","last_unstale":"2024-09-17T08:17:23.322892+0000","last_undegraded":"2024-09-17T08:17:23.322892+0000","last_fullsized":"2024-09-17T08:17:23.322892+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:46.312367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:15:46.312367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012162289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":182,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.316305+0000","last_change":"2024-09-17T08:16:23.763256+0000","last_active":"2024-09-17T08:17:23.316305+0000","last_peered":"2024-09-17T08:17:23.316305+0000","last_clean":"2024-09-17T08:17:23.316305+0000","last_became_active":"2024-09-17T08:15:29.384203+0000","last_became_peered":"2024-09-17T08:15:29.384203+0000","last_unstale":"2024-09-17T08:17:23.316305+0000","last_undegraded":"2024-09-17T08:17:23.316305+0000","last_fullsized":"2024-09-17T08:17:23.316305+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.763173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.763173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.300130+0000","scrub_duration":0.001814333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":188,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311308+0000","last_change":"2024-09-17T08:16:36.341678+0000","last_active":"2024-09-17T08:17:23.311308+0000","last_peered":"2024-09-17T08:17:23.311308+0000","last_clean":"2024-09-17T08:17:23.311308+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:17:23.311308+0000","last_undegraded":"2024-09-17T08:17:23.311308+0000","last_fullsized":"2024-09-17T08:17:23.311308+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:16:36.341539+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:36.341539+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:37.074093+0000","scrub_duration":0.011421961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":199,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.323004+0000","last_change":"2024-09-17T08:16:49.193851+0000","last_active":"2024-09-17T08:17:23.323004+0000","last_peered":"2024-09-17T08:17:23.323004+0000","last_clean":"2024-09-17T08:17:23.323004+0000","last_became_active":"2024-09-17T08:15:29.373114+0000","last_became_peered":"2024-09-17T08:15:29.373114+0000","last_unstale":"2024-09-17T08:17:23.323004+0000","last_undegraded":"2024-09-17T08:17:23.323004+0000","last_fullsized":"2024-09-17T08:17:23.323004+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.193338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.193338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:59.549312+0000","scrub_duration":0.0089744739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":173,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314464+0000","last_change":"2024-09-17T08:15:33.385747+0000","last_active":"2024-09-17T08:17:23.314464+0000","last_peered":"2024-09-17T08:17:23.314464+0000","last_clean":"2024-09-17T08:17:23.314464+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:17:23.314464+0000","last_undegraded":"2024-09-17T08:17:23.314464+0000","last_fullsized":"2024-09-17T08:17:23.314464+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:33.385285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:33.385285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077031670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.323731+0000","last_change":"2024-09-17T08:16:48.222810+0000","last_active":"2024-09-17T08:17:23.323731+0000","last_peered":"2024-09-17T08:17:23.323731+0000","last_clean":"2024-09-17T08:17:23.323731+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:17:23.323731+0000","last_undegraded":"2024-09-17T08:17:23.323731+0000","last_fullsized":"2024-09-17T08:17:23.323731+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.222304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.222304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.070451+0000","scrub_duration":0.003060929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":139,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.326150+0000","last_change":"2024-09-17T08:16:29.839526+0000","last_active":"2024-09-17T08:17:27.326150+0000","last_peered":"2024-09-17T08:17:27.326150+0000","last_clean":"2024-09-17T08:17:27.326150+0000","last_became_active":"2024-09-17T08:15:29.358382+0000","last_became_peered":"2024-09-17T08:15:29.358382+0000","last_unstale":"2024-09-17T08:17:27.326150+0000","last_undegraded":"2024-09-17T08:17:27.326150+0000","last_fullsized":"2024-09-17T08:17:27.326150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:29.839416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.839416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:32.179348+0000","scrub_duration":0.002703015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":211,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314574+0000","last_change":"2024-09-17T08:16:30.802344+0000","last_active":"2024-09-17T08:17:23.314574+0000","last_peered":"2024-09-17T08:17:23.314574+0000","last_clean":"2024-09-17T08:17:23.314574+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:17:23.314574+0000","last_undegraded":"2024-09-17T08:17:23.314574+0000","last_fullsized":"2024-09-17T08:17:23.314574+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:16:30.800480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.800480+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:43.471047+0000","scrub_duration":0.0031490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":76,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.836401+0000","last_change":"2024-09-17T08:15:29.383868+0000","last_active":"2024-09-17T08:17:27.836401+0000","last_peered":"2024-09-17T08:17:27.836401+0000","last_clean":"2024-09-17T08:17:27.836401+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:17:27.836401+0000","last_undegraded":"2024-09-17T08:17:27.836401+0000","last_fullsized":"2024-09-17T08:17:27.836401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:51.191602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:51.191602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.325361+0000","last_change":"2024-09-17T08:16:51.020489+0000","last_active":"2024-09-17T08:17:23.325361+0000","last_peered":"2024-09-17T08:17:23.325361+0000","last_clean":"2024-09-17T08:17:23.325361+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:17:23.325361+0000","last_undegraded":"2024-09-17T08:17:23.325361+0000","last_fullsized":"2024-09-17T08:17:23.325361+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.020318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.020318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.433650+0000","scrub_duration":0.0019765820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":225,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313535+0000","last_change":"2024-09-17T08:16:45.629362+0000","last_active":"2024-09-17T08:17:23.313535+0000","last_peered":"2024-09-17T08:17:23.313535+0000","last_clean":"2024-09-17T08:17:23.313535+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:17:23.313535+0000","last_undegraded":"2024-09-17T08:17:23.313535+0000","last_fullsized":"2024-09-17T08:17:23.313535+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:45.626489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:45.626489+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.848535+0000","scrub_duration":0.0060751379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":101,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.313192+0000","last_change":"2024-09-17T08:14:58.721591+0000","last_active":"2024-09-17T08:17:23.313192+0000","last_peered":"2024-09-17T08:17:23.313192+0000","last_clean":"2024-09-17T08:17:23.313192+0000","last_became_active":"2024-09-17T08:14:58.720104+0000","last_became_peered":"2024-09-17T08:14:58.720104+0000","last_unstale":"2024-09-17T08:17:23.313192+0000","last_undegraded":"2024-09-17T08:17:23.313192+0000","last_fullsized":"2024-09-17T08:17:23.313192+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:47.207288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:47.207288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"152'11","reported_seq":88,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.913097+0000","last_change":"2024-09-17T08:15:52.034956+0000","last_active":"2024-09-17T08:17:27.913097+0000","last_peered":"2024-09-17T08:17:27.913097+0000","last_clean":"2024-09-17T08:17:27.913097+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:17:27.913097+0000","last_undegraded":"2024-09-17T08:17:27.913097+0000","last_fullsized":"2024-09-17T08:17:27.913097+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'11","last_scrub_stamp":"2024-09-17T08:15:52.034662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.034662+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038045739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.310890+0000","last_change":"2024-09-17T08:15:49.070971+0000","last_active":"2024-09-17T08:17:23.310890+0000","last_peered":"2024-09-17T08:17:23.310890+0000","last_clean":"2024-09-17T08:17:23.310890+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:17:23.310890+0000","last_undegraded":"2024-09-17T08:17:23.310890+0000","last_fullsized":"2024-09-17T08:17:23.310890+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:49.070359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:49.070359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082685020000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"177'13","reported_seq":113,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.308905+0000","last_change":"2024-09-17T08:17:00.945597+0000","last_active":"2024-09-17T08:17:27.308905+0000","last_peered":"2024-09-17T08:17:27.308905+0000","last_clean":"2024-09-17T08:17:27.308905+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:17:27.308905+0000","last_undegraded":"2024-09-17T08:17:27.308905+0000","last_fullsized":"2024-09-17T08:17:27.308905+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"177'13","last_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.945463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.215485+0000","scrub_duration":0.016379540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":181,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.308536+0000","last_change":"2024-09-17T08:16:46.264262+0000","last_active":"2024-09-17T08:17:23.308536+0000","last_peered":"2024-09-17T08:17:23.308536+0000","last_clean":"2024-09-17T08:17:23.308536+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:17:23.308536+0000","last_undegraded":"2024-09-17T08:17:23.308536+0000","last_fullsized":"2024-09-17T08:17:23.308536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:46.264138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.264138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.163330+0000","scrub_duration":0.0046054340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":255,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.914132+0000","last_change":"2024-09-17T08:14:39.413470+0000","last_active":"2024-09-17T08:17:27.914132+0000","last_peered":"2024-09-17T08:17:27.914132+0000","last_clean":"2024-09-17T08:17:27.914132+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:17:27.914132+0000","last_undegraded":"2024-09-17T08:17:27.914132+0000","last_fullsized":"2024-09-17T08:17:27.914132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:14:39.413074+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:14:39.413074+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015248731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.301910+0000","last_change":"2024-09-17T08:16:44.899582+0000","last_active":"2024-09-17T08:17:23.301910+0000","last_peered":"2024-09-17T08:17:23.301910+0000","last_clean":"2024-09-17T08:17:23.301910+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:17:23.301910+0000","last_undegraded":"2024-09-17T08:17:23.301910+0000","last_fullsized":"2024-09-17T08:17:23.301910+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.899473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.899473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.638190+0000","scrub_duration":0.0048647389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":69,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.426207+0000","last_change":"2024-09-17T08:17:31.426355+0000","last_active":"2024-09-17T08:17:31.426207+0000","last_peered":"2024-09-17T08:17:31.426207+0000","last_clean":"2024-09-17T08:17:31.426207+0000","last_became_active":"2024-09-17T08:15:29.364047+0000","last_became_peered":"2024-09-17T08:15:29.364047+0000","last_unstale":"2024-09-17T08:17:31.426207+0000","last_undegraded":"2024-09-17T08:17:31.426207+0000","last_fullsized":"2024-09-17T08:17:31.426207+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:31.426185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.426185+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.778192+0000","scrub_duration":0.028345339000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":121,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.836368+0000","last_change":"2024-09-17T08:15:54.838921+0000","last_active":"2024-09-17T08:17:27.836368+0000","last_peered":"2024-09-17T08:17:27.836368+0000","last_clean":"2024-09-17T08:17:27.836368+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:17:27.836368+0000","last_undegraded":"2024-09-17T08:17:27.836368+0000","last_fullsized":"2024-09-17T08:17:27.836368+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.838847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.838847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":173,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.316373+0000","last_change":"2024-09-17T08:15:29.384146+0000","last_active":"2024-09-17T08:17:23.316373+0000","last_peered":"2024-09-17T08:17:23.316373+0000","last_clean":"2024-09-17T08:17:23.316373+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:17:23.316373+0000","last_undegraded":"2024-09-17T08:17:23.316373+0000","last_fullsized":"2024-09-17T08:17:23.316373+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:13:58.267204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:13:58.267204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072680419999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":182,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.306831+0000","last_change":"2024-09-17T08:15:36.046869+0000","last_active":"2024-09-17T08:17:23.306831+0000","last_peered":"2024-09-17T08:17:23.306831+0000","last_clean":"2024-09-17T08:17:23.306831+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:17:23.306831+0000","last_undegraded":"2024-09-17T08:17:23.306831+0000","last_fullsized":"2024-09-17T08:17:23.306831+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:36.046738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:36.046738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002465362,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":240,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302456+0000","last_change":"2024-09-17T08:16:59.607476+0000","last_active":"2024-09-17T08:17:23.302456+0000","last_peered":"2024-09-17T08:17:23.302456+0000","last_clean":"2024-09-17T08:17:23.302456+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:17:23.302456+0000","last_undegraded":"2024-09-17T08:17:23.302456+0000","last_fullsized":"2024-09-17T08:17:23.302456+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:16:59.607325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.607325+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.411643+0000","scrub_duration":0.0083300879999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315704+0000","last_change":"2024-09-17T08:17:01.469617+0000","last_active":"2024-09-17T08:17:23.315704+0000","last_peered":"2024-09-17T08:17:23.315704+0000","last_clean":"2024-09-17T08:17:23.315704+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:17:23.315704+0000","last_undegraded":"2024-09-17T08:17:23.315704+0000","last_fullsized":"2024-09-17T08:17:23.315704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.469473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.469473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:22.713180+0000","scrub_duration":0.0032550880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":59,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.455149+0000","last_change":"2024-09-17T08:17:32.455271+0000","last_active":"2024-09-17T08:17:32.455149+0000","last_peered":"2024-09-17T08:17:32.455149+0000","last_clean":"2024-09-17T08:17:32.455149+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:17:32.455149+0000","last_undegraded":"2024-09-17T08:17:32.455149+0000","last_fullsized":"2024-09-17T08:17:32.455149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.455128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.455128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.037261+0000","scrub_duration":0.0091779470000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312271+0000","last_change":"2024-09-17T08:16:39.766499+0000","last_active":"2024-09-17T08:17:23.312271+0000","last_peered":"2024-09-17T08:17:23.312271+0000","last_clean":"2024-09-17T08:17:23.312271+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:17:23.312271+0000","last_undegraded":"2024-09-17T08:17:23.312271+0000","last_fullsized":"2024-09-17T08:17:23.312271+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:39.766371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.766371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:51.034351+0000","scrub_duration":0.003380568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":208,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312995+0000","last_change":"2024-09-17T08:16:33.342841+0000","last_active":"2024-09-17T08:17:23.312995+0000","last_peered":"2024-09-17T08:17:23.312995+0000","last_clean":"2024-09-17T08:17:23.312995+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:17:23.312995+0000","last_undegraded":"2024-09-17T08:17:23.312995+0000","last_fullsized":"2024-09-17T08:17:23.312995+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:14:35.433619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.433619+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079958110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":70,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.684056+0000","last_change":"2024-09-17T08:15:47.267637+0000","last_active":"2024-09-17T08:17:27.684056+0000","last_peered":"2024-09-17T08:17:27.684056+0000","last_clean":"2024-09-17T08:17:27.684056+0000","last_became_active":"2024-09-17T08:14:58.715288+0000","last_became_peered":"2024-09-17T08:14:58.715288+0000","last_unstale":"2024-09-17T08:17:27.684056+0000","last_undegraded":"2024-09-17T08:17:27.684056+0000","last_fullsized":"2024-09-17T08:17:27.684056+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:47.264639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:47.264639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070869089999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":133,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.626770+0000","last_change":"2024-09-17T08:17:31.626868+0000","last_active":"2024-09-17T08:17:31.626770+0000","last_peered":"2024-09-17T08:17:31.626770+0000","last_clean":"2024-09-17T08:17:31.626770+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:17:31.626770+0000","last_undegraded":"2024-09-17T08:17:31.626770+0000","last_fullsized":"2024-09-17T08:17:31.626770+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.626749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:52.132279+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.626749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.963940+0000","scrub_duration":0.002332518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.324546+0000","last_change":"2024-09-17T08:16:52.066187+0000","last_active":"2024-09-17T08:17:23.324546+0000","last_peered":"2024-09-17T08:17:23.324546+0000","last_clean":"2024-09-17T08:17:23.324546+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:17:23.324546+0000","last_undegraded":"2024-09-17T08:17:23.324546+0000","last_fullsized":"2024-09-17T08:17:23.324546+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.066058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.066058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.573732+0000","scrub_duration":0.0023799279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":176,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312450+0000","last_change":"2024-09-17T08:13:48.179594+0000","last_active":"2024-09-17T08:17:23.312450+0000","last_peered":"2024-09-17T08:17:23.312450+0000","last_clean":"2024-09-17T08:17:23.312450+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:17:23.312450+0000","last_undegraded":"2024-09-17T08:17:23.312450+0000","last_fullsized":"2024-09-17T08:17:23.312450+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:13:48.179449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:13:48.179449+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059153540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":107,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.684093+0000","last_change":"2024-09-17T08:16:27.299767+0000","last_active":"2024-09-17T08:17:27.684093+0000","last_peered":"2024-09-17T08:17:27.684093+0000","last_clean":"2024-09-17T08:17:27.684093+0000","last_became_active":"2024-09-17T08:14:58.716813+0000","last_became_peered":"2024-09-17T08:14:58.716813+0000","last_unstale":"2024-09-17T08:17:27.684093+0000","last_undegraded":"2024-09-17T08:17:27.684093+0000","last_fullsized":"2024-09-17T08:17:27.684093+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:16:27.299675+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.299675+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.173560+0000","scrub_duration":0.0018966499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":166,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.323987+0000","last_change":"2024-09-17T08:15:55.859292+0000","last_active":"2024-09-17T08:17:23.323987+0000","last_peered":"2024-09-17T08:17:23.323987+0000","last_clean":"2024-09-17T08:17:23.323987+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:17:23.323987+0000","last_undegraded":"2024-09-17T08:17:23.323987+0000","last_fullsized":"2024-09-17T08:17:23.323987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:55.858966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:55.858966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021326819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.316226+0000","last_change":"2024-09-17T08:15:51.743705+0000","last_active":"2024-09-17T08:17:23.316226+0000","last_peered":"2024-09-17T08:17:23.316226+0000","last_clean":"2024-09-17T08:17:23.316226+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:17:23.316226+0000","last_undegraded":"2024-09-17T08:17:23.316226+0000","last_fullsized":"2024-09-17T08:17:23.316226+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:51.721883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:51.721883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082713839999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":166,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427441+0000","last_change":"2024-09-17T08:16:40.783169+0000","last_active":"2024-09-17T08:17:30.427441+0000","last_peered":"2024-09-17T08:17:30.427441+0000","last_clean":"2024-09-17T08:17:30.427441+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:17:30.427441+0000","last_undegraded":"2024-09-17T08:17:30.427441+0000","last_fullsized":"2024-09-17T08:17:30.427441+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:16:40.783041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.783041+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.590749+0000","scrub_duration":0.0030603710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":70,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.326086+0000","last_change":"2024-09-17T08:15:29.977775+0000","last_active":"2024-09-17T08:17:27.326086+0000","last_peered":"2024-09-17T08:17:27.326086+0000","last_clean":"2024-09-17T08:17:27.326086+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:17:27.326086+0000","last_undegraded":"2024-09-17T08:17:27.326086+0000","last_fullsized":"2024-09-17T08:17:27.326086+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:29.977666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:29.977666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012253139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":194,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311230+0000","last_change":"2024-09-17T08:16:47.247472+0000","last_active":"2024-09-17T08:17:23.311230+0000","last_peered":"2024-09-17T08:17:23.311230+0000","last_clean":"2024-09-17T08:17:23.311230+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:17:23.311230+0000","last_undegraded":"2024-09-17T08:17:23.311230+0000","last_fullsized":"2024-09-17T08:17:23.311230+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:47.247339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.247339+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:15.653964+0000","scrub_duration":0.0072034789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":201,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.913181+0000","last_change":"2024-09-17T08:16:52.823084+0000","last_active":"2024-09-17T08:17:27.913181+0000","last_peered":"2024-09-17T08:17:27.913181+0000","last_clean":"2024-09-17T08:17:27.913181+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:17:27.913181+0000","last_undegraded":"2024-09-17T08:17:27.913181+0000","last_fullsized":"2024-09-17T08:17:27.913181+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:16:52.822979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.822979+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.218032+0000","scrub_duration":0.0016070290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315430+0000","last_change":"2024-09-17T08:15:33.983721+0000","last_active":"2024-09-17T08:17:23.315430+0000","last_peered":"2024-09-17T08:17:23.315430+0000","last_clean":"2024-09-17T08:17:23.315430+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:17:23.315430+0000","last_undegraded":"2024-09-17T08:17:23.315430+0000","last_fullsized":"2024-09-17T08:17:23.315430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:33.983570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:33.983570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007829595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":170,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.312751+0000","last_change":"2024-09-17T08:16:58.658509+0000","last_active":"2024-09-17T08:17:23.312751+0000","last_peered":"2024-09-17T08:17:23.312751+0000","last_clean":"2024-09-17T08:17:23.312751+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:17:23.312751+0000","last_undegraded":"2024-09-17T08:17:23.312751+0000","last_fullsized":"2024-09-17T08:17:23.312751+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.658382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.658382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.307934+0000","scrub_duration":0.0056271569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":60,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315029+0000","last_change":"2024-09-17T08:15:29.399964+0000","last_active":"2024-09-17T08:17:23.315029+0000","last_peered":"2024-09-17T08:17:23.315029+0000","last_clean":"2024-09-17T08:17:23.315029+0000","last_became_active":"2024-09-17T08:15:29.361714+0000","last_became_peered":"2024-09-17T08:15:29.361714+0000","last_unstale":"2024-09-17T08:17:23.315029+0000","last_undegraded":"2024-09-17T08:17:23.315029+0000","last_fullsized":"2024-09-17T08:17:23.315029+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:14:52.156316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:52.156316+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":177,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.311526+0000","last_change":"2024-09-17T08:16:49.210341+0000","last_active":"2024-09-17T08:17:23.311526+0000","last_peered":"2024-09-17T08:17:23.311526+0000","last_clean":"2024-09-17T08:17:23.311526+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:17:23.311526+0000","last_undegraded":"2024-09-17T08:17:23.311526+0000","last_fullsized":"2024-09-17T08:17:23.311526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.210217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.210217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.183066+0000","scrub_duration":0.0066617630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":164,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302559+0000","last_change":"2024-09-17T08:16:37.835578+0000","last_active":"2024-09-17T08:17:23.302559+0000","last_peered":"2024-09-17T08:17:23.302559+0000","last_clean":"2024-09-17T08:17:23.302559+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:17:23.302559+0000","last_undegraded":"2024-09-17T08:17:23.302559+0000","last_fullsized":"2024-09-17T08:17:23.302559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.835368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.835368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.709790+0000","scrub_duration":0.001605574,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":120,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.302126+0000","last_change":"2024-09-17T08:14:48.151364+0000","last_active":"2024-09-17T08:17:23.302126+0000","last_peered":"2024-09-17T08:17:23.302126+0000","last_clean":"2024-09-17T08:17:23.302126+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:17:23.302126+0000","last_undegraded":"2024-09-17T08:17:23.302126+0000","last_fullsized":"2024-09-17T08:17:23.302126+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:48.151226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:14:48.151226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023704030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":286,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.684015+0000","last_change":"2024-09-17T08:16:59.092346+0000","last_active":"2024-09-17T08:17:27.684015+0000","last_peered":"2024-09-17T08:17:27.684015+0000","last_clean":"2024-09-17T08:17:27.684015+0000","last_became_active":"2024-09-17T08:15:29.388191+0000","last_became_peered":"2024-09-17T08:15:29.388191+0000","last_unstale":"2024-09-17T08:17:27.684015+0000","last_undegraded":"2024-09-17T08:17:27.684015+0000","last_fullsized":"2024-09-17T08:17:27.684015+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:16:59.092016+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.092016+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.568034+0000","scrub_duration":0.0088076999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":59,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.315566+0000","last_change":"2024-09-17T08:16:51.343826+0000","last_active":"2024-09-17T08:17:23.315566+0000","last_peered":"2024-09-17T08:17:23.315566+0000","last_clean":"2024-09-17T08:17:23.315566+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:17:23.315566+0000","last_undegraded":"2024-09-17T08:17:23.315566+0000","last_fullsized":"2024-09-17T08:17:23.315566+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.343712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.343712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.929418+0000","scrub_duration":0.010532788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:23.314391+0000","last_change":"2024-09-17T08:15:34.433273+0000","last_active":"2024-09-17T08:17:23.314391+0000","last_peered":"2024-09-17T08:17:23.314391+0000","last_clean":"2024-09-17T08:17:23.314391+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:17:23.314391+0000","last_undegraded":"2024-09-17T08:17:23.314391+0000","last_fullsized":"2024-09-17T08:17:23.314391+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:34.432898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:34.432898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012758472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":170,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T08:17:27.836178+0000","last_change":"2024-09-17T08:15:48.646475+0000","last_active":"2024-09-17T08:17:27.836178+0000","last_peered":"2024-09-17T08:17:27.836178+0000","last_clean":"2024-09-17T08:17:27.836178+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:17:27.836178+0000","last_undegraded":"2024-09-17T08:17:27.836178+0000","last_fullsized":"2024-09-17T08:17:27.836178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.646329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.646329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076865340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":233,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.435900+0000","last_change":"2024-09-17T08:16:29.751857+0000","last_active":"2024-09-17T08:17:30.435900+0000","last_peered":"2024-09-17T08:17:30.435900+0000","last_clean":"2024-09-17T08:17:30.435900+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:17:30.435900+0000","last_undegraded":"2024-09-17T08:17:30.435900+0000","last_fullsized":"2024-09-17T08:17:30.435900+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:16:29.751693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.751693+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.331793+0000","scrub_duration":0.0054470619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":166,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420833+0000","last_change":"2024-09-17T08:14:55.113389+0000","last_active":"2024-09-17T08:17:30.420833+0000","last_peered":"2024-09-17T08:17:30.420833+0000","last_clean":"2024-09-17T08:17:30.420833+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:17:30.420833+0000","last_undegraded":"2024-09-17T08:17:30.420833+0000","last_fullsized":"2024-09-17T08:17:30.420833+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:55.113268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:55.113268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045114200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6797983744,"num_objects":103729,"num_object_clones":0,"num_object_copies":311187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103729,"num_write_kb":6638656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20367212544,"data_stored":20367212544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83829,"ondisk_log_size":83829,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":690,"num_read_kb":447,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":451,"ondisk_log_size":451,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070561,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2211800,"kb_used_data":2054348,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":91533352,"statfs":{"total":95995035648,"available":93730152448,"internally_reserved":0,"allocated":2103652352,"data_stored":2103357783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":161216971},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.3899999999999999,"15min":0.82699999999999996},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.665,"5min":27.751999999999999,"15min":27.751999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.5660000000000001,"15min":0.88500000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":4.0510000000000002,"5min":37.954000000000001,"15min":37.954000000000001},"last":1.1279999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.85399999999999998,"15min":0.68100000000000005},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.708,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.505},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.628},"min":{"1min":0.26100000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.109}]},{"osd":2,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.028,"15min":0.70099999999999996},"min":{"1min":0.38300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.423,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":0.72599999999999998},"min":{"1min":0.32700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":3.7469999999999999,"5min":4.8310000000000004,"15min":4.8310000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.3129999999999999,"15min":0.79100000000000004},"min":{"1min":0.39600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.625,"5min":11.298,"15min":11.298},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.397,"15min":0.85699999999999998},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.5499999999999998,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.45300000000000001},"min":{"1min":0.252,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1890000000000001,"5min":2.21,"15min":2.21},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.33200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.745,"5min":1.492,"15min":1.492},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":0.68300000000000005,"15min":0.55100000000000005},"min":{"1min":0.26100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":16.808,"5min":16.808,"15min":16.808},"last":0.628},{"interface":"front","average":{"1min":1.115,"5min":0.61699999999999999,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":11.842000000000001,"5min":11.842000000000001,"15min":11.842000000000001},"last":1.087}]},{"osd":6,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.5369999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.45900000000000002}]}]},{"osd":6,"up_from":50,"seq":214748364902,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2214920,"kb_used_data":2049276,"kb_used_omap":1,"kb_used_meta":165630,"kb_avail":91530232,"statfs":{"total":95995035648,"available":93726957568,"internally_reserved":0,"allocated":2098458624,"data_stored":2098179699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":169605575},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":3.1110000000000002,"15min":1.3759999999999999},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.6890000000000001,"5min":73.757999999999996,"15min":73.757999999999996},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":3.2130000000000001,"15min":1.397},"min":{"1min":0.26200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.9820000000000002,"5min":73.811000000000007,"15min":73.811000000000007},"last":0.32800000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":1.1459999999999999,"15min":0.72799999999999998},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":4.0140000000000002,"5min":28.288,"15min":28.288},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.746,"5min":1.214,"15min":0.76200000000000001},"min":{"1min":0.36299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.857,"5min":28.939,"15min":28.939},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":1.502,"15min":0.85399999999999998},"min":{"1min":0.28599999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.823,"5min":68.165000000000006,"15min":68.165000000000006},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.7549999999999999,"15min":0.92000000000000004},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.948,"5min":67.997,"15min":67.997},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":2.2679999999999998,"15min":1.127},"min":{"1min":0.35499999999999998,"5min":0.30599999999999999,"15min":0.29899999999999999},"max":{"1min":3.028,"5min":77.010999999999996,"15min":77.010999999999996},"last":1.101},{"interface":"front","average":{"1min":1.05,"5min":2.2160000000000002,"15min":1.0980000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":7.1210000000000004,"5min":76.375,"15min":76.375},"last":0.95899999999999996}]},{"osd":4,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58299999999999996,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.919,"5min":2.964,"15min":2.964},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.438,"15min":0.39800000000000002},"min":{"1min":0.312,"5min":0.25,"15min":0.25},"max":{"1min":0.76300000000000001,"5min":1.099,"15min":1.099},"last":0.38600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.245,"15min":0.245},"max":{"1min":2.8410000000000002,"5min":2.9289999999999998,"15min":2.9289999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.44800000000000001},"min":{"1min":0.33300000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72699999999999998,"5min":1.645,"15min":1.645},"last":0.41299999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.66100000000000003,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.438}]}]},{"osd":5,"up_from":41,"seq":176093659243,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3513800,"kb_used_data":3288252,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":90231352,"statfs":{"total":95995035648,"available":92396904448,"internally_reserved":0,"allocated":3367170048,"data_stored":3366890528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":230947270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":2.1000000000000001,"15min":1.0389999999999999},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":5.3079999999999998,"5min":63.098999999999997,"15min":63.098999999999997},"last":3.9359999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.948,"15min":0.97199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.0899999999999999,"5min":62.567,"15min":62.567},"last":1.0549999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.64000000000000001},"min":{"1min":0.33300000000000002,"5min":0.222,"15min":0.222},"max":{"1min":2.532,"5min":5.056,"15min":5.056},"last":1.7090000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.107,"15min":0.70299999999999996},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":2.4580000000000002,"5min":8.8119999999999994,"15min":8.8119999999999994},"last":2.577}]},{"osd":2,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.014,"15min":0.68899999999999995},"min":{"1min":0.247,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.589,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.89300000000000002,"15min":0.63300000000000001},"min":{"1min":0.315,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.141,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.313}]},{"osd":3,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":2.149,"15min":1.05},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.6259999999999999,"5min":75.286000000000001,"15min":75.286000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.999,"5min":2.089,"15min":1.05},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.29499999999999998},"max":{"1min":5.8300000000000001,"5min":77.206999999999994,"15min":77.206999999999994},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.40699999999999997},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.19},"max":{"1min":1.1899999999999999,"5min":3.544,"15min":3.544},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.29199999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.45300000000000001},"min":{"1min":0.38100000000000001,"5min":0.29499999999999998,"15min":0.254},"max":{"1min":1.2569999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.60199999999999998,"15min":0.51600000000000001},"min":{"1min":0.38100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3330000000000002,"5min":3.645,"15min":3.645},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.49199999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":1.0309999999999999,"5min":3.613,"15min":3.613},"last":0.73499999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953584,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2215568,"kb_used_data":2049928,"kb_used_omap":1,"kb_used_meta":165630,"kb_avail":91529584,"statfs":{"total":95995035648,"available":93726294016,"internally_reserved":0,"allocated":2099126272,"data_stored":2098835093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169605574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":1.3200000000000001,"15min":0.77300000000000002},"min":{"1min":0.253,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.2759999999999998,"5min":23.786000000000001,"15min":23.786000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.27,"15min":0.76300000000000001},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":1.4610000000000001,"5min":23.245000000000001,"15min":23.245000000000001},"last":1.7629999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":1.0569999999999999,"15min":0.69899999999999995},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":3.5779999999999998,"5min":23.135000000000002,"15min":23.135000000000002},"last":3.5230000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":1.1339999999999999,"15min":0.71399999999999997},"min":{"1min":0.38600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.1640000000000001,"5min":23.434999999999999,"15min":23.434999999999999},"last":3.4910000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.81499999999999995,"15min":0.624},"min":{"1min":0.32000000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.94699999999999995,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":2.2040000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.83299999999999996,"15min":0.60599999999999998},"min":{"1min":0.27100000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.016,"5min":3.4470000000000001,"15min":3.4470000000000001},"last":2.1299999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":3.3300000000000001,"15min":1.46},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.319,"5min":154.137,"15min":154.137},"last":3.4359999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":3.2109999999999999,"15min":1.4159999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.1800000000000002,"5min":153.54400000000001,"15min":153.54400000000001},"last":1.8799999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":2.1789999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.504,"15min":0.432},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":2.1070000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.441,"15min":0.40200000000000002},"min":{"1min":0.27300000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.86599999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.9079999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.51400000000000001,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":2.1179999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0229999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":2.0760000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":2.5800000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117108,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2729884,"kb_used_data":2460052,"kb_used_omap":1,"kb_used_meta":269822,"kb_avail":91015268,"statfs":{"total":95995035648,"available":93199634432,"internally_reserved":0,"allocated":2519093248,"data_stored":2518789258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":276298184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.851,"5min":1.9790000000000001,"15min":0.90300000000000002},"min":{"1min":0.23799999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":5.1879999999999997,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.8140000000000001,"15min":0.85099999999999998},"min":{"1min":0.28999999999999998,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":3.3940000000000001,"5min":10.362,"15min":10.362},"last":0.95799999999999996}]},{"osd":1,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.825,"15min":0.879},"min":{"1min":0.25800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":4.7990000000000004,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":1.579},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.867,"15min":0.89400000000000002},"min":{"1min":0.374,"5min":0.188,"15min":0.188},"max":{"1min":3.0539999999999998,"5min":10.891999999999999,"15min":10.891999999999999},"last":2.0449999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.9319999999999999,"15min":0.91000000000000003},"min":{"1min":0.26400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":2.976,"5min":10.786,"15min":10.786},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.8140000000000001,"5min":2.2669999999999999,"15min":1.0189999999999999},"min":{"1min":0.30299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.46,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":0.749}]},{"osd":4,"last update":"Tue Sep 17 08:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.373,"15min":0.81999999999999995},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":2.8460000000000001,"5min":7.742,"15min":7.742},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2609999999999999,"15min":0.78100000000000003},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.9850000000000003,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.83399999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.4490000000000001,"15min":0.86199999999999999},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.4870000000000001,"5min":8.2240000000000002,"15min":8.2240000000000002},"last":2.093},{"interface":"front","average":{"1min":1.724,"5min":1.482,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.4359999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Sep 17 08:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.373,"15min":0.83499999999999996},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":6.2869999999999999,"5min":10.945,"15min":10.945},"last":0.29499999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.401,"15min":0.84799999999999998},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":5.4690000000000003,"5min":5.798,"15min":5.798},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.4059999999999999,"15min":0.85999999999999999},"min":{"1min":0.437,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.4630000000000001,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.3580000000000001,"15min":0.84499999999999997},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5179999999999998,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":0.78400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2714376,"kb_used_data":2484736,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":91030776,"statfs":{"total":95995035648,"available":93215514624,"internally_reserved":0,"allocated":2544369664,"data_stored":2544086446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":235141574},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":2.0459999999999998,"15min":0.90200000000000002},"min":{"1min":0.20999999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":7.1600000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.163,"5min":2.226,"15min":0.97899999999999998},"min":{"1min":0.19,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":7.0419999999999998,"5min":23.341000000000001,"15min":23.341000000000001},"last":1.5820000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.575,"15min":0.77000000000000002},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":3.2829999999999999,"5min":6.1360000000000001,"15min":6.1360000000000001},"last":6.8879999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.4059999999999999,"15min":0.72799999999999998},"min":{"1min":0.22500000000000001,"5min":0.184,"15min":0.184},"max":{"1min":4.7930000000000001,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.754}]},{"osd":3,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":2.2370000000000001,"15min":1.0129999999999999},"min":{"1min":0.246,"5min":0.23499999999999999,"15min":0.20300000000000001},"max":{"1min":36.792999999999999,"5min":36.792999999999999,"15min":36.792999999999999},"last":1.353},{"interface":"front","average":{"1min":6.7859999999999996,"5min":2.9940000000000002,"15min":1.288},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":56.533000000000001,"5min":56.533000000000001,"15min":56.533000000000001},"last":1.1699999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":1.3320000000000001,"15min":0.80700000000000005},"min":{"1min":0.28999999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.4849999999999999,"5min":13.58,"15min":13.58},"last":1.5009999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.2430000000000001,"15min":0.78800000000000003},"min":{"1min":0.371,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.9329999999999998,"5min":13.818,"15min":13.818},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":1.1639999999999999,"15min":0.752},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.8280000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":1.2090000000000001,"15min":0.76100000000000001},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9910000000000001,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":1.6160000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.349,"15min":0.83599999999999997},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.9379999999999999,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3169999999999999,"15min":0.84699999999999998},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.4399999999999999,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.81200000000000006}]},{"osd":7,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.206,"15min":0.77700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6989999999999998,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":1.212,"15min":0.77400000000000002},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7210000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.476}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3145572,"kb_used_data":2897884,"kb_used_omap":1,"kb_used_meta":247678,"kb_avail":90599580,"statfs":{"total":95995035648,"available":92773969920,"internally_reserved":0,"allocated":2967433216,"data_stored":2967121624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253622726},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":2.1509999999999998,"15min":0.90900000000000003},"min":{"1min":0.25,"5min":0.16,"15min":0.16},"max":{"1min":5.8879999999999999,"5min":31.585999999999999,"15min":31.585999999999999},"last":1.762},{"interface":"front","average":{"1min":1.2589999999999999,"5min":2.3029999999999999,"15min":0.97399999999999998},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":7.1740000000000004,"5min":35.807000000000002,"15min":35.807000000000002},"last":0.498}]},{"osd":2,"last update":"Tue Sep 17 08:17:02 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.5169999999999999,"15min":0.73699999999999999},"min":{"1min":0.23999999999999999,"5min":0.20100000000000001,"15min":0.19700000000000001},"max":{"1min":4.7009999999999996,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.675,"15min":0.81899999999999995},"min":{"1min":0.28899999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.1740000000000004,"5min":10.638,"15min":10.638},"last":0.99299999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8460000000000001,"5min":1.6890000000000001,"15min":0.82699999999999996},"min":{"1min":0.26100000000000001,"5min":0.23300000000000001,"15min":0.20300000000000001},"max":{"1min":5.2759999999999998,"5min":7.8499999999999996,"15min":7.8499999999999996},"last":2.133},{"interface":"front","average":{"1min":1.696,"5min":1.6399999999999999,"15min":0.81499999999999995},"min":{"1min":0.35499999999999998,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":6.2759999999999998,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":3.9950000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.1950000000000001,"15min":0.755},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.708,"5min":7.4859999999999998,"15min":7.4859999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.121,"15min":0.73999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.194,"5min":10.208,"15min":10.208},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:17:07 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.169,"15min":0.746},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.0090000000000003,"5min":5.9249999999999998,"15min":5.9249999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.3,"5min":1.3300000000000001,"15min":0.80900000000000005},"min":{"1min":0.45600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.383,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.242,"15min":0.76700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.919,"5min":10.253,"15min":10.253},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.3109999999999999,"15min":0.82699999999999996},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2590000000000003,"5min":7.2590000000000003,"15min":7.2590000000000003},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.3069999999999999,"15min":0.81599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.6859999999999999,"5min":10.066000000000001,"15min":10.066000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.2569999999999999,"15min":0.81499999999999995},"min":{"1min":0.43099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":3.0339999999999998,"5min":10.279,"15min":10.279},"last":0.82999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2750480,"kb_used_data":2476552,"kb_used_omap":1,"kb_used_meta":273918,"kb_avail":90994672,"statfs":{"total":95995035648,"available":93178544128,"internally_reserved":0,"allocated":2535989248,"data_stored":2535698459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280492486},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":1.3520000000000001,"15min":0.66700000000000004},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.19900000000000001},"max":{"1min":2.1230000000000002,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.716,"15min":0.79900000000000004},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":3.7130000000000001,"5min":13.537000000000001,"15min":13.537000000000001},"last":1.1830000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.6799999999999999,"15min":0.81000000000000005},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.8100000000000001,"5min":12.355,"15min":12.355},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.6759999999999999,"15min":0.82599999999999996},"min":{"1min":0.216,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":8.0489999999999995,"5min":8.0489999999999995,"15min":8.0489999999999995},"last":1.4159999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6879999999999997,"5min":2.903,"15min":1.254},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.20999999999999999},"max":{"1min":61.926000000000002,"5min":61.926000000000002,"15min":61.926000000000002},"last":2.351},{"interface":"front","average":{"1min":5.9459999999999997,"5min":2.6840000000000002,"15min":1.194},"min":{"1min":0.32500000000000001,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":61.616,"5min":61.616,"15min":61.616},"last":1.389}]},{"osd":4,"last update":"Tue Sep 17 08:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":1.2729999999999999,"15min":0.79200000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.635,"5min":5.6890000000000001,"15min":5.6890000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.2689999999999999,"15min":0.81000000000000005},"min":{"1min":0.41199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.581,"5min":6.548,"15min":6.548},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1619999999999999,"15min":0.76200000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":5.5499999999999998,"5min":7.835,"15min":7.835},"last":1.4390000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":1.3720000000000001,"15min":0.83899999999999997},"min":{"1min":0.435,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":6.8789999999999996,"15min":6.8789999999999996},"last":1.0720000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.1040000000000001,"15min":0.75700000000000001},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.4470000000000001,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":1.478},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.321,"15min":0.83099999999999996},"min":{"1min":0.36799999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.6619999999999999,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.437}]},{"osd":7,"last update":"Tue Sep 17 08:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":1.706,"15min":0.94699999999999995},"min":{"1min":0.38100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.861,"5min":31.585999999999999,"15min":31.585999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.754,"5min":1.5309999999999999,"15min":0.89600000000000002},"min":{"1min":0.42999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":2.3519999999999999,"5min":30.225000000000001,"15min":30.225000000000001},"last":1.4590000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2553544704,"data_stored":2553544704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2971402240,"data_stored":2971402240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2554068992,"data_stored":2554068992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2545156096,"data_stored":2545156096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2117402624,"data_stored":2117402624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3405447168,"data_stored":3405447168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2112421888,"data_stored":2112421888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2107768832,"data_stored":2107768832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:17:37.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:17:37.524 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:37.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 68 16 109422 109406 100.538 76.5625 0.00758354 0.00994142 2024-09-17T08:17:37.554 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:37.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:37.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:37.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:37.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:17:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:37 smithi173 ceph-mon[33187]: pgmap v330: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:37.805 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:37.839 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:37.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[40649]: pgmap v330: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:17:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:17:37.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:37 smithi079 ceph-mon[29565]: pgmap v330: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:17:37.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:37.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:37.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:37.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:38.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:38.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:38.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:38.216 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:38.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:38.251 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:38.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:38.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:38.399 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:38.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:38.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:38.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 69 16 110532 110516 100.086 69.375 0.0153276 0.00998641 2024-09-17T08:17:38.527 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:38.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:38.566 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:17:38.567 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:17:38.568 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:38 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:17:38.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:38.673 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:38.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:17:38.771 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:17:38.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:17:38.772 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:38 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:17:38.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:38.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:38.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:38.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:38.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:39.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:39.085 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:39.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:39.187 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:39.204 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:17:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:111320 8.9 e189/165 8:95a7d5c6:::benchmark_data_smithi079_115040_object111319:head v 189'6981, mlcod=189'6978)", 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:39.199932+0000", 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.003707134, 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037356490000000002, 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.199932+0000", 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.199932+0000", 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.199931+0000", 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.199998+0000", 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7172e-05 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.200014+0000", 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5460000000000001e-05 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.200021+0000", 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5809999999999999e-06 2024-09-17T08:17:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.200100+0000", 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8738000000000004e-05 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.200120+0000", 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9741e-05 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:111323 8.8 e189/165 8:1b2944ed:::benchmark_data_smithi079_115040_object111322:head v 189'6982, mlcod=189'6980)", 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:39.202323+0000", 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0013160750000000001, 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014043479999999999, 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202323+0000", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202323+0000", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202322+0000", 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202401+0000", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8684999999999996e-05 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202427+0000", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6485000000000001e-05 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202435+0000", 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7309999999999993e-06 2024-09-17T08:17:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202531+0000", 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6100999999999994e-05 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:39.202583+0000", 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1731999999999999e-05 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:17:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 2 2024-09-17T08:17:39.213 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:39.223 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:39.270 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:39.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:39.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.359 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:39.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:39.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:39.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:39.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:39.509 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:17:39.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 70 16 111715 111699 99.7127 73.9375 0.00843341 0.010024 2024-09-17T08:17:39.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:39.634 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:39.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:39.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:39.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.772 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:39.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[40649]: pgmap v331: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:17:39.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:39.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:39 smithi079 ceph-mon[29565]: pgmap v331: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:39.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:39.926 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:39.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:39.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:39 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:17:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:39 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:17:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:17:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:39 smithi173 ceph-mon[33187]: pgmap v331: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:40.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:40.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:40.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:40.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:17:40.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:40.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.202 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:40.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:40.340 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:40.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:40.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:40.471 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:40.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:40.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 71 16 112933 112917 99.3805 76.125 0.00715241 0.0100574 2024-09-17T08:17:40.532 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:40.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:40.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:40.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:40.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:40.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:40.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:40.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:17:40.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:17:40.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:17:40.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:17:40.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:40.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:40 smithi173 ceph-mon[33187]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T08:17:40.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:17:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[40649]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/419139084' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:17:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:40 smithi079 ceph-mon[29565]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T08:17:40.893 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:40.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:40.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:40.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:41.035 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:41.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:41.137 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:41.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:41.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:41.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:41.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.312 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:41.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:41.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:41.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.446 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:41.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:41.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 72 16 114139 114123 99.047 75.375 0.00643454 0.0100917 2024-09-17T08:17:41.548 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:41.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:41.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:41.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:41.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.709 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:41.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:41.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:41.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.849 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:41.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:41.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 8.d scrub starts 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:17:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:17:41.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[29565]: pgmap v333: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:41.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:17:41.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:17:41.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 8.d scrub starts 2024-09-17T08:17:41.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:17:41.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:17:41.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:17:41.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:17:41.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:41 smithi079 ceph-mon[40649]: pgmap v333: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:41.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:41.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:41.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:41.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 8.d scrub starts 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:17:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:41 smithi173 ceph-mon[33187]: pgmap v333: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:42.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:42.109 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:42.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:42.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:42.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:42.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:42.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:42.382 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:42.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:42.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:42.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 73 16 115315 115299 98.697 73.5 0.00504524 0.0101275 2024-09-17T08:17:42.529 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:42.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:42.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:42.663 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:42.663 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:42 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:17:42.664 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:42 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:17:42.665 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:42.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:42.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:42.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:42 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:17:42.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:42 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:17:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:42 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:17:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:42 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:17:42.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:42.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:42.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:42.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:43.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:17:43.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:43.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.075 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:43.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:43.137 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:43.138 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:17:43.138 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:17:43.138 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:17:43.138 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:43.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:43.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:43.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:43.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:43.371 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:43.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:43.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:43.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:43.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:43.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:43.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 74 16 116508 116492 98.3708 74.5625 0.0232315 0.0101612 2024-09-17T08:17:43.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:43.645 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:43.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:43.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:43.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.776 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:43.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:43.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:43 smithi173 ceph-mon[33187]: 8.d scrub ok 2024-09-17T08:17:43.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:43 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:17:43.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:43 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:17:43.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:43.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:43 smithi173 ceph-mon[33187]: pgmap v334: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:43.837 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:17:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[29565]: 8.d scrub ok 2024-09-17T08:17:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:17:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[29565]: pgmap v334: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[40649]: 8.d scrub ok 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:43 smithi079 ceph-mon[40649]: pgmap v334: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:43.877 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:43.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:43.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:43.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:44.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:44.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:44.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:44.118 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:44.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:44.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:44.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:44.284 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:44.317 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:44.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:44.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:44.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:44.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:44.526 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 75 15 117523 117508 97.9058 63.5 0.00601823 0.0102099 2024-09-17T08:17:44.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:44.592 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:44.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:44.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:44.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:44.797 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:44 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:17:44.797 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:44 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:17:44.797 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:44 smithi173 ceph-mon[33187]: 5.8 deep-scrub starts 2024-09-17T08:17:44.797 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:44 smithi173 ceph-mon[33187]: 5.8 deep-scrub ok 2024-09-17T08:17:44.797 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:44 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/801089562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:44.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:44.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:44.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:44.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:17:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:17:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[40649]: 5.8 deep-scrub starts 2024-09-17T08:17:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[40649]: 5.8 deep-scrub ok 2024-09-17T08:17:44.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/801089562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:17:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:17:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[29565]: 5.8 deep-scrub starts 2024-09-17T08:17:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[29565]: 5.8 deep-scrub ok 2024-09-17T08:17:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:44 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/801089562' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:44.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:45.003 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:17:45.006 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:17:45.007 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:17:45.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:45.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:45.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:45.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:45.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:45.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:45.527 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 76 16 118741 118725 97.6183 76.0625 0.0159325 0.0102394 2024-09-17T08:17:45.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:45.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:45.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:45.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:45.841 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:45.847 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:17:45.847 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:17:45.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 5.2 deep-scrub starts 2024-09-17T08:17:45.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 5.2 deep-scrub ok 2024-09-17T08:17:45.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:17:45.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:17:45.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:45 smithi173 ceph-mon[33187]: pgmap v335: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:17:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:17:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 5.2 deep-scrub starts 2024-09-17T08:17:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 5.2 deep-scrub ok 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[29565]: pgmap v335: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 5.2 deep-scrub starts 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 5.2 deep-scrub ok 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:17:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:45 smithi079 ceph-mon[40649]: pgmap v335: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:45.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:46.004 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:46.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:46.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:46.153 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:46.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:46.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:46.256 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:17:46.256 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:17:46.302 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:46.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.303 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:46.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:46.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:46.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:46.519 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:46.527 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 77 16 119982 119966 97.3578 77.5625 0.0115339 0.0102666 2024-09-17T08:17:46.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:46.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:46.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:46.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:46.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:46.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:46.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:46 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:17:46.843 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:46 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:17:46.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:46 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:17:46.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:46 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:17:46.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:46.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:46.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:46.899 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:46 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:17:46.900 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:46 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:17:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:17:46.907 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:17:46.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:17:46.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:47.018 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:47.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:47.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:47.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:47.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:47.250 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:47.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:47.294 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:47.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:47.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:47.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:47.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:47.527 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 78 16 121153 121137 97.0479 73.1875 0.0205305 0.0103001 2024-09-17T08:17:47.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:47.567 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:47.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:47.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:47.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:47.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:47.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:47.835 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:47.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:47.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:17:47.937 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:47.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:47.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:47.967 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:48.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:48.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 1.0 deep-scrub starts 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: 1.0 deep-scrub ok 2024-09-17T08:17:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[40649]: pgmap v336: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 1.0 deep-scrub starts 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: 1.0 deep-scrub ok 2024-09-17T08:17:48.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:47 smithi079 ceph-mon[29565]: pgmap v336: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:48.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.135 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:48.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:48.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 1.0 deep-scrub starts 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: 1.0 deep-scrub ok 2024-09-17T08:17:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:47 smithi173 ceph-mon[33187]: pgmap v336: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:48.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:48.277 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:48.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:48.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:48.404 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:48.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:48.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:48.527 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 79 16 122381 122365 96.7909 76.75 0.00754221 0.0103269 2024-09-17T08:17:48.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.542 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:48.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:48.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:48.645 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-17T08:17:45.356773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7573412118,"num_objects":115748,"num_object_clones":0,"num_object_copies":347244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115748,"num_whiteouts":0,"num_read":1053,"num_read_kb":899,"num_write":116161,"num_write_kb":7397787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":1776389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84622,"ondisk_log_size":84622,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23805352,"kb_used_data":22033892,"kb_used_omap":12,"kb_used_meta":1771379,"kb_avail":726155864,"statfs":{"total":767960285184,"available":743583604736,"internally_reserved":0,"allocated":22562705408,"data_stored":22560338394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":1813892664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,7,3,0,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":774832128,"num_objects":11823,"num_object_clones":0,"num_object_copies":35469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11823,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11823,"num_write_kb":756672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002835"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:34.931931+0000","last_change":"2024-09-17T08:17:34.931931+0000","last_active":"2024-09-17T08:17:34.931931+0000","last_peered":"2024-09-17T08:17:34.931931+0000","last_clean":"2024-09-17T08:17:34.931931+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:17:34.931931+0000","last_undegraded":"2024-09-17T08:17:34.931931+0000","last_fullsized":"2024-09-17T08:17:34.931931+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.931732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.931732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.380258+0000","scrub_duration":0.0066078860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429924+0000","last_change":"2024-09-17T08:16:42.337270+0000","last_active":"2024-09-17T08:17:30.429924+0000","last_peered":"2024-09-17T08:17:30.429924+0000","last_clean":"2024-09-17T08:17:30.429924+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:17:30.429924+0000","last_undegraded":"2024-09-17T08:17:30.429924+0000","last_fullsized":"2024-09-17T08:17:30.429924+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.337157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:08.710909+0000","scrub_duration":0.001642129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":183,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436667+0000","last_change":"2024-09-17T08:14:56.372401+0000","last_active":"2024-09-17T08:17:30.436667+0000","last_peered":"2024-09-17T08:17:30.436667+0000","last_clean":"2024-09-17T08:17:30.436667+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:17:30.436667+0000","last_undegraded":"2024-09-17T08:17:30.436667+0000","last_fullsized":"2024-09-17T08:17:30.436667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:14:56.372270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:56.372270+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010005975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":185,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.419995+0000","last_change":"2024-09-17T08:15:52.984443+0000","last_active":"2024-09-17T08:17:30.419995+0000","last_peered":"2024-09-17T08:17:30.419995+0000","last_clean":"2024-09-17T08:17:30.419995+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:17:30.419995+0000","last_undegraded":"2024-09-17T08:17:30.419995+0000","last_fullsized":"2024-09-17T08:17:30.419995+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.984337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.984337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030030090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":160,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436315+0000","last_change":"2024-09-17T08:15:43.592053+0000","last_active":"2024-09-17T08:17:30.436315+0000","last_peered":"2024-09-17T08:17:30.436315+0000","last_clean":"2024-09-17T08:17:30.436315+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:17:30.436315+0000","last_undegraded":"2024-09-17T08:17:30.436315+0000","last_fullsized":"2024-09-17T08:17:30.436315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:15:43.591667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013131347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":265,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420031+0000","last_change":"2024-09-17T08:16:35.827557+0000","last_active":"2024-09-17T08:17:30.420031+0000","last_peered":"2024-09-17T08:17:30.420031+0000","last_clean":"2024-09-17T08:17:30.420031+0000","last_became_active":"2024-09-17T08:16:33.326106+0000","last_became_peered":"2024-09-17T08:16:33.326106+0000","last_unstale":"2024-09-17T08:17:30.420031+0000","last_undegraded":"2024-09-17T08:17:30.420031+0000","last_fullsized":"2024-09-17T08:17:30.420031+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:16:35.827443+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.827443+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:39.548050+0000","scrub_duration":0.0033149960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431834+0000","last_change":"2024-09-17T08:16:44.335176+0000","last_active":"2024-09-17T08:17:30.431834+0000","last_peered":"2024-09-17T08:17:30.431834+0000","last_clean":"2024-09-17T08:17:30.431834+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:17:30.431834+0000","last_undegraded":"2024-09-17T08:17:30.431834+0000","last_fullsized":"2024-09-17T08:17:30.431834+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.335057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.335057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:49.556092+0000","scrub_duration":0.0065690419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":194,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420245+0000","last_change":"2024-09-17T08:16:48.900627+0000","last_active":"2024-09-17T08:17:30.420245+0000","last_peered":"2024-09-17T08:17:30.420245+0000","last_clean":"2024-09-17T08:17:30.420245+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:17:30.420245+0000","last_undegraded":"2024-09-17T08:17:30.420245+0000","last_fullsized":"2024-09-17T08:17:30.420245+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.900525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.943918+0000","scrub_duration":0.0033962240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":171,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.428381+0000","last_change":"2024-09-17T08:16:33.826157+0000","last_active":"2024-09-17T08:17:30.428381+0000","last_peered":"2024-09-17T08:17:30.428381+0000","last_clean":"2024-09-17T08:17:30.428381+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:17:30.428381+0000","last_undegraded":"2024-09-17T08:17:30.428381+0000","last_fullsized":"2024-09-17T08:17:30.428381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.826006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.826006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.282923+0000","scrub_duration":0.0026989420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":211,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420482+0000","last_change":"2024-09-17T08:15:31.963456+0000","last_active":"2024-09-17T08:17:30.420482+0000","last_peered":"2024-09-17T08:17:30.420482+0000","last_clean":"2024-09-17T08:17:30.420482+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:17:30.420482+0000","last_undegraded":"2024-09-17T08:17:30.420482+0000","last_fullsized":"2024-09-17T08:17:30.420482+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:15:31.963328+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:15:31.963328+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040865249999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":196,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427289+0000","last_change":"2024-09-17T08:16:49.012348+0000","last_active":"2024-09-17T08:17:30.427289+0000","last_peered":"2024-09-17T08:17:30.427289+0000","last_clean":"2024-09-17T08:17:30.427289+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:17:30.427289+0000","last_undegraded":"2024-09-17T08:17:30.427289+0000","last_fullsized":"2024-09-17T08:17:30.427289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.012248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.718200+0000","scrub_duration":0.0070975750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431537+0000","last_change":"2024-09-17T08:16:23.065070+0000","last_active":"2024-09-17T08:17:30.431537+0000","last_peered":"2024-09-17T08:17:30.431537+0000","last_clean":"2024-09-17T08:17:30.431537+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:17:30.431537+0000","last_undegraded":"2024-09-17T08:17:30.431537+0000","last_fullsized":"2024-09-17T08:17:30.431537+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.064993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.064993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015683839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.428014+0000","last_change":"2024-09-17T08:17:01.908852+0000","last_active":"2024-09-17T08:17:30.428014+0000","last_peered":"2024-09-17T08:17:30.428014+0000","last_clean":"2024-09-17T08:17:30.428014+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:17:30.428014+0000","last_undegraded":"2024-09-17T08:17:30.428014+0000","last_fullsized":"2024-09-17T08:17:30.428014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.908732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.507561+0000","scrub_duration":0.0085101169999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":196,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420354+0000","last_change":"2024-09-17T08:16:46.894268+0000","last_active":"2024-09-17T08:17:30.420354+0000","last_peered":"2024-09-17T08:17:30.420354+0000","last_clean":"2024-09-17T08:17:30.420354+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:17:30.420354+0000","last_undegraded":"2024-09-17T08:17:30.420354+0000","last_fullsized":"2024-09-17T08:17:30.420354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:16:46.894134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.894134+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.434020+0000","scrub_duration":0.0053467080000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":70,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.072019+0000","last_change":"2024-09-17T08:15:54.069982+0000","last_active":"2024-09-17T08:17:33.072019+0000","last_peered":"2024-09-17T08:17:33.072019+0000","last_clean":"2024-09-17T08:17:33.072019+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:17:33.072019+0000","last_undegraded":"2024-09-17T08:17:33.072019+0000","last_fullsized":"2024-09-17T08:17:33.072019+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.069886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.069886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001445294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":194,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.421060+0000","last_change":"2024-09-17T08:17:31.421060+0000","last_active":"2024-09-17T08:17:31.421060+0000","last_peered":"2024-09-17T08:17:31.421060+0000","last_clean":"2024-09-17T08:17:31.421060+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:17:31.421060+0000","last_undegraded":"2024-09-17T08:17:31.421060+0000","last_fullsized":"2024-09-17T08:17:31.421060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.420934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.420934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.381148+0000","scrub_duration":0.0086368369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"190'7241","reported_seq":7275,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.553618+0000","last_change":"2024-09-17T08:16:16.645652+0000","last_active":"2024-09-17T08:17:42.553618+0000","last_peered":"2024-09-17T08:17:42.553618+0000","last_clean":"2024-09-17T08:17:42.553618+0000","last_became_active":"2024-09-17T08:16:16.645277+0000","last_became_peered":"2024-09-17T08:16:16.645277+0000","last_unstale":"2024-09-17T08:17:42.553618+0000","last_undegraded":"2024-09-17T08:17:42.553618+0000","last_fullsized":"2024-09-17T08:17:42.553618+0000","mapping_epoch":165,"log_start":"176'2200","ondisk_log_start":"176'2200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5041,"log_dups_size":0,"ondisk_log_size":5041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474546176,"num_objects":7241,"num_object_clones":0,"num_object_copies":21723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":463424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":251,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.421068+0000","last_change":"2024-09-17T08:16:26.873516+0000","last_active":"2024-09-17T08:17:30.421068+0000","last_peered":"2024-09-17T08:17:30.421068+0000","last_clean":"2024-09-17T08:17:30.421068+0000","last_became_active":"2024-09-17T08:15:29.362201+0000","last_became_peered":"2024-09-17T08:15:29.362201+0000","last_unstale":"2024-09-17T08:17:30.421068+0000","last_undegraded":"2024-09-17T08:17:30.421068+0000","last_fullsized":"2024-09-17T08:17:30.421068+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:16:26.873402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.873402+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:39.799533+0000","scrub_duration":0.002741856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:36.164173+0000","last_change":"2024-09-17T08:17:36.164173+0000","last_active":"2024-09-17T08:17:36.164173+0000","last_peered":"2024-09-17T08:17:36.164173+0000","last_clean":"2024-09-17T08:17:36.164173+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:17:36.164173+0000","last_undegraded":"2024-09-17T08:17:36.164173+0000","last_fullsized":"2024-09-17T08:17:36.164173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.164053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.164053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:45.324794+0000","scrub_duration":0.0014889840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431408+0000","last_change":"2024-09-17T08:15:48.924108+0000","last_active":"2024-09-17T08:17:30.431408+0000","last_peered":"2024-09-17T08:17:30.431408+0000","last_clean":"2024-09-17T08:17:30.431408+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:17:30.431408+0000","last_undegraded":"2024-09-17T08:17:30.431408+0000","last_fullsized":"2024-09-17T08:17:30.431408+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.923947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.923947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039645160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431156+0000","last_change":"2024-09-17T08:16:26.262129+0000","last_active":"2024-09-17T08:17:30.431156+0000","last_peered":"2024-09-17T08:17:30.431156+0000","last_clean":"2024-09-17T08:17:30.431156+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:17:30.431156+0000","last_undegraded":"2024-09-17T08:17:30.431156+0000","last_fullsized":"2024-09-17T08:17:30.431156+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:26.262049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.262049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001547122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"190'7160","reported_seq":7193,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.849644+0000","last_change":"2024-09-17T08:16:16.648796+0000","last_active":"2024-09-17T08:17:42.849644+0000","last_peered":"2024-09-17T08:17:42.849644+0000","last_clean":"2024-09-17T08:17:42.849644+0000","last_became_active":"2024-09-17T08:16:16.648526+0000","last_became_peered":"2024-09-17T08:16:16.648526+0000","last_unstale":"2024-09-17T08:17:42.849644+0000","last_undegraded":"2024-09-17T08:17:42.849644+0000","last_fullsized":"2024-09-17T08:17:42.849644+0000","mapping_epoch":165,"log_start":"176'2200","ondisk_log_start":"176'2200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469237760,"num_objects":7160,"num_object_clones":0,"num_object_copies":21480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7160,"num_write_kb":458240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":195,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420918+0000","last_change":"2024-09-17T08:16:56.688647+0000","last_active":"2024-09-17T08:17:30.420918+0000","last_peered":"2024-09-17T08:17:30.420918+0000","last_clean":"2024-09-17T08:17:30.420918+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:17:30.420918+0000","last_undegraded":"2024-09-17T08:17:30.420918+0000","last_fullsized":"2024-09-17T08:17:30.420918+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:56.688373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.688373+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.791670+0000","scrub_duration":0.0062131820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":61,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:40.659072+0000","last_change":"2024-09-17T08:17:40.659072+0000","last_active":"2024-09-17T08:17:40.659072+0000","last_peered":"2024-09-17T08:17:40.659072+0000","last_clean":"2024-09-17T08:17:40.659072+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:17:40.659072+0000","last_undegraded":"2024-09-17T08:17:40.659072+0000","last_fullsized":"2024-09-17T08:17:40.659072+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.658816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:51.813043+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.658816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:47.720191+0000","scrub_duration":0.011311672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.415178+0000","last_change":"2024-09-17T08:17:42.416017+0000","last_active":"2024-09-17T08:17:42.415178+0000","last_peered":"2024-09-17T08:17:42.415178+0000","last_clean":"2024-09-17T08:17:42.415178+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:17:42.415178+0000","last_undegraded":"2024-09-17T08:17:42.415178+0000","last_fullsized":"2024-09-17T08:17:42.415178+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:17:42.415151+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:17:42.415151+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.003958+0000","scrub_duration":0.0143199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":183,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.421300+0000","last_change":"2024-09-17T08:16:30.856483+0000","last_active":"2024-09-17T08:17:30.421300+0000","last_peered":"2024-09-17T08:17:30.421300+0000","last_clean":"2024-09-17T08:17:30.421300+0000","last_became_active":"2024-09-17T08:15:29.358693+0000","last_became_peered":"2024-09-17T08:15:29.358693+0000","last_unstale":"2024-09-17T08:17:30.421300+0000","last_undegraded":"2024-09-17T08:17:30.421300+0000","last_fullsized":"2024-09-17T08:17:30.421300+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:30.856368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.856368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.067757+0000","scrub_duration":0.0044042839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"190'7108","reported_seq":7142,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.555501+0000","last_change":"2024-09-17T08:16:16.648125+0000","last_active":"2024-09-17T08:17:42.555501+0000","last_peered":"2024-09-17T08:17:42.555501+0000","last_clean":"2024-09-17T08:17:42.555501+0000","last_became_active":"2024-09-17T08:16:16.647816+0000","last_became_peered":"2024-09-17T08:16:16.647816+0000","last_unstale":"2024-09-17T08:17:42.555501+0000","last_undegraded":"2024-09-17T08:17:42.555501+0000","last_fullsized":"2024-09-17T08:17:42.555501+0000","mapping_epoch":165,"log_start":"176'2100","ondisk_log_start":"176'2100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465829888,"num_objects":7108,"num_object_clones":0,"num_object_copies":21324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7108,"num_write_kb":454912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":55,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.430976+0000","last_change":"2024-09-17T08:16:53.116193+0000","last_active":"2024-09-17T08:17:30.430976+0000","last_peered":"2024-09-17T08:17:30.430976+0000","last_clean":"2024-09-17T08:17:30.430976+0000","last_became_active":"2024-09-17T08:16:33.328511+0000","last_became_peered":"2024-09-17T08:16:33.328511+0000","last_unstale":"2024-09-17T08:17:30.430976+0000","last_undegraded":"2024-09-17T08:17:30.430976+0000","last_fullsized":"2024-09-17T08:17:30.430976+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:16:53.115891+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:16:53.115891+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.168215+0000","scrub_duration":0.0043675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":61,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.503959+0000","last_change":"2024-09-17T08:16:40.302468+0000","last_active":"2024-09-17T08:17:39.503959+0000","last_peered":"2024-09-17T08:17:39.503959+0000","last_clean":"2024-09-17T08:17:39.503959+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:17:39.503959+0000","last_undegraded":"2024-09-17T08:17:39.503959+0000","last_fullsized":"2024-09-17T08:17:39.503959+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.302235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.302235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.381029+0000","scrub_duration":0.0071342690000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":268,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.914312+0000","last_change":"2024-09-17T08:15:40.042590+0000","last_active":"2024-09-17T08:17:37.914312+0000","last_peered":"2024-09-17T08:17:37.914312+0000","last_clean":"2024-09-17T08:17:37.914312+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:17:37.914312+0000","last_undegraded":"2024-09-17T08:17:37.914312+0000","last_fullsized":"2024-09-17T08:17:37.914312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:15:40.042507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:40.042507+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048815059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.432048+0000","last_change":"2024-09-17T08:16:56.421870+0000","last_active":"2024-09-17T08:17:30.432048+0000","last_peered":"2024-09-17T08:17:30.432048+0000","last_clean":"2024-09-17T08:17:30.432048+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:17:30.432048+0000","last_undegraded":"2024-09-17T08:17:30.432048+0000","last_fullsized":"2024-09-17T08:17:30.432048+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:56.421736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.421736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.576854+0000","scrub_duration":0.0048071169999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"190'7319","reported_seq":7352,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.341270+0000","last_change":"2024-09-17T08:16:16.649421+0000","last_active":"2024-09-17T08:17:42.341270+0000","last_peered":"2024-09-17T08:17:42.341270+0000","last_clean":"2024-09-17T08:17:42.341270+0000","last_became_active":"2024-09-17T08:16:16.648969+0000","last_became_peered":"2024-09-17T08:16:16.648969+0000","last_unstale":"2024-09-17T08:17:42.341270+0000","last_undegraded":"2024-09-17T08:17:42.341270+0000","last_fullsized":"2024-09-17T08:17:42.341270+0000","mapping_epoch":165,"log_start":"176'2000","ondisk_log_start":"176'2000","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5319,"log_dups_size":0,"ondisk_log_size":5319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479657984,"num_objects":7319,"num_object_clones":0,"num_object_copies":21957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7319,"num_write_kb":468416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"156'16","reported_seq":235,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.421425+0000","last_change":"2024-09-17T08:15:44.029509+0000","last_active":"2024-09-17T08:17:30.421425+0000","last_peered":"2024-09-17T08:17:30.421425+0000","last_clean":"2024-09-17T08:17:30.421425+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:17:30.421425+0000","last_undegraded":"2024-09-17T08:17:30.421425+0000","last_fullsized":"2024-09-17T08:17:30.421425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'16","last_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.029355+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011380917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.428965+0000","last_change":"2024-09-17T08:16:37.745942+0000","last_active":"2024-09-17T08:17:30.428965+0000","last_peered":"2024-09-17T08:17:30.428965+0000","last_clean":"2024-09-17T08:17:30.428965+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:17:30.428965+0000","last_undegraded":"2024-09-17T08:17:30.428965+0000","last_fullsized":"2024-09-17T08:17:30.428965+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.745837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.563608+0000","scrub_duration":0.0044495799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420830+0000","last_change":"2024-09-17T08:16:25.867376+0000","last_active":"2024-09-17T08:17:30.420830+0000","last_peered":"2024-09-17T08:17:30.420830+0000","last_clean":"2024-09-17T08:17:30.420830+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:17:30.420830+0000","last_undegraded":"2024-09-17T08:17:30.420830+0000","last_fullsized":"2024-09-17T08:17:30.420830+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:25.867267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:25.867267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.318499+0000","scrub_duration":0.0015224139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":165,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.909610+0000","last_change":"2024-09-17T08:17:39.909841+0000","last_active":"2024-09-17T08:17:39.909610+0000","last_peered":"2024-09-17T08:17:39.909610+0000","last_clean":"2024-09-17T08:17:39.909610+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:17:39.909610+0000","last_undegraded":"2024-09-17T08:17:39.909610+0000","last_fullsized":"2024-09-17T08:17:39.909610+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.909586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.909586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.911382+0000","scrub_duration":0.001911444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"190'7206","reported_seq":7395,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.302808+0000","last_change":"2024-09-17T08:17:41.995541+0000","last_active":"2024-09-17T08:17:42.302808+0000","last_peered":"2024-09-17T08:17:42.302808+0000","last_clean":"2024-09-17T08:17:42.302808+0000","last_became_active":"2024-09-17T08:16:16.648937+0000","last_became_peered":"2024-09-17T08:16:16.648937+0000","last_unstale":"2024-09-17T08:17:42.302808+0000","last_undegraded":"2024-09-17T08:17:42.302808+0000","last_fullsized":"2024-09-17T08:17:42.302808+0000","mapping_epoch":165,"log_start":"176'2200","ondisk_log_start":"176'2200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'7187","last_scrub_stamp":"2024-09-17T08:17:41.995394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.995394+0000","objects_scrubbed":7135,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:50.208844+0000","scrub_duration":1.840802743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472252416,"num_objects":7206,"num_object_clones":0,"num_object_copies":21618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7206,"num_write_kb":461184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":190,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420328+0000","last_change":"2024-09-17T08:16:27.874195+0000","last_active":"2024-09-17T08:17:30.420328+0000","last_peered":"2024-09-17T08:17:30.420328+0000","last_clean":"2024-09-17T08:17:30.420328+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:17:30.420328+0000","last_undegraded":"2024-09-17T08:17:30.420328+0000","last_fullsized":"2024-09-17T08:17:30.420328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:16:27.874083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.874083+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:45.441731+0000","scrub_duration":0.0022658510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.428055+0000","last_change":"2024-09-17T08:16:42.793996+0000","last_active":"2024-09-17T08:17:30.428055+0000","last_peered":"2024-09-17T08:17:30.428055+0000","last_clean":"2024-09-17T08:17:30.428055+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:17:30.428055+0000","last_undegraded":"2024-09-17T08:17:30.428055+0000","last_fullsized":"2024-09-17T08:17:30.428055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.793906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.082126+0000","scrub_duration":0.0013736460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":300,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.922760+0000","last_change":"2024-09-17T08:16:48.678053+0000","last_active":"2024-09-17T08:17:37.922760+0000","last_peered":"2024-09-17T08:17:37.922760+0000","last_clean":"2024-09-17T08:17:37.922760+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:17:37.922760+0000","last_undegraded":"2024-09-17T08:17:37.922760+0000","last_fullsized":"2024-09-17T08:17:37.922760+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.677913+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.360519+0000","scrub_duration":0.032798792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427439+0000","last_change":"2024-09-17T08:16:47.964627+0000","last_active":"2024-09-17T08:17:30.427439+0000","last_peered":"2024-09-17T08:17:30.427439+0000","last_clean":"2024-09-17T08:17:30.427439+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:17:30.427439+0000","last_undegraded":"2024-09-17T08:17:30.427439+0000","last_fullsized":"2024-09-17T08:17:30.427439+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.964546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.603179+0000","scrub_duration":0.0015627130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"190'7144","reported_seq":7178,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:43.087371+0000","last_change":"2024-09-17T08:16:16.404499+0000","last_active":"2024-09-17T08:17:43.087371+0000","last_peered":"2024-09-17T08:17:43.087371+0000","last_clean":"2024-09-17T08:17:43.087371+0000","last_became_active":"2024-09-17T08:16:16.404227+0000","last_became_peered":"2024-09-17T08:16:16.404227+0000","last_unstale":"2024-09-17T08:17:43.087371+0000","last_undegraded":"2024-09-17T08:17:43.087371+0000","last_fullsized":"2024-09-17T08:17:43.087371+0000","mapping_epoch":165,"log_start":"173'1400","ondisk_log_start":"173'1400","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5744,"log_dups_size":0,"ondisk_log_size":5744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468189184,"num_objects":7144,"num_object_clones":0,"num_object_copies":21432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7144,"num_write_kb":457216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":201,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.437311+0000","last_change":"2024-09-17T08:15:52.770015+0000","last_active":"2024-09-17T08:17:30.437311+0000","last_peered":"2024-09-17T08:17:30.437311+0000","last_clean":"2024-09-17T08:17:30.437311+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:17:30.437311+0000","last_undegraded":"2024-09-17T08:17:30.437311+0000","last_fullsized":"2024-09-17T08:17:30.437311+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:15:52.769743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.769743+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.023040108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.503676+0000","last_change":"2024-09-17T08:17:02.482908+0000","last_active":"2024-09-17T08:17:39.503676+0000","last_peered":"2024-09-17T08:17:39.503676+0000","last_clean":"2024-09-17T08:17:39.503676+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:17:39.503676+0000","last_undegraded":"2024-09-17T08:17:39.503676+0000","last_fullsized":"2024-09-17T08:17:39.503676+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:02.482796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.482796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:24.312024+0000","scrub_duration":0.006488972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":167,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420175+0000","last_change":"2024-09-17T08:15:48.938396+0000","last_active":"2024-09-17T08:17:30.420175+0000","last_peered":"2024-09-17T08:17:30.420175+0000","last_clean":"2024-09-17T08:17:30.420175+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:17:30.420175+0000","last_undegraded":"2024-09-17T08:17:30.420175+0000","last_fullsized":"2024-09-17T08:17:30.420175+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.938247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.938247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00524202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420315+0000","last_change":"2024-09-17T08:15:35.024756+0000","last_active":"2024-09-17T08:17:30.420315+0000","last_peered":"2024-09-17T08:17:30.420315+0000","last_clean":"2024-09-17T08:17:30.420315+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:17:30.420315+0000","last_undegraded":"2024-09-17T08:17:30.420315+0000","last_fullsized":"2024-09-17T08:17:30.420315+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:35.024623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:35.024623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010817126999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"190'7241","reported_seq":7275,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.808500+0000","last_change":"2024-09-17T08:16:16.647250+0000","last_active":"2024-09-17T08:17:42.808500+0000","last_peered":"2024-09-17T08:17:42.808500+0000","last_clean":"2024-09-17T08:17:42.808500+0000","last_became_active":"2024-09-17T08:16:16.646899+0000","last_became_peered":"2024-09-17T08:16:16.646899+0000","last_unstale":"2024-09-17T08:17:42.808500+0000","last_undegraded":"2024-09-17T08:17:42.808500+0000","last_fullsized":"2024-09-17T08:17:42.808500+0000","mapping_epoch":165,"log_start":"176'2300","ondisk_log_start":"176'2300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474546176,"num_objects":7241,"num_object_clones":0,"num_object_copies":21723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":463424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":216,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429090+0000","last_change":"2024-09-17T08:16:48.235582+0000","last_active":"2024-09-17T08:17:30.429090+0000","last_peered":"2024-09-17T08:17:30.429090+0000","last_clean":"2024-09-17T08:17:30.429090+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:17:30.429090+0000","last_undegraded":"2024-09-17T08:17:30.429090+0000","last_fullsized":"2024-09-17T08:17:30.429090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:16:48.235424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.235424+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.405506+0000","scrub_duration":0.0061761419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":187,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:35.141768+0000","last_change":"2024-09-17T08:17:35.141768+0000","last_active":"2024-09-17T08:17:35.141768+0000","last_peered":"2024-09-17T08:17:35.141768+0000","last_clean":"2024-09-17T08:17:35.141768+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:17:35.141768+0000","last_undegraded":"2024-09-17T08:17:35.141768+0000","last_fullsized":"2024-09-17T08:17:35.141768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:17:35.141551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.141551+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.706752+0000","scrub_duration":0.0071014449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":123,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.423319+0000","last_change":"2024-09-17T08:15:46.091387+0000","last_active":"2024-09-17T08:17:30.423319+0000","last_peered":"2024-09-17T08:17:30.423319+0000","last_clean":"2024-09-17T08:17:30.423319+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:17:30.423319+0000","last_undegraded":"2024-09-17T08:17:30.423319+0000","last_fullsized":"2024-09-17T08:17:30.423319+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:46.091278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:46.091278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020828919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":61,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.610038+0000","last_change":"2024-09-17T08:15:32.989425+0000","last_active":"2024-09-17T08:17:37.610038+0000","last_peered":"2024-09-17T08:17:37.610038+0000","last_clean":"2024-09-17T08:17:37.610038+0000","last_became_active":"2024-09-17T08:15:29.372791+0000","last_became_peered":"2024-09-17T08:15:29.372791+0000","last_unstale":"2024-09-17T08:17:37.610038+0000","last_undegraded":"2024-09-17T08:17:37.610038+0000","last_fullsized":"2024-09-17T08:17:37.610038+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:32.986201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:32.986201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0116349,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"190'7203","reported_seq":7237,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:43.061778+0000","last_change":"2024-09-17T08:16:16.645635+0000","last_active":"2024-09-17T08:17:43.061778+0000","last_peered":"2024-09-17T08:17:43.061778+0000","last_clean":"2024-09-17T08:17:43.061778+0000","last_became_active":"2024-09-17T08:16:16.645315+0000","last_became_peered":"2024-09-17T08:16:16.645315+0000","last_unstale":"2024-09-17T08:17:43.061778+0000","last_undegraded":"2024-09-17T08:17:43.061778+0000","last_fullsized":"2024-09-17T08:17:43.061778+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472055808,"num_objects":7203,"num_object_clones":0,"num_object_copies":21609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":460992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"156'15","reported_seq":78,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.610066+0000","last_change":"2024-09-17T08:15:44.010651+0000","last_active":"2024-09-17T08:17:37.610066+0000","last_peered":"2024-09-17T08:17:37.610066+0000","last_clean":"2024-09-17T08:17:37.610066+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:17:37.610066+0000","last_undegraded":"2024-09-17T08:17:37.610066+0000","last_fullsized":"2024-09-17T08:17:37.610066+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'15","last_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.010126+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012291415999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.610172+0000","last_change":"2024-09-17T08:17:37.532454+0000","last_active":"2024-09-17T08:17:37.610172+0000","last_peered":"2024-09-17T08:17:37.610172+0000","last_clean":"2024-09-17T08:17:37.610172+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:17:37.610172+0000","last_undegraded":"2024-09-17T08:17:37.610172+0000","last_fullsized":"2024-09-17T08:17:37.610172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.532313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.532313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:43.540050+0000","scrub_duration":0.012255401000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.504641+0000","last_change":"2024-09-17T08:17:00.237348+0000","last_active":"2024-09-17T08:17:39.504641+0000","last_peered":"2024-09-17T08:17:39.504641+0000","last_clean":"2024-09-17T08:17:39.504641+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:17:39.504641+0000","last_undegraded":"2024-09-17T08:17:39.504641+0000","last_fullsized":"2024-09-17T08:17:39.504641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:17:00.237229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.237229+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:10.239107+0000","scrub_duration":0.004406877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.423323+0000","last_change":"2024-09-17T08:16:35.335671+0000","last_active":"2024-09-17T08:17:30.423323+0000","last_peered":"2024-09-17T08:17:30.423323+0000","last_clean":"2024-09-17T08:17:30.423323+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:17:30.423323+0000","last_undegraded":"2024-09-17T08:17:30.423323+0000","last_fullsized":"2024-09-17T08:17:30.423323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.335551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.335551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.895726+0000","scrub_duration":0.007005336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"190'7251","reported_seq":7285,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.827688+0000","last_change":"2024-09-17T08:16:16.648773+0000","last_active":"2024-09-17T08:17:42.827688+0000","last_peered":"2024-09-17T08:17:42.827688+0000","last_clean":"2024-09-17T08:17:42.827688+0000","last_became_active":"2024-09-17T08:16:16.648491+0000","last_became_peered":"2024-09-17T08:16:16.648491+0000","last_unstale":"2024-09-17T08:17:42.827688+0000","last_undegraded":"2024-09-17T08:17:42.827688+0000","last_fullsized":"2024-09-17T08:17:42.827688+0000","mapping_epoch":165,"log_start":"176'2300","ondisk_log_start":"176'2300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:44.077296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475201536,"num_objects":7251,"num_object_clones":0,"num_object_copies":21753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":464064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"152'30","reported_seq":110,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:41.611704+0000","last_change":"2024-09-17T08:17:41.611847+0000","last_active":"2024-09-17T08:17:41.611704+0000","last_peered":"2024-09-17T08:17:41.611704+0000","last_clean":"2024-09-17T08:17:41.611704+0000","last_became_active":"2024-09-17T08:15:29.385461+0000","last_became_peered":"2024-09-17T08:15:29.385461+0000","last_unstale":"2024-09-17T08:17:41.611704+0000","last_undegraded":"2024-09-17T08:17:41.611704+0000","last_fullsized":"2024-09-17T08:17:41.611704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'30","last_scrub_stamp":"2024-09-17T08:17:41.611678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.611678+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.598787+0000","scrub_duration":0.006094777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.610090+0000","last_change":"2024-09-17T08:15:53.067473+0000","last_active":"2024-09-17T08:17:37.610090+0000","last_peered":"2024-09-17T08:17:37.610090+0000","last_clean":"2024-09-17T08:17:37.610090+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:17:37.610090+0000","last_undegraded":"2024-09-17T08:17:37.610090+0000","last_fullsized":"2024-09-17T08:17:37.610090+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:15:53.067191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081478090000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":185,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429352+0000","last_change":"2024-09-17T08:15:52.083346+0000","last_active":"2024-09-17T08:17:30.429352+0000","last_peered":"2024-09-17T08:17:30.429352+0000","last_clean":"2024-09-17T08:17:30.429352+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:17:30.429352+0000","last_undegraded":"2024-09-17T08:17:30.429352+0000","last_fullsized":"2024-09-17T08:17:30.429352+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.083080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.083080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011225087999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436170+0000","last_change":"2024-09-17T08:15:44.574330+0000","last_active":"2024-09-17T08:17:30.436170+0000","last_peered":"2024-09-17T08:17:30.436170+0000","last_clean":"2024-09-17T08:17:30.436170+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:17:30.436170+0000","last_undegraded":"2024-09-17T08:17:30.436170+0000","last_fullsized":"2024-09-17T08:17:30.436170+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:44.574209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.574209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050670539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"190'7214","reported_seq":7248,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.831852+0000","last_change":"2024-09-17T08:16:16.402968+0000","last_active":"2024-09-17T08:17:42.831852+0000","last_peered":"2024-09-17T08:17:42.831852+0000","last_clean":"2024-09-17T08:17:42.831852+0000","last_became_active":"2024-09-17T08:16:16.402701+0000","last_became_peered":"2024-09-17T08:16:16.402701+0000","last_unstale":"2024-09-17T08:17:42.831852+0000","last_undegraded":"2024-09-17T08:17:42.831852+0000","last_fullsized":"2024-09-17T08:17:42.831852+0000","mapping_epoch":165,"log_start":"176'2200","ondisk_log_start":"176'2200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472776704,"num_objects":7214,"num_object_clones":0,"num_object_copies":21642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7214,"num_write_kb":461696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":73,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.504463+0000","last_change":"2024-09-17T08:16:33.206426+0000","last_active":"2024-09-17T08:17:39.504463+0000","last_peered":"2024-09-17T08:17:39.504463+0000","last_clean":"2024-09-17T08:17:39.504463+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:17:39.504463+0000","last_undegraded":"2024-09-17T08:17:39.504463+0000","last_fullsized":"2024-09-17T08:17:39.504463+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:16:33.206032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.206032+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:49.087945+0000","scrub_duration":0.0092541819999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":198,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:40.789450+0000","last_change":"2024-09-17T08:15:39.964221+0000","last_active":"2024-09-17T08:17:40.789450+0000","last_peered":"2024-09-17T08:17:40.789450+0000","last_clean":"2024-09-17T08:17:40.789450+0000","last_became_active":"2024-09-17T08:15:29.370817+0000","last_became_peered":"2024-09-17T08:15:29.370817+0000","last_unstale":"2024-09-17T08:17:40.789450+0000","last_undegraded":"2024-09-17T08:17:40.789450+0000","last_fullsized":"2024-09-17T08:17:40.789450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:15:39.964058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:39.964058+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015935661,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":139,"num_read_kb":134,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.432096+0000","last_change":"2024-09-17T08:16:35.307716+0000","last_active":"2024-09-17T08:17:30.432096+0000","last_peered":"2024-09-17T08:17:30.432096+0000","last_clean":"2024-09-17T08:17:30.432096+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:17:30.432096+0000","last_undegraded":"2024-09-17T08:17:30.432096+0000","last_fullsized":"2024-09-17T08:17:30.432096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.307594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.307594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090619600000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":178,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431619+0000","last_change":"2024-09-17T08:16:52.152636+0000","last_active":"2024-09-17T08:17:30.431619+0000","last_peered":"2024-09-17T08:17:30.431619+0000","last_clean":"2024-09-17T08:17:30.431619+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:17:30.431619+0000","last_undegraded":"2024-09-17T08:17:30.431619+0000","last_fullsized":"2024-09-17T08:17:30.431619+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.152367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.124044+0000","scrub_duration":0.0020852959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427990+0000","last_change":"2024-09-17T08:16:44.868243+0000","last_active":"2024-09-17T08:17:30.427990+0000","last_peered":"2024-09-17T08:17:30.427990+0000","last_clean":"2024-09-17T08:17:30.427990+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:17:30.427990+0000","last_undegraded":"2024-09-17T08:17:30.427990+0000","last_fullsized":"2024-09-17T08:17:30.427990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.868012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.868012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:06.438510+0000","scrub_duration":0.0025635359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"190'7324","reported_seq":7358,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.578441+0000","last_change":"2024-09-17T08:16:16.648374+0000","last_active":"2024-09-17T08:17:42.578441+0000","last_peered":"2024-09-17T08:17:42.578441+0000","last_clean":"2024-09-17T08:17:42.578441+0000","last_became_active":"2024-09-17T08:16:16.647832+0000","last_became_peered":"2024-09-17T08:16:16.647832+0000","last_unstale":"2024-09-17T08:17:42.578441+0000","last_undegraded":"2024-09-17T08:17:42.578441+0000","last_fullsized":"2024-09-17T08:17:42.578441+0000","mapping_epoch":165,"log_start":"176'2300","ondisk_log_start":"176'2300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5024,"log_dups_size":0,"ondisk_log_size":5024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479985664,"num_objects":7324,"num_object_clones":0,"num_object_copies":21972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":468736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":250,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.426155+0000","last_change":"2024-09-17T08:17:02.332268+0000","last_active":"2024-09-17T08:17:30.426155+0000","last_peered":"2024-09-17T08:17:30.426155+0000","last_clean":"2024-09-17T08:17:30.426155+0000","last_became_active":"2024-09-17T08:15:29.379763+0000","last_became_peered":"2024-09-17T08:15:29.379763+0000","last_unstale":"2024-09-17T08:17:30.426155+0000","last_undegraded":"2024-09-17T08:17:30.426155+0000","last_fullsized":"2024-09-17T08:17:30.426155+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:17:02.331862+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.331862+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.727563+0000","scrub_duration":0.0094874130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.424507+0000","last_change":"2024-09-17T08:16:50.181218+0000","last_active":"2024-09-17T08:17:30.424507+0000","last_peered":"2024-09-17T08:17:30.424507+0000","last_clean":"2024-09-17T08:17:30.424507+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:17:30.424507+0000","last_undegraded":"2024-09-17T08:17:30.424507+0000","last_fullsized":"2024-09-17T08:17:30.424507+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:16:50.181067+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.169610+0000","scrub_duration":0.0098447959999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429040+0000","last_change":"2024-09-17T08:16:35.781222+0000","last_active":"2024-09-17T08:17:30.429040+0000","last_peered":"2024-09-17T08:17:30.429040+0000","last_clean":"2024-09-17T08:17:30.429040+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:17:30.429040+0000","last_undegraded":"2024-09-17T08:17:30.429040+0000","last_fullsized":"2024-09-17T08:17:30.429040+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.781112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.781112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.512658+0000","scrub_duration":0.0040414090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":61,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.504603+0000","last_change":"2024-09-17T08:15:47.973188+0000","last_active":"2024-09-17T08:17:39.504603+0000","last_peered":"2024-09-17T08:17:39.504603+0000","last_clean":"2024-09-17T08:17:39.504603+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:17:39.504603+0000","last_undegraded":"2024-09-17T08:17:39.504603+0000","last_fullsized":"2024-09-17T08:17:39.504603+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:47.973067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:47.973067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046832360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":61,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.610142+0000","last_change":"2024-09-17T08:16:42.309390+0000","last_active":"2024-09-17T08:17:37.610142+0000","last_peered":"2024-09-17T08:17:37.610142+0000","last_clean":"2024-09-17T08:17:37.610142+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:17:37.610142+0000","last_undegraded":"2024-09-17T08:17:37.610142+0000","last_fullsized":"2024-09-17T08:17:37.610142+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.309103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.309103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:45.195168+0000","scrub_duration":0.0057864980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"190'7230","reported_seq":7264,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:43.090225+0000","last_change":"2024-09-17T08:16:16.648023+0000","last_active":"2024-09-17T08:17:43.090225+0000","last_peered":"2024-09-17T08:17:43.090225+0000","last_clean":"2024-09-17T08:17:43.090225+0000","last_became_active":"2024-09-17T08:16:16.647748+0000","last_became_peered":"2024-09-17T08:16:16.647748+0000","last_unstale":"2024-09-17T08:17:43.090225+0000","last_undegraded":"2024-09-17T08:17:43.090225+0000","last_fullsized":"2024-09-17T08:17:43.090225+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":473825280,"num_objects":7230,"num_object_clones":0,"num_object_copies":21690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7230,"num_write_kb":462720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"139'21","reported_seq":164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.634899+0000","last_change":"2024-09-17T08:17:37.634899+0000","last_active":"2024-09-17T08:17:37.634899+0000","last_peered":"2024-09-17T08:17:37.634899+0000","last_clean":"2024-09-17T08:17:37.634899+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:17:37.634899+0000","last_undegraded":"2024-09-17T08:17:37.634899+0000","last_fullsized":"2024-09-17T08:17:37.634899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'21","last_scrub_stamp":"2024-09-17T08:17:37.634598+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.634598+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.942132+0000","scrub_duration":0.0074777940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":178,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:36.901495+0000","last_change":"2024-09-17T08:17:36.901495+0000","last_active":"2024-09-17T08:17:36.901495+0000","last_peered":"2024-09-17T08:17:36.901495+0000","last_clean":"2024-09-17T08:17:36.901495+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:17:36.901495+0000","last_undegraded":"2024-09-17T08:17:36.901495+0000","last_fullsized":"2024-09-17T08:17:36.901495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:17:36.901360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.901360+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:41.027333+0000","scrub_duration":0.002432104,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.430868+0000","last_change":"2024-09-17T08:16:43.234496+0000","last_active":"2024-09-17T08:17:30.430868+0000","last_peered":"2024-09-17T08:17:30.430868+0000","last_clean":"2024-09-17T08:17:30.430868+0000","last_became_active":"2024-09-17T08:14:58.717890+0000","last_became_peered":"2024-09-17T08:14:58.717890+0000","last_unstale":"2024-09-17T08:17:30.430868+0000","last_undegraded":"2024-09-17T08:17:30.430868+0000","last_fullsized":"2024-09-17T08:17:30.430868+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:16:43.234376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.329605+0000","scrub_duration":0.0026197199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":432,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429188+0000","last_change":"2024-09-17T08:16:41.854709+0000","last_active":"2024-09-17T08:17:30.429188+0000","last_peered":"2024-09-17T08:17:30.429188+0000","last_clean":"2024-09-17T08:17:30.429188+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:17:30.429188+0000","last_undegraded":"2024-09-17T08:17:30.429188+0000","last_fullsized":"2024-09-17T08:17:30.429188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:16:41.854569+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2024-09-17T08:10:08.344133+0000","last_clean_scrub_stamp":"2024-09-17T08:16:41.854569+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:43.215244+0000","scrub_duration":0.077111972000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":167,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427767+0000","last_change":"2024-09-17T08:16:34.817029+0000","last_active":"2024-09-17T08:17:30.427767+0000","last_peered":"2024-09-17T08:17:30.427767+0000","last_clean":"2024-09-17T08:17:30.427767+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:17:30.427767+0000","last_undegraded":"2024-09-17T08:17:30.427767+0000","last_fullsized":"2024-09-17T08:17:30.427767+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:34.816846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:16:34.816846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:44.180844+0000","scrub_duration":0.001439311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":70,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436721+0000","last_change":"2024-09-17T08:16:33.884267+0000","last_active":"2024-09-17T08:17:30.436721+0000","last_peered":"2024-09-17T08:17:30.436721+0000","last_clean":"2024-09-17T08:17:30.436721+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:17:30.436721+0000","last_undegraded":"2024-09-17T08:17:30.436721+0000","last_fullsized":"2024-09-17T08:17:30.436721+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.884139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.884139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.670082+0000","scrub_duration":0.0075210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"190'7207","reported_seq":7241,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:43.064995+0000","last_change":"2024-09-17T08:16:16.646700+0000","last_active":"2024-09-17T08:17:43.064995+0000","last_peered":"2024-09-17T08:17:43.064995+0000","last_clean":"2024-09-17T08:17:43.064995+0000","last_became_active":"2024-09-17T08:16:16.646368+0000","last_became_peered":"2024-09-17T08:16:16.646368+0000","last_unstale":"2024-09-17T08:17:43.064995+0000","last_undegraded":"2024-09-17T08:17:43.064995+0000","last_fullsized":"2024-09-17T08:17:43.064995+0000","mapping_epoch":165,"log_start":"174'1500","ondisk_log_start":"174'1500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5707,"log_dups_size":0,"ondisk_log_size":5707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472317952,"num_objects":7207,"num_object_clones":0,"num_object_copies":21621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":461248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":81,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.505225+0000","last_change":"2024-09-17T08:15:51.007614+0000","last_active":"2024-09-17T08:17:39.505225+0000","last_peered":"2024-09-17T08:17:39.505225+0000","last_clean":"2024-09-17T08:17:39.505225+0000","last_became_active":"2024-09-17T08:15:29.360821+0000","last_became_peered":"2024-09-17T08:15:29.360821+0000","last_unstale":"2024-09-17T08:17:39.505225+0000","last_undegraded":"2024-09-17T08:17:39.505225+0000","last_fullsized":"2024-09-17T08:17:39.505225+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:15:51.004953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:51.004953+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048347169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":182,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:40.401286+0000","last_change":"2024-09-17T08:17:40.401393+0000","last_active":"2024-09-17T08:17:40.401286+0000","last_peered":"2024-09-17T08:17:40.401286+0000","last_clean":"2024-09-17T08:17:40.401286+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:17:40.401286+0000","last_undegraded":"2024-09-17T08:17:40.401286+0000","last_fullsized":"2024-09-17T08:17:40.401286+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.401262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.401262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:53.178795+0000","scrub_duration":0.0066464740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:36.372735+0000","last_change":"2024-09-17T08:17:36.372735+0000","last_active":"2024-09-17T08:17:36.372735+0000","last_peered":"2024-09-17T08:17:36.372735+0000","last_clean":"2024-09-17T08:17:36.372735+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:17:36.372735+0000","last_undegraded":"2024-09-17T08:17:36.372735+0000","last_fullsized":"2024-09-17T08:17:36.372735+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.372603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.372603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.970558+0000","scrub_duration":0.010983404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:34.135923+0000","last_change":"2024-09-17T08:17:34.135923+0000","last_active":"2024-09-17T08:17:34.135923+0000","last_peered":"2024-09-17T08:17:34.135923+0000","last_clean":"2024-09-17T08:17:34.135923+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:17:34.135923+0000","last_undegraded":"2024-09-17T08:17:34.135923+0000","last_fullsized":"2024-09-17T08:17:34.135923+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.135505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.135505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.113488+0000","scrub_duration":0.01331624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"190'7297","reported_seq":7331,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.579059+0000","last_change":"2024-09-17T08:16:16.650552+0000","last_active":"2024-09-17T08:17:42.579059+0000","last_peered":"2024-09-17T08:17:42.579059+0000","last_clean":"2024-09-17T08:17:42.579059+0000","last_became_active":"2024-09-17T08:16:16.650307+0000","last_became_peered":"2024-09-17T08:16:16.650307+0000","last_unstale":"2024-09-17T08:17:42.579059+0000","last_undegraded":"2024-09-17T08:17:42.579059+0000","last_fullsized":"2024-09-17T08:17:42.579059+0000","mapping_epoch":165,"log_start":"176'2200","ondisk_log_start":"176'2200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":478216192,"num_objects":7297,"num_object_clones":0,"num_object_copies":21891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":467008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":223,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429067+0000","last_change":"2024-09-17T08:15:55.110906+0000","last_active":"2024-09-17T08:17:30.429067+0000","last_peered":"2024-09-17T08:17:30.429067+0000","last_clean":"2024-09-17T08:17:30.429067+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:17:30.429067+0000","last_undegraded":"2024-09-17T08:17:30.429067+0000","last_fullsized":"2024-09-17T08:17:30.429067+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:15:55.110811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:55.110811+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001860472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431920+0000","last_change":"2024-09-17T08:15:40.995249+0000","last_active":"2024-09-17T08:17:30.431920+0000","last_peered":"2024-09-17T08:17:30.431920+0000","last_clean":"2024-09-17T08:17:30.431920+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:17:30.431920+0000","last_undegraded":"2024-09-17T08:17:30.431920+0000","last_fullsized":"2024-09-17T08:17:30.431920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:15:40.994687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042524099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":178,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.297792+0000","last_change":"2024-09-17T08:17:31.297792+0000","last_active":"2024-09-17T08:17:31.297792+0000","last_peered":"2024-09-17T08:17:31.297792+0000","last_clean":"2024-09-17T08:17:31.297792+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:17:31.297792+0000","last_undegraded":"2024-09-17T08:17:31.297792+0000","last_fullsized":"2024-09-17T08:17:31.297792+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.297237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.297237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.099318+0000","scrub_duration":0.0044744609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.424440+0000","last_change":"2024-09-17T08:16:54.185896+0000","last_active":"2024-09-17T08:17:30.424440+0000","last_peered":"2024-09-17T08:17:30.424440+0000","last_clean":"2024-09-17T08:17:30.424440+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:17:30.424440+0000","last_undegraded":"2024-09-17T08:17:30.424440+0000","last_fullsized":"2024-09-17T08:17:30.424440+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:54.185311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:54.185311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:16.656330+0000","scrub_duration":0.0041021820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"190'7249","reported_seq":7283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.323602+0000","last_change":"2024-09-17T08:16:16.649082+0000","last_active":"2024-09-17T08:17:42.323602+0000","last_peered":"2024-09-17T08:17:42.323602+0000","last_clean":"2024-09-17T08:17:42.323602+0000","last_became_active":"2024-09-17T08:16:16.648709+0000","last_became_peered":"2024-09-17T08:16:16.648709+0000","last_unstale":"2024-09-17T08:17:42.323602+0000","last_undegraded":"2024-09-17T08:17:42.323602+0000","last_fullsized":"2024-09-17T08:17:42.323602+0000","mapping_epoch":165,"log_start":"176'1900","ondisk_log_start":"176'1900","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475070464,"num_objects":7249,"num_object_clones":0,"num_object_copies":21747,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":463936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"140'22","reported_seq":207,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427254+0000","last_change":"2024-09-17T08:16:57.000722+0000","last_active":"2024-09-17T08:17:30.427254+0000","last_peered":"2024-09-17T08:17:30.427254+0000","last_clean":"2024-09-17T08:17:30.427254+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:17:30.427254+0000","last_undegraded":"2024-09-17T08:17:30.427254+0000","last_fullsized":"2024-09-17T08:17:30.427254+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'22","last_scrub_stamp":"2024-09-17T08:16:57.000450+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:14:50.155143+0000","last_clean_scrub_stamp":"2024-09-17T08:16:57.000450+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.273416+0000","scrub_duration":0.0060780779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431825+0000","last_change":"2024-09-17T08:16:40.246174+0000","last_active":"2024-09-17T08:17:30.431825+0000","last_peered":"2024-09-17T08:17:30.431825+0000","last_clean":"2024-09-17T08:17:30.431825+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:17:30.431825+0000","last_undegraded":"2024-09-17T08:17:30.431825+0000","last_fullsized":"2024-09-17T08:17:30.431825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.246057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.246057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.996076+0000","scrub_duration":0.0035572160000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":268,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.914154+0000","last_change":"2024-09-17T08:16:24.936605+0000","last_active":"2024-09-17T08:17:37.914154+0000","last_peered":"2024-09-17T08:17:37.914154+0000","last_clean":"2024-09-17T08:17:37.914154+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:17:37.914154+0000","last_undegraded":"2024-09-17T08:17:37.914154+0000","last_fullsized":"2024-09-17T08:17:37.914154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:16:24.936485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:24.936485+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.936007+0000","scrub_duration":0.0033469289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":81,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.423252+0000","last_change":"2024-09-17T08:16:58.243174+0000","last_active":"2024-09-17T08:17:30.423252+0000","last_peered":"2024-09-17T08:17:30.423252+0000","last_clean":"2024-09-17T08:17:30.423252+0000","last_became_active":"2024-09-17T08:14:58.717856+0000","last_became_peered":"2024-09-17T08:14:58.717856+0000","last_unstale":"2024-09-17T08:17:30.423252+0000","last_undegraded":"2024-09-17T08:17:30.423252+0000","last_fullsized":"2024-09-17T08:17:30.423252+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.242877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.242877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.573195+0000","scrub_duration":0.0056167969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"190'7158","reported_seq":7192,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-17T08:17:42.302323+0000","last_change":"2024-09-17T08:16:16.647722+0000","last_active":"2024-09-17T08:17:42.302323+0000","last_peered":"2024-09-17T08:17:42.302323+0000","last_clean":"2024-09-17T08:17:42.302323+0000","last_became_active":"2024-09-17T08:16:16.647324+0000","last_became_peered":"2024-09-17T08:16:16.647324+0000","last_unstale":"2024-09-17T08:17:42.302323+0000","last_undegraded":"2024-09-17T08:17:42.302323+0000","last_fullsized":"2024-09-17T08:17:42.302323+0000","mapping_epoch":165,"log_start":"175'1800","ondisk_log_start":"175'1800","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5358,"log_dups_size":0,"ondisk_log_size":5358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469106688,"num_objects":7158,"num_object_clones":0,"num_object_copies":21474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7158,"num_write_kb":458112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":245,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420252+0000","last_change":"2024-09-17T08:17:00.600357+0000","last_active":"2024-09-17T08:17:30.420252+0000","last_peered":"2024-09-17T08:17:30.420252+0000","last_clean":"2024-09-17T08:17:30.420252+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:17:30.420252+0000","last_undegraded":"2024-09-17T08:17:30.420252+0000","last_fullsized":"2024-09-17T08:17:30.420252+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:17:00.600225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.600225+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.375848+0000","scrub_duration":0.0071635179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.421134+0000","last_change":"2024-09-17T08:16:39.838818+0000","last_active":"2024-09-17T08:17:30.421134+0000","last_peered":"2024-09-17T08:17:30.421134+0000","last_clean":"2024-09-17T08:17:30.421134+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:17:30.421134+0000","last_undegraded":"2024-09-17T08:17:30.421134+0000","last_fullsized":"2024-09-17T08:17:30.421134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:16:39.838667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.838667+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:42.922775+0000","scrub_duration":0.0050598739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":196,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.430052+0000","last_change":"2024-09-17T08:15:46.312539+0000","last_active":"2024-09-17T08:17:30.430052+0000","last_peered":"2024-09-17T08:17:30.430052+0000","last_clean":"2024-09-17T08:17:30.430052+0000","last_became_active":"2024-09-17T08:15:29.374092+0000","last_became_peered":"2024-09-17T08:15:29.374092+0000","last_unstale":"2024-09-17T08:17:30.430052+0000","last_undegraded":"2024-09-17T08:17:30.430052+0000","last_fullsized":"2024-09-17T08:17:30.430052+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:46.312367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:03.042476+0000","last_clean_scrub_stamp":"2024-09-17T08:15:46.312367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012162289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":183,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.438453+0000","last_change":"2024-09-17T08:16:23.763256+0000","last_active":"2024-09-17T08:17:30.438453+0000","last_peered":"2024-09-17T08:17:30.438453+0000","last_clean":"2024-09-17T08:17:30.438453+0000","last_became_active":"2024-09-17T08:15:29.384203+0000","last_became_peered":"2024-09-17T08:15:29.384203+0000","last_unstale":"2024-09-17T08:17:30.438453+0000","last_undegraded":"2024-09-17T08:17:30.438453+0000","last_fullsized":"2024-09-17T08:17:30.438453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.763173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.763173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.300130+0000","scrub_duration":0.001814333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":189,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.426080+0000","last_change":"2024-09-17T08:16:36.341678+0000","last_active":"2024-09-17T08:17:30.426080+0000","last_peered":"2024-09-17T08:17:30.426080+0000","last_clean":"2024-09-17T08:17:30.426080+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:17:30.426080+0000","last_undegraded":"2024-09-17T08:17:30.426080+0000","last_fullsized":"2024-09-17T08:17:30.426080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:16:36.341539+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:36.341539+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011421961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":200,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.430175+0000","last_change":"2024-09-17T08:16:49.193851+0000","last_active":"2024-09-17T08:17:30.430175+0000","last_peered":"2024-09-17T08:17:30.430175+0000","last_clean":"2024-09-17T08:17:30.430175+0000","last_became_active":"2024-09-17T08:15:29.373114+0000","last_became_peered":"2024-09-17T08:15:29.373114+0000","last_unstale":"2024-09-17T08:17:30.430175+0000","last_undegraded":"2024-09-17T08:17:30.430175+0000","last_fullsized":"2024-09-17T08:17:30.430175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.193338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.193338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:59.549312+0000","scrub_duration":0.0089744739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":174,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436930+0000","last_change":"2024-09-17T08:15:33.385747+0000","last_active":"2024-09-17T08:17:30.436930+0000","last_peered":"2024-09-17T08:17:30.436930+0000","last_clean":"2024-09-17T08:17:30.436930+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:17:30.436930+0000","last_undegraded":"2024-09-17T08:17:30.436930+0000","last_fullsized":"2024-09-17T08:17:30.436930+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:33.385285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:33.385285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077031670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431910+0000","last_change":"2024-09-17T08:16:48.222810+0000","last_active":"2024-09-17T08:17:30.431910+0000","last_peered":"2024-09-17T08:17:30.431910+0000","last_clean":"2024-09-17T08:17:30.431910+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:17:30.431910+0000","last_undegraded":"2024-09-17T08:17:30.431910+0000","last_fullsized":"2024-09-17T08:17:30.431910+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.222304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.222304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.070451+0000","scrub_duration":0.003060929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":140,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420395+0000","last_change":"2024-09-17T08:16:29.839526+0000","last_active":"2024-09-17T08:17:30.420395+0000","last_peered":"2024-09-17T08:17:30.420395+0000","last_clean":"2024-09-17T08:17:30.420395+0000","last_became_active":"2024-09-17T08:15:29.358382+0000","last_became_peered":"2024-09-17T08:15:29.358382+0000","last_unstale":"2024-09-17T08:17:30.420395+0000","last_undegraded":"2024-09-17T08:17:30.420395+0000","last_fullsized":"2024-09-17T08:17:30.420395+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:29.839416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.839416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002703015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":212,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.437050+0000","last_change":"2024-09-17T08:16:30.802344+0000","last_active":"2024-09-17T08:17:30.437050+0000","last_peered":"2024-09-17T08:17:30.437050+0000","last_clean":"2024-09-17T08:17:30.437050+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:17:30.437050+0000","last_undegraded":"2024-09-17T08:17:30.437050+0000","last_fullsized":"2024-09-17T08:17:30.437050+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:16:30.800480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.800480+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:43.471047+0000","scrub_duration":0.0031490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":77,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436099+0000","last_change":"2024-09-17T08:15:29.383868+0000","last_active":"2024-09-17T08:17:30.436099+0000","last_peered":"2024-09-17T08:17:30.436099+0000","last_clean":"2024-09-17T08:17:30.436099+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:17:30.436099+0000","last_undegraded":"2024-09-17T08:17:30.436099+0000","last_fullsized":"2024-09-17T08:17:30.436099+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:51.191602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:51.191602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427922+0000","last_change":"2024-09-17T08:16:51.020489+0000","last_active":"2024-09-17T08:17:30.427922+0000","last_peered":"2024-09-17T08:17:30.427922+0000","last_clean":"2024-09-17T08:17:30.427922+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:17:30.427922+0000","last_undegraded":"2024-09-17T08:17:30.427922+0000","last_fullsized":"2024-09-17T08:17:30.427922+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.020318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.020318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.433650+0000","scrub_duration":0.0019765820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":226,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.437446+0000","last_change":"2024-09-17T08:16:45.629362+0000","last_active":"2024-09-17T08:17:30.437446+0000","last_peered":"2024-09-17T08:17:30.437446+0000","last_clean":"2024-09-17T08:17:30.437446+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:17:30.437446+0000","last_undegraded":"2024-09-17T08:17:30.437446+0000","last_fullsized":"2024-09-17T08:17:30.437446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:45.626489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:45.626489+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.848535+0000","scrub_duration":0.0060751379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":102,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.435984+0000","last_change":"2024-09-17T08:14:58.721591+0000","last_active":"2024-09-17T08:17:30.435984+0000","last_peered":"2024-09-17T08:17:30.435984+0000","last_clean":"2024-09-17T08:17:30.435984+0000","last_became_active":"2024-09-17T08:14:58.720104+0000","last_became_peered":"2024-09-17T08:14:58.720104+0000","last_unstale":"2024-09-17T08:17:30.435984+0000","last_undegraded":"2024-09-17T08:17:30.435984+0000","last_fullsized":"2024-09-17T08:17:30.435984+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:47.207288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:14:47.207288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024252850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"152'11","reported_seq":91,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.917995+0000","last_change":"2024-09-17T08:15:52.034956+0000","last_active":"2024-09-17T08:17:37.917995+0000","last_peered":"2024-09-17T08:17:37.917995+0000","last_clean":"2024-09-17T08:17:37.917995+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:17:37.917995+0000","last_undegraded":"2024-09-17T08:17:37.917995+0000","last_fullsized":"2024-09-17T08:17:37.917995+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'11","last_scrub_stamp":"2024-09-17T08:15:52.034662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.034662+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038045739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.406086+0000","last_change":"2024-09-17T08:17:33.406086+0000","last_active":"2024-09-17T08:17:33.406086+0000","last_peered":"2024-09-17T08:17:33.406086+0000","last_clean":"2024-09-17T08:17:33.406086+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:17:33.406086+0000","last_undegraded":"2024-09-17T08:17:33.406086+0000","last_fullsized":"2024-09-17T08:17:33.406086+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:33.405939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.405939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.635332+0000","scrub_duration":0.0072126600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"177'13","reported_seq":114,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.428212+0000","last_change":"2024-09-17T08:17:00.945597+0000","last_active":"2024-09-17T08:17:30.428212+0000","last_peered":"2024-09-17T08:17:30.428212+0000","last_clean":"2024-09-17T08:17:30.428212+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:17:30.428212+0000","last_undegraded":"2024-09-17T08:17:30.428212+0000","last_fullsized":"2024-09-17T08:17:30.428212+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"177'13","last_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.945463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.215485+0000","scrub_duration":0.016379540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.429158+0000","last_change":"2024-09-17T08:16:46.264262+0000","last_active":"2024-09-17T08:17:30.429158+0000","last_peered":"2024-09-17T08:17:30.429158+0000","last_clean":"2024-09-17T08:17:30.429158+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:17:30.429158+0000","last_undegraded":"2024-09-17T08:17:30.429158+0000","last_fullsized":"2024-09-17T08:17:30.429158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:46.264138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.264138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.163330+0000","scrub_duration":0.0046054340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":269,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.914509+0000","last_change":"2024-09-17T08:17:35.880572+0000","last_active":"2024-09-17T08:17:37.914509+0000","last_peered":"2024-09-17T08:17:37.914509+0000","last_clean":"2024-09-17T08:17:37.914509+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:17:37.914509+0000","last_undegraded":"2024-09-17T08:17:37.914509+0000","last_fullsized":"2024-09-17T08:17:37.914509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:17:35.880439+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.880439+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.829845+0000","scrub_duration":0.0069173250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420927+0000","last_change":"2024-09-17T08:16:44.899582+0000","last_active":"2024-09-17T08:17:30.420927+0000","last_peered":"2024-09-17T08:17:30.420927+0000","last_clean":"2024-09-17T08:17:30.420927+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:17:30.420927+0000","last_undegraded":"2024-09-17T08:17:30.420927+0000","last_fullsized":"2024-09-17T08:17:30.420927+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.899473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.899473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.638190+0000","scrub_duration":0.0048647389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":70,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.426355+0000","last_change":"2024-09-17T08:17:31.426355+0000","last_active":"2024-09-17T08:17:31.426355+0000","last_peered":"2024-09-17T08:17:31.426355+0000","last_clean":"2024-09-17T08:17:31.426355+0000","last_became_active":"2024-09-17T08:15:29.364047+0000","last_became_peered":"2024-09-17T08:15:29.364047+0000","last_unstale":"2024-09-17T08:17:31.426355+0000","last_undegraded":"2024-09-17T08:17:31.426355+0000","last_fullsized":"2024-09-17T08:17:31.426355+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:31.426185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.426185+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.778192+0000","scrub_duration":0.028345339000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":122,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.437748+0000","last_change":"2024-09-17T08:15:54.838921+0000","last_active":"2024-09-17T08:17:30.437748+0000","last_peered":"2024-09-17T08:17:30.437748+0000","last_clean":"2024-09-17T08:17:30.437748+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:17:30.437748+0000","last_undegraded":"2024-09-17T08:17:30.437748+0000","last_fullsized":"2024-09-17T08:17:30.437748+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.838847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.838847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":185,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.933012+0000","last_change":"2024-09-17T08:17:32.933012+0000","last_active":"2024-09-17T08:17:32.933012+0000","last_peered":"2024-09-17T08:17:32.933012+0000","last_clean":"2024-09-17T08:17:32.933012+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:17:32.933012+0000","last_undegraded":"2024-09-17T08:17:32.933012+0000","last_fullsized":"2024-09-17T08:17:32.933012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.932889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.932889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.988363+0000","scrub_duration":0.017331688000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":194,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.431486+0000","last_change":"2024-09-17T08:17:37.431486+0000","last_active":"2024-09-17T08:17:37.431486+0000","last_peered":"2024-09-17T08:17:37.431486+0000","last_clean":"2024-09-17T08:17:37.431486+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:17:37.431486+0000","last_undegraded":"2024-09-17T08:17:37.431486+0000","last_fullsized":"2024-09-17T08:17:37.431486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.431366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.431366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.282544+0000","scrub_duration":0.0073929879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":241,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420559+0000","last_change":"2024-09-17T08:16:59.607476+0000","last_active":"2024-09-17T08:17:30.420559+0000","last_peered":"2024-09-17T08:17:30.420559+0000","last_clean":"2024-09-17T08:17:30.420559+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:17:30.420559+0000","last_undegraded":"2024-09-17T08:17:30.420559+0000","last_fullsized":"2024-09-17T08:17:30.420559+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:16:59.607325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.607325+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.411643+0000","scrub_duration":0.0083300879999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.434265+0000","last_change":"2024-09-17T08:17:01.469617+0000","last_active":"2024-09-17T08:17:30.434265+0000","last_peered":"2024-09-17T08:17:30.434265+0000","last_clean":"2024-09-17T08:17:30.434265+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:17:30.434265+0000","last_undegraded":"2024-09-17T08:17:30.434265+0000","last_fullsized":"2024-09-17T08:17:30.434265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.469473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.469473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:22.713180+0000","scrub_duration":0.0032550880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.455271+0000","last_change":"2024-09-17T08:17:32.455271+0000","last_active":"2024-09-17T08:17:32.455271+0000","last_peered":"2024-09-17T08:17:32.455271+0000","last_clean":"2024-09-17T08:17:32.455271+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:17:32.455271+0000","last_undegraded":"2024-09-17T08:17:32.455271+0000","last_fullsized":"2024-09-17T08:17:32.455271+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.455128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.455128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.037261+0000","scrub_duration":0.0091779470000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436407+0000","last_change":"2024-09-17T08:16:39.766499+0000","last_active":"2024-09-17T08:17:30.436407+0000","last_peered":"2024-09-17T08:17:30.436407+0000","last_clean":"2024-09-17T08:17:30.436407+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:17:30.436407+0000","last_undegraded":"2024-09-17T08:17:30.436407+0000","last_fullsized":"2024-09-17T08:17:30.436407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:39.766371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.766371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:51.034351+0000","scrub_duration":0.003380568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":209,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.437204+0000","last_change":"2024-09-17T08:16:33.342841+0000","last_active":"2024-09-17T08:17:30.437204+0000","last_peered":"2024-09-17T08:17:30.437204+0000","last_clean":"2024-09-17T08:17:30.437204+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:17:30.437204+0000","last_undegraded":"2024-09-17T08:17:30.437204+0000","last_fullsized":"2024-09-17T08:17:30.437204+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:14:35.433619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:35.433619+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079958110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":82,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.395358+0000","last_change":"2024-09-17T08:17:37.395358+0000","last_active":"2024-09-17T08:17:37.395358+0000","last_peered":"2024-09-17T08:17:37.395358+0000","last_clean":"2024-09-17T08:17:37.395358+0000","last_became_active":"2024-09-17T08:14:58.715288+0000","last_became_peered":"2024-09-17T08:14:58.715288+0000","last_unstale":"2024-09-17T08:17:37.395358+0000","last_undegraded":"2024-09-17T08:17:37.395358+0000","last_fullsized":"2024-09-17T08:17:37.395358+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.394936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.394936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.208429+0000","scrub_duration":0.0060254940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":134,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:31.626868+0000","last_change":"2024-09-17T08:17:31.626868+0000","last_active":"2024-09-17T08:17:31.626868+0000","last_peered":"2024-09-17T08:17:31.626868+0000","last_clean":"2024-09-17T08:17:31.626868+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:17:31.626868+0000","last_undegraded":"2024-09-17T08:17:31.626868+0000","last_fullsized":"2024-09-17T08:17:31.626868+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.626749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:52.132279+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.626749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.963940+0000","scrub_duration":0.002332518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":176,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.427213+0000","last_change":"2024-09-17T08:16:52.066187+0000","last_active":"2024-09-17T08:17:30.427213+0000","last_peered":"2024-09-17T08:17:30.427213+0000","last_clean":"2024-09-17T08:17:30.427213+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:17:30.427213+0000","last_undegraded":"2024-09-17T08:17:30.427213+0000","last_fullsized":"2024-09-17T08:17:30.427213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.066058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.066058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.573732+0000","scrub_duration":0.0023799279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":188,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:33.932414+0000","last_change":"2024-09-17T08:17:33.932414+0000","last_active":"2024-09-17T08:17:33.932414+0000","last_peered":"2024-09-17T08:17:33.932414+0000","last_clean":"2024-09-17T08:17:33.932414+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:17:33.932414+0000","last_undegraded":"2024-09-17T08:17:33.932414+0000","last_fullsized":"2024-09-17T08:17:33.932414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:33.932279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.932279+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:03.370870+0000","scrub_duration":0.0031072140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":108,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431228+0000","last_change":"2024-09-17T08:16:27.299767+0000","last_active":"2024-09-17T08:17:30.431228+0000","last_peered":"2024-09-17T08:17:30.431228+0000","last_clean":"2024-09-17T08:17:30.431228+0000","last_became_active":"2024-09-17T08:14:58.716813+0000","last_became_peered":"2024-09-17T08:14:58.716813+0000","last_unstale":"2024-09-17T08:17:30.431228+0000","last_undegraded":"2024-09-17T08:17:30.431228+0000","last_fullsized":"2024-09-17T08:17:30.431228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:16:27.299675+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.299675+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.173560+0000","scrub_duration":0.0018966499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":178,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:39.165283+0000","last_change":"2024-09-17T08:17:39.165283+0000","last_active":"2024-09-17T08:17:39.165283+0000","last_peered":"2024-09-17T08:17:39.165283+0000","last_clean":"2024-09-17T08:17:39.165283+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:17:39.165283+0000","last_undegraded":"2024-09-17T08:17:39.165283+0000","last_fullsized":"2024-09-17T08:17:39.165283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.164442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.164442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:07.439327+0000","scrub_duration":0.0024479319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":154,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.438386+0000","last_change":"2024-09-17T08:15:51.743705+0000","last_active":"2024-09-17T08:17:30.438386+0000","last_peered":"2024-09-17T08:17:30.438386+0000","last_clean":"2024-09-17T08:17:30.438386+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:17:30.438386+0000","last_undegraded":"2024-09-17T08:17:30.438386+0000","last_fullsized":"2024-09-17T08:17:30.438386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:51.721883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:51.721883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082713839999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":167,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.311582+0000","last_change":"2024-09-17T08:16:40.783169+0000","last_active":"2024-09-17T08:17:32.311582+0000","last_peered":"2024-09-17T08:17:32.311582+0000","last_clean":"2024-09-17T08:17:32.311582+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:17:32.311582+0000","last_undegraded":"2024-09-17T08:17:32.311582+0000","last_fullsized":"2024-09-17T08:17:32.311582+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:16:40.783041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.783041+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.590749+0000","scrub_duration":0.0030603710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.420082+0000","last_change":"2024-09-17T08:15:29.977775+0000","last_active":"2024-09-17T08:17:30.420082+0000","last_peered":"2024-09-17T08:17:30.420082+0000","last_clean":"2024-09-17T08:17:30.420082+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:17:30.420082+0000","last_undegraded":"2024-09-17T08:17:30.420082+0000","last_fullsized":"2024-09-17T08:17:30.420082+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:29.977666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:29.977666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012253139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":195,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.426011+0000","last_change":"2024-09-17T08:16:47.247472+0000","last_active":"2024-09-17T08:17:30.426011+0000","last_peered":"2024-09-17T08:17:30.426011+0000","last_clean":"2024-09-17T08:17:30.426011+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:17:30.426011+0000","last_undegraded":"2024-09-17T08:17:30.426011+0000","last_fullsized":"2024-09-17T08:17:30.426011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:47.247339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.247339+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:15.653964+0000","scrub_duration":0.0072034789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":204,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:37.914289+0000","last_change":"2024-09-17T08:16:52.823084+0000","last_active":"2024-09-17T08:17:37.914289+0000","last_peered":"2024-09-17T08:17:37.914289+0000","last_clean":"2024-09-17T08:17:37.914289+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:17:37.914289+0000","last_undegraded":"2024-09-17T08:17:37.914289+0000","last_fullsized":"2024-09-17T08:17:37.914289+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:16:52.822979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.822979+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.218032+0000","scrub_duration":0.0016070290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.431986+0000","last_change":"2024-09-17T08:15:33.983721+0000","last_active":"2024-09-17T08:17:30.431986+0000","last_peered":"2024-09-17T08:17:30.431986+0000","last_clean":"2024-09-17T08:17:30.431986+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:17:30.431986+0000","last_undegraded":"2024-09-17T08:17:30.431986+0000","last_fullsized":"2024-09-17T08:17:30.431986+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:33.983570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:33.983570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007829595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":171,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436906+0000","last_change":"2024-09-17T08:16:58.658509+0000","last_active":"2024-09-17T08:17:30.436906+0000","last_peered":"2024-09-17T08:17:30.436906+0000","last_clean":"2024-09-17T08:17:30.436906+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:17:30.436906+0000","last_undegraded":"2024-09-17T08:17:30.436906+0000","last_fullsized":"2024-09-17T08:17:30.436906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.658382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.658382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.307934+0000","scrub_duration":0.0056271569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":73,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:38.572088+0000","last_change":"2024-09-17T08:17:38.572088+0000","last_active":"2024-09-17T08:17:38.572088+0000","last_peered":"2024-09-17T08:17:38.572088+0000","last_clean":"2024-09-17T08:17:38.572088+0000","last_became_active":"2024-09-17T08:15:29.361714+0000","last_became_peered":"2024-09-17T08:15:29.361714+0000","last_unstale":"2024-09-17T08:17:38.572088+0000","last_undegraded":"2024-09-17T08:17:38.572088+0000","last_fullsized":"2024-09-17T08:17:38.572088+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:38.571885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:38.571885+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:56.028615+0000","scrub_duration":0.0076271409999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":178,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.425930+0000","last_change":"2024-09-17T08:16:49.210341+0000","last_active":"2024-09-17T08:17:30.425930+0000","last_peered":"2024-09-17T08:17:30.425930+0000","last_clean":"2024-09-17T08:17:30.425930+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:17:30.425930+0000","last_undegraded":"2024-09-17T08:17:30.425930+0000","last_fullsized":"2024-09-17T08:17:30.425930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.210217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.210217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.183066+0000","scrub_duration":0.0066617630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":165,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.421343+0000","last_change":"2024-09-17T08:16:37.835578+0000","last_active":"2024-09-17T08:17:30.421343+0000","last_peered":"2024-09-17T08:17:30.421343+0000","last_clean":"2024-09-17T08:17:30.421343+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:17:30.421343+0000","last_undegraded":"2024-09-17T08:17:30.421343+0000","last_fullsized":"2024-09-17T08:17:30.421343+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.835368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.835368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:47.709790+0000","scrub_duration":0.001605574,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":132,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:36.583106+0000","last_change":"2024-09-17T08:17:36.583106+0000","last_active":"2024-09-17T08:17:36.583106+0000","last_peered":"2024-09-17T08:17:36.583106+0000","last_clean":"2024-09-17T08:17:36.583106+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:17:36.583106+0000","last_undegraded":"2024-09-17T08:17:36.583106+0000","last_fullsized":"2024-09-17T08:17:36.583106+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.582852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.582852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.756203+0000","scrub_duration":0.001816591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":287,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.432203+0000","last_change":"2024-09-17T08:16:59.092346+0000","last_active":"2024-09-17T08:17:30.432203+0000","last_peered":"2024-09-17T08:17:30.432203+0000","last_clean":"2024-09-17T08:17:30.432203+0000","last_became_active":"2024-09-17T08:15:29.388191+0000","last_became_peered":"2024-09-17T08:15:29.388191+0000","last_unstale":"2024-09-17T08:17:30.432203+0000","last_undegraded":"2024-09-17T08:17:30.432203+0000","last_fullsized":"2024-09-17T08:17:30.432203+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:16:59.092016+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.092016+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.568034+0000","scrub_duration":0.0088076999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":60,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.434110+0000","last_change":"2024-09-17T08:16:51.343826+0000","last_active":"2024-09-17T08:17:30.434110+0000","last_peered":"2024-09-17T08:17:30.434110+0000","last_clean":"2024-09-17T08:17:30.434110+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:17:30.434110+0000","last_undegraded":"2024-09-17T08:17:30.434110+0000","last_fullsized":"2024-09-17T08:17:30.434110+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.343712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.343712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.929418+0000","scrub_duration":0.010532788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":154,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.436861+0000","last_change":"2024-09-17T08:15:34.433273+0000","last_active":"2024-09-17T08:17:30.436861+0000","last_peered":"2024-09-17T08:17:30.436861+0000","last_clean":"2024-09-17T08:17:30.436861+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:17:30.436861+0000","last_undegraded":"2024-09-17T08:17:30.436861+0000","last_fullsized":"2024-09-17T08:17:30.436861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:34.432898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:34.432898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012758472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":171,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:30.438742+0000","last_change":"2024-09-17T08:15:48.646475+0000","last_active":"2024-09-17T08:17:30.438742+0000","last_peered":"2024-09-17T08:17:30.438742+0000","last_clean":"2024-09-17T08:17:30.438742+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:17:30.438742+0000","last_undegraded":"2024-09-17T08:17:30.438742+0000","last_fullsized":"2024-09-17T08:17:30.438742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.646329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.646329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076865340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":234,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.843906+0000","last_change":"2024-09-17T08:16:29.751857+0000","last_active":"2024-09-17T08:17:32.843906+0000","last_peered":"2024-09-17T08:17:32.843906+0000","last_clean":"2024-09-17T08:17:32.843906+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:17:32.843906+0000","last_undegraded":"2024-09-17T08:17:32.843906+0000","last_fullsized":"2024-09-17T08:17:32.843906+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:16:29.751693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.751693+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:48.331793+0000","scrub_duration":0.0054470619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":167,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T08:17:32.330657+0000","last_change":"2024-09-17T08:14:55.113389+0000","last_active":"2024-09-17T08:17:32.330657+0000","last_peered":"2024-09-17T08:17:32.330657+0000","last_clean":"2024-09-17T08:17:32.330657+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:17:32.330657+0000","last_undegraded":"2024-09-17T08:17:32.330657+0000","last_fullsized":"2024-09-17T08:17:32.330657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:14:55.113268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:14:55.113268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045114200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":7572815872,"num_objects":115552,"num_object_clones":0,"num_object_copies":346656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115552,"num_write_kb":7395328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22686072832,"data_stored":22686072832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83952,"ondisk_log_size":83952,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":690,"num_read_kb":447,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":451,"ondisk_log_size":451,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":139,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070563,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2459744,"kb_used_data":2288344,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":91285408,"statfs":{"total":95995035648,"available":93476257792,"internally_reserved":0,"allocated":2343264256,"data_stored":2342965533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":175503819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.3899999999999999,"15min":0.82699999999999996},"min":{"1min":0.253,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.665,"5min":27.751999999999999,"15min":27.751999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.5660000000000001,"15min":0.88500000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":4.0510000000000002,"5min":37.954000000000001,"15min":37.954000000000001},"last":1.0820000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.85399999999999998,"15min":0.68100000000000005},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.708,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.628},"min":{"1min":0.26100000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":1.105}]},{"osd":2,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.028,"15min":0.70099999999999996},"min":{"1min":0.38300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.423,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":0.72599999999999998},"min":{"1min":0.32700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":3.7469999999999999,"5min":4.8310000000000004,"15min":4.8310000000000004},"last":0.623}]},{"osd":3,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.3129999999999999,"15min":0.79100000000000004},"min":{"1min":0.39600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.625,"5min":11.298,"15min":11.298},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.397,"15min":0.85699999999999998},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.5499999999999998,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.45300000000000001},"min":{"1min":0.252,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1890000000000001,"5min":2.21,"15min":2.21},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.33200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.745,"5min":1.492,"15min":1.492},"last":0.44700000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":0.68300000000000005,"15min":0.55100000000000005},"min":{"1min":0.26100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":16.808,"5min":16.808,"15min":16.808},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.115,"5min":0.61699999999999999,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":11.842000000000001,"5min":11.842000000000001,"15min":11.842000000000001},"last":1.1279999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.5369999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.379}]}]},{"osd":6,"up_from":50,"seq":214748364904,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2443984,"kb_used_data":2278344,"kb_used_omap":1,"kb_used_meta":165630,"kb_avail":91301168,"statfs":{"total":95995035648,"available":93492396032,"internally_reserved":0,"allocated":2333024256,"data_stored":2332741177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":169605575},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":3.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.248,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":36.784999999999997,"5min":73.757999999999996,"15min":73.757999999999996},"last":3.1960000000000002},{"interface":"front","average":{"1min":2.7719999999999998,"5min":3.6709999999999998,"15min":1.5489999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":37.073,"5min":73.811000000000007,"15min":73.811000000000007},"last":1.1970000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.6829999999999998,"5min":1.5509999999999999,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":37.256999999999998,"5min":37.256999999999998,"15min":37.256999999999998},"last":3.5139999999999998},{"interface":"front","average":{"1min":2.9670000000000001,"5min":1.708,"15min":0.92300000000000004},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":37.415999999999997,"5min":37.415999999999997,"15min":37.415999999999997},"last":3.4870000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":1.5569999999999999,"15min":0.871},"min":{"1min":0.25700000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.6539999999999999,"5min":68.165000000000006,"15min":68.165000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.821,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4359999999999999,"5min":67.997,"15min":67.997},"last":0.29499999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.633,"5min":2.6829999999999998,"15min":1.2649999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":39.198999999999998,"5min":77.010999999999996,"15min":77.010999999999996},"last":0.37},{"interface":"front","average":{"1min":2.605,"5min":2.6259999999999999,"15min":1.236},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":36.804000000000002,"5min":76.375,"15min":76.375},"last":3.327}]},{"osd":4,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.629,"15min":0.45900000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.248},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.26600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.63300000000000001,"15min":0.47399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51000000000000001,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.34200000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.66100000000000003,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.31900000000000001}]}]},{"osd":5,"up_from":41,"seq":176093659245,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3890452,"kb_used_data":3660808,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":89854700,"statfs":{"total":95995035648,"available":92011212800,"internally_reserved":0,"allocated":3748667392,"data_stored":3748383718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":235141574},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":2.1000000000000001,"15min":1.0389999999999999},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":5.3079999999999998,"5min":63.098999999999997,"15min":63.098999999999997},"last":2.8450000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.948,"15min":0.97199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.0899999999999999,"5min":62.567,"15min":62.567},"last":0.85299999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.64000000000000001},"min":{"1min":0.33300000000000002,"5min":0.222,"15min":0.222},"max":{"1min":2.532,"5min":5.056,"15min":5.056},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.107,"15min":0.70299999999999996},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":2.4580000000000002,"5min":8.8119999999999994,"15min":8.8119999999999994},"last":0.92200000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.014,"15min":0.68899999999999995},"min":{"1min":0.247,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.589,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":4.3689999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.89300000000000002,"15min":0.63300000000000001},"min":{"1min":0.315,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.141,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":2.149,"15min":1.05},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.6259999999999999,"5min":75.286000000000001,"15min":75.286000000000001},"last":2.8780000000000001},{"interface":"front","average":{"1min":0.999,"5min":2.089,"15min":1.05},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.29499999999999998},"max":{"1min":5.8300000000000001,"5min":77.206999999999994,"15min":77.206999999999994},"last":0.376}]},{"osd":4,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.40699999999999997},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.19},"max":{"1min":1.1899999999999999,"5min":3.544,"15min":3.544},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":2.919}]},{"osd":6,"last update":"Tue Sep 17 08:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.2709999999999999,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":2.802},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.623,"15min":0.48699999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.254},"max":{"1min":5.1779999999999999,"5min":5.1779999999999999,"15min":5.1779999999999999},"last":0.44400000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.60199999999999998,"15min":0.51600000000000001},"min":{"1min":0.38100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3330000000000002,"5min":3.645,"15min":3.645},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.49199999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":1.0309999999999999,"5min":3.613,"15min":3.613},"last":0.55400000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953586,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2455772,"kb_used_data":2290132,"kb_used_omap":1,"kb_used_meta":165630,"kb_avail":91289380,"statfs":{"total":95995035648,"available":93480325120,"internally_reserved":0,"allocated":2345095168,"data_stored":2344799835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169605574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":1.3200000000000001,"15min":0.77300000000000002},"min":{"1min":0.253,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.2759999999999998,"5min":23.786000000000001,"15min":23.786000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.27,"15min":0.76300000000000001},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":1.4610000000000001,"5min":23.245000000000001,"15min":23.245000000000001},"last":4.0030000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":1.0569999999999999,"15min":0.69899999999999995},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":3.5779999999999998,"5min":23.135000000000002,"15min":23.135000000000002},"last":3.129},{"interface":"front","average":{"1min":0.70199999999999996,"5min":1.1339999999999999,"15min":0.71399999999999997},"min":{"1min":0.38600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.1640000000000001,"5min":23.434999999999999,"15min":23.434999999999999},"last":7.6180000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.81499999999999995,"15min":0.624},"min":{"1min":0.32000000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.94699999999999995,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":1.536},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.83299999999999996,"15min":0.60599999999999998},"min":{"1min":0.27100000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.016,"5min":3.4470000000000001,"15min":3.4470000000000001},"last":1.069}]},{"osd":3,"last update":"Tue Sep 17 08:16:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":3.3300000000000001,"15min":1.46},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.319,"5min":154.137,"15min":154.137},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":3.2109999999999999,"15min":1.4159999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.1800000000000002,"5min":153.54400000000001,"15min":153.54400000000001},"last":3.351}]},{"osd":5,"last update":"Tue Sep 17 08:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.504,"15min":0.432},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.9079999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.623,"15min":0.505},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.318}]},{"osd":7,"last update":"Tue Sep 17 08:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0229999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.57799999999999996}]}]},{"osd":3,"up_from":27,"seq":115964117110,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3032104,"kb_used_data":2744224,"kb_used_omap":1,"kb_used_meta":287870,"kb_avail":90713048,"statfs":{"total":95995035648,"available":92890161152,"internally_reserved":0,"allocated":2810085376,"data_stored":2809777232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":294779336},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.851,"5min":1.9790000000000001,"15min":0.90300000000000002},"min":{"1min":0.23799999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":5.1879999999999997,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.8140000000000001,"15min":0.85099999999999998},"min":{"1min":0.28999999999999998,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":3.3940000000000001,"5min":10.362,"15min":10.362},"last":0.79100000000000004}]},{"osd":1,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.825,"15min":0.879},"min":{"1min":0.25800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":4.7990000000000004,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.867,"15min":0.89400000000000002},"min":{"1min":0.374,"5min":0.188,"15min":0.188},"max":{"1min":3.0539999999999998,"5min":10.891999999999999,"15min":10.891999999999999},"last":1.7749999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.9319999999999999,"15min":0.91000000000000003},"min":{"1min":0.26400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":2.976,"5min":10.786,"15min":10.786},"last":0.29599999999999999},{"interface":"front","average":{"1min":1.8140000000000001,"5min":2.2669999999999999,"15min":1.0189999999999999},"min":{"1min":0.30299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.46,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":1.52}]},{"osd":4,"last update":"Tue Sep 17 08:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.373,"15min":0.81999999999999995},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":2.8460000000000001,"5min":7.742,"15min":7.742},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.2609999999999999,"15min":0.78100000000000003},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.9850000000000003,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":2.0710000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.4490000000000001,"15min":0.86199999999999999},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.4870000000000001,"5min":8.2240000000000002,"15min":8.2240000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.724,"5min":1.482,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.4359999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":5.6070000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.393,"15min":0.85299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4870000000000001,"5min":10.945,"15min":10.945},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.796,"5min":1.6339999999999999,"15min":0.92900000000000005},"min":{"1min":0.41699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":5.1360000000000001,"5min":5.798,"15min":5.798},"last":1.0309999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.4059999999999999,"15min":0.85999999999999999},"min":{"1min":0.437,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.4630000000000001,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":4.7069999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.3580000000000001,"15min":0.84499999999999997},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5179999999999998,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":1.756}]}]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3004436,"kb_used_data":2774796,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":90740716,"statfs":{"total":95995035648,"available":92918493184,"internally_reserved":0,"allocated":2841391104,"data_stored":2841103732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":235141574},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":2.0459999999999998,"15min":0.90200000000000002},"min":{"1min":0.20999999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":7.1600000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":2.0459999999999998},{"interface":"front","average":{"1min":1.163,"5min":2.226,"15min":0.97899999999999998},"min":{"1min":0.19,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":7.0419999999999998,"5min":23.341000000000001,"15min":23.341000000000001},"last":6.6619999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.575,"15min":0.77000000000000002},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":3.2829999999999999,"5min":6.1360000000000001,"15min":6.1360000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.4059999999999999,"15min":0.72799999999999998},"min":{"1min":0.22500000000000001,"5min":0.184,"15min":0.184},"max":{"1min":4.7930000000000001,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":6.0289999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":2.2370000000000001,"15min":1.0129999999999999},"min":{"1min":0.246,"5min":0.23499999999999999,"15min":0.20300000000000001},"max":{"1min":36.792999999999999,"5min":36.792999999999999,"15min":36.792999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":6.7859999999999996,"5min":2.9940000000000002,"15min":1.288},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":56.533000000000001,"5min":56.533000000000001,"15min":56.533000000000001},"last":6.3380000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.3899999999999999,"15min":0.83699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.5939999999999999,"5min":13.58,"15min":13.58},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.427,"15min":0.85199999999999998},"min":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.3820000000000001,"5min":13.818,"15min":13.818},"last":5.9930000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":1.1639999999999999,"15min":0.752},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.8280000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":1.2090000000000001,"15min":0.76100000000000001},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9910000000000001,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.349,"15min":0.83599999999999997},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.9379999999999999,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3169999999999999,"15min":0.84699999999999998},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.4399999999999999,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":6.5190000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.206,"15min":0.77700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6989999999999998,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":1.212,"15min":0.77400000000000002},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7210000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3486000,"kb_used_data":3234216,"kb_used_omap":1,"kb_used_meta":251774,"kb_avail":90259152,"statfs":{"total":95995035648,"available":92425371648,"internally_reserved":0,"allocated":3311837184,"data_stored":3311521438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257817030},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":2.1509999999999998,"15min":0.90900000000000003},"min":{"1min":0.25,"5min":0.16,"15min":0.16},"max":{"1min":5.8879999999999999,"5min":31.585999999999999,"15min":31.585999999999999},"last":2.1110000000000002},{"interface":"front","average":{"1min":1.2589999999999999,"5min":2.3029999999999999,"15min":0.97399999999999998},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":7.1740000000000004,"5min":35.807000000000002,"15min":35.807000000000002},"last":2.3639999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:17:02 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.5169999999999999,"15min":0.73699999999999999},"min":{"1min":0.23999999999999999,"5min":0.20100000000000001,"15min":0.19700000000000001},"max":{"1min":4.7009999999999996,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":2.8900000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.675,"15min":0.81899999999999995},"min":{"1min":0.28899999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.1740000000000004,"5min":10.638,"15min":10.638},"last":2.2650000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8460000000000001,"5min":1.6890000000000001,"15min":0.82699999999999996},"min":{"1min":0.26100000000000001,"5min":0.23300000000000001,"15min":0.20300000000000001},"max":{"1min":5.2759999999999998,"5min":7.8499999999999996,"15min":7.8499999999999996},"last":3.9780000000000002},{"interface":"front","average":{"1min":1.696,"5min":1.6399999999999999,"15min":0.81499999999999995},"min":{"1min":0.35499999999999998,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":6.2759999999999998,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":3.6459999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.1950000000000001,"15min":0.755},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.708,"5min":7.4859999999999998,"15min":7.4859999999999998},"last":2.2879999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.121,"15min":0.73999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.194,"5min":10.208,"15min":10.208},"last":2.0339999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:17:07 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.169,"15min":0.746},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.0090000000000003,"5min":5.9249999999999998,"15min":5.9249999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.3300000000000001,"15min":0.80900000000000005},"min":{"1min":0.45600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.383,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":2.3140000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.242,"15min":0.76700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.919,"5min":10.253,"15min":10.253},"last":2.0819999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.3109999999999999,"15min":0.82699999999999996},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2590000000000003,"5min":7.2590000000000003,"15min":7.2590000000000003},"last":1.0620000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.3069999999999999,"15min":0.81599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.6859999999999999,"5min":10.066000000000001,"15min":10.066000000000001},"last":2.3340000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.2569999999999999,"15min":0.81499999999999995},"min":{"1min":0.43099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":3.0339999999999998,"5min":10.279,"15min":10.279},"last":2.0619999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738498,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3032860,"kb_used_data":2763028,"kb_used_omap":1,"kb_used_meta":269822,"kb_avail":90712292,"statfs":{"total":95995035648,"available":92889387008,"internally_reserved":0,"allocated":2829340672,"data_stored":2829045729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276298182},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":5.6630000000000003,"5min":2.3660000000000001,"15min":1.024},"min":{"1min":0.30499999999999999,"5min":0.21299999999999999,"15min":0.19900000000000001},"max":{"1min":61.649000000000001,"5min":61.649000000000001,"15min":61.649000000000001},"last":0.189},{"interface":"front","average":{"1min":5.452,"5min":2.6859999999999999,"15min":1.1399999999999999},"min":{"1min":0.183,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":61.281999999999996,"5min":61.281999999999996,"15min":61.281999999999996},"last":0.433}]},{"osd":2,"last update":"Tue Sep 17 08:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.6799999999999999,"15min":0.81000000000000005},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.8100000000000001,"5min":12.355,"15min":12.355},"last":0.373},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.6759999999999999,"15min":0.82599999999999996},"min":{"1min":0.216,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":8.0489999999999995,"5min":8.0489999999999995,"15min":8.0489999999999995},"last":0.46300000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:17:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6879999999999997,"5min":2.903,"15min":1.254},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.20999999999999999},"max":{"1min":61.926000000000002,"5min":61.926000000000002,"15min":61.926000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":5.9459999999999997,"5min":2.6840000000000002,"15min":1.194},"min":{"1min":0.32500000000000001,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":61.616,"5min":61.616,"15min":61.616},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Sep 17 08:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":1.2729999999999999,"15min":0.79200000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.635,"5min":5.6890000000000001,"15min":5.6890000000000001},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.2689999999999999,"15min":0.81000000000000005},"min":{"1min":0.41199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.581,"5min":6.548,"15min":6.548},"last":0.83299999999999996}]},{"osd":5,"last update":"Tue Sep 17 08:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1619999999999999,"15min":0.76200000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":5.5499999999999998,"5min":7.835,"15min":7.835},"last":0.502},{"interface":"front","average":{"1min":0.81100000000000005,"5min":1.3720000000000001,"15min":0.83899999999999997},"min":{"1min":0.435,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":6.8789999999999996,"15min":6.8789999999999996},"last":0.48299999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.228,"15min":0.79900000000000004},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.4800000000000004,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.4159999999999999,"15min":0.86699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.4529999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":1.931}]},{"osd":7,"last update":"Tue Sep 17 08:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":1.706,"15min":0.94699999999999995},"min":{"1min":0.38100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.861,"5min":31.585999999999999,"15min":31.585999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.754,"5min":1.5309999999999999,"15min":0.89600000000000002},"min":{"1min":0.42999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":2.3519999999999999,"5min":30.225000000000001,"15min":30.225000000000001},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2846883840,"data_stored":2846883840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3312713728,"data_stored":3312713728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2849832960,"data_stored":2849832960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2838364160,"data_stored":2838364160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2361851904,"data_stored":2361851904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3780837376,"data_stored":3780837376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2349203456,"data_stored":2349203456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2346385408,"data_stored":2346385408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:17:48.711 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-17T08:17:48.712 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:17:48.712 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T08:17:48.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:17:48.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:48.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:48.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:17:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:17:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:17:48.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:17:48.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:48 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:17:48.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:48.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:48.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:48.975 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:49.005 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:49.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:49.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:17:49.119 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:17:49.120 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:48 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:17:49.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.162 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:49.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:49.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:49.300 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:49.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.301 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:49.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:49.435 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:49.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:49.511 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:49.531 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:17:49.531793+0000 min lat: 0.00121752 max lat: 0.193327 avg lat: 0.0103622 2024-09-17T08:17:49.531 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:17:49.531 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 80 16 123500 123484 96.4504 69.9375 0.0184818 0.0103622 2024-09-17T08:17:49.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:49.600 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:49.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:49.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:49.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:49.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:49.805 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:17:49.805 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:17:49.842 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:49.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:49.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:49.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 5.5 deep-scrub starts 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 5.5 deep-scrub ok 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:17:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:49 smithi173 ceph-mon[33187]: pgmap v337: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:49.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:49.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:49.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:49.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:50.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:50.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 5.5 deep-scrub starts 2024-09-17T08:17:50.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 5.5 deep-scrub ok 2024-09-17T08:17:50.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:17:50.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[29565]: pgmap v337: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 5.5 deep-scrub starts 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 5.5 deep-scrub ok 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:17:50.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:49 smithi079 ceph-mon[40649]: pgmap v337: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:17:50.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:50.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:50.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:50.277 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:50.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:50.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:50.410 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:50.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:50.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:50.532 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 81 16 124348 124332 95.9129 53 0.0104024 0.010421 2024-09-17T08:17:50.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:50.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:50.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:17:50.655 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:50.701 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:50.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:50.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T08:17:50.842 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/85436506' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:50.843 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:50.843 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:17:50.843 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:50 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:17:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:17:50.851 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:50.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:50.882 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:50.953 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:50.979 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:50.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:50.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:51.080 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:51.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:51.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:51.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:17:51.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:17:51.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:17:51.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/85436506' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T08:17:51.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/85436506' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:51.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:17:51.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:17:51.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:50 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:17:51.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:51.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:51.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:51.358 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:51.398 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:51.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:51.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:51.532 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 82 16 125446 125430 95.5801 68.625 0.0235571 0.0104573 2024-09-17T08:17:51.538 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:51.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:51.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:51.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:51.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:51.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:51.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:51.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:51.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:51.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:51.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:52.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:52.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:52.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:52.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:17:52.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:17:52.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[29565]: pgmap v340: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T08:17:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:51 smithi079 ceph-mon[40649]: pgmap v340: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-17T08:17:52.173 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:17:52.173 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:17:52.173 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:17:52.173 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:17:52.174 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T08:17:52.174 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:51 smithi173 ceph-mon[33187]: pgmap v340: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 732 op/s 2024-09-17T08:17:52.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:52.216 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:52.318 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:52.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.350 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:52.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:52.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:52.533 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 83 16 126722 126706 95.3883 79.75 0.013382 0.0104793 2024-09-17T08:17:52.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:52.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:52.722 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:52.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.764 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:52.795 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:17:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:17:52.852 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:17:52.853 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:17:52.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:52.906 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:52.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:52.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:53.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:53.041 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:53.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:53.143 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:53.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:53.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:53.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:53.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:53.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:17:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:17:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:17:53.422 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:53.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:53.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:53.533 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 84 16 127984 127968 95.1917 78.875 0.01178 0.0105009 2024-09-17T08:17:53.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:53.591 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:53.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:53.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:17:53.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:53.728 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:53.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:53.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:53.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:53.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:53.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:53.976 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:53.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:54.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:54.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:54.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:54.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:54.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.177 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:54.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:54.301 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:17:54.301 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:17:54.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:54.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:54.333 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:17:54.334 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:17:54.363 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:17:54.363 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:17:54.402 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:17:54.402 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:17:54.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:54.435 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:17:54.436 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:17:54.460 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:54.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.461 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:54.466 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:17:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:17:54.467 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:54.467 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[40649]: pgmap v341: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:17:54.467 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4165595569' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:54.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:54.467 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[29565]: pgmap v341: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:17:54.468 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4165595569' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:54.496 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:17:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:17:54.533 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 85 16 129178 129162 94.9497 74.625 0.0148028 0.0105271 2024-09-17T08:17:54.539 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:17:54.539 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:17:54.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:54.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:54.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:54.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:54.715 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:54.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:17:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:54 smithi173 ceph-mon[33187]: pgmap v341: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:17:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4165595569' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:17:54.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:17:54.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:54.841 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:54.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:54.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:54.974 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:54.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:54.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:55.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:55.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:55.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:55.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:55.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:55.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:55.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:55.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:55.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:17:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 192, 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:17:50.827793+0000", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:09:41.129965+0000", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:15:28.332713+0000", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5801984, 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:17:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 18, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 8, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:17:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.417 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.418 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.419 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.420 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:17:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.5, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.423 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.424 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.426 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.427 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:17:55.428 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:17:55.429 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.430 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.431 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.432 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:17:55.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9900000095367432, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9900000095367432, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1699999570846558, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:17:55.434 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.435 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:17:55.436 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.437 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.438 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.8799999952316284, 2024-09-17T08:17:55.439 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.8799999952316284, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.5, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.5, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:17:55.441 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.442 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.443 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.444 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.445 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.446 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:17:55.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.448 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.449 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5, 2024-09-17T08:17:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 8, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_1", 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:16:15.207121+0000", 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:17:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:17:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "170", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:17:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:17:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:17:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:17:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:17:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 165, 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:17:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:17:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:17:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 165, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:17:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 172, 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:17:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 383883652 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/383883652", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/383883652", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/383883652", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/383883652", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:17:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 172, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:17:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 172, 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:17:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 103, 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:17:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:17:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 91, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:17:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:17:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 165, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:17:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:17:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:17:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:16:39.698143+0000", 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:17:13.493701+0000", 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:16:37.627807+0000", 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:16:36.461140+0000", 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:17:07.797968+0000", 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:17:09.511414+0000", 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:17:20.362047+0000", 2024-09-17T08:17:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:17:49.562116+0000", 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:17:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:17:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:17:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:17:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:17:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:17:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:17:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:17:55.517 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:17:55.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:55.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 86 16 130225 130209 94.6048 65.4375 0.00897222 0.0105659 2024-09-17T08:17:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:55.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:55.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:55.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:55.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2794877955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:17:55.786 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:55.822 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:55.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:55.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2794877955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:17:55.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2794877955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:17:55.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:17:55.946 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:55.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:55.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:17:56.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:56.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:56.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:56.209 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:56.311 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:56.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:56.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:21658 7.d 7:b8f077bb:::benchmark_data_smithi079_83551_object21657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 330.740766415, 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.031442645999999998, 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.471 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 21658 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727081+0000", 2024-09-17T08:17:56.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727080+0000", 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727136+0000", 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5928999999999999e-05 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727147+0000", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1473e-05 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727153+0000", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0599999999999998e-06 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727188+0000", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5267e-05 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727275+0000", 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7330000000000006e-05 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.727369+0000", 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3528000000000003e-05 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.728187+0000", 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00081786399999999996 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.731982+0000", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003795344 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758465+0000", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.026482793000000001 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758496+0000", 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0979000000000003e-05 2024-09-17T08:17:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:25.758524+0000", 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8070000000000001e-05 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:26092 7.d 7:b0769b2a:::benchmark_data_smithi079_83551_object26091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:17:56.477 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 328.74846571400002, 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.028404776999999999, 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 26092 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719382+0000", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719381+0000", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719497+0000", 2024-09-17T08:17:56.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115707 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719504+0000", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7069999999999992e-06 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.719510+0000", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7729999999999996e-06 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721232+0000", 2024-09-17T08:17:56.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017213860000000001 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721364+0000", 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013278500000000001 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.721512+0000", 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014775899999999999 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.722256+0000", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074349699999999997 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.724025+0000", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017694690000000001 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747692+0000", 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023667157000000001 2024-09-17T08:17:56.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747741+0000", 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9162999999999997e-05 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:27.747787+0000", 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5074999999999998e-05 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28716 7.7 7:edba59f3:::benchmark_data_smithi079_83551_object28715:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.491921768, 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.063303783000000002, 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.484 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.485 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.485 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28716 2024-09-17T08:17:56.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.485 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975925+0000", 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976073+0000", 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014827699999999999 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976086+0000", 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2918000000000001e-05 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976096+0000", 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0256e-05 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976145+0000", 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9243999999999998e-05 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976258+0000", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011240500000000001 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976379+0000", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121612 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.977391+0000", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001011685 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033061+0000", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055669996999999999 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039074+0000", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0060129399999999996 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039129+0000", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4897e-05 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039229+0000", 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100722 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28722 7.7 7:e67c57be:::benchmark_data_smithi079_83551_object28721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.48860754700002, 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.060131948999999997, 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28722 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979239+0000", 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979357+0000", 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011785099999999999 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979367+0000", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0438999999999999e-05 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979374+0000", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0400000000000004e-06 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979415+0000", 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1279999999999998e-05 2024-09-17T08:17:56.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979502+0000", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6149999999999993e-05 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979586+0000", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4332999999999994e-05 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.980492+0000", 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090632099999999997 2024-09-17T08:17:56.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033104+0000", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.052611766999999997 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039310+0000", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0062060930000000002 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039339+0000", 2024-09-17T08:17:56.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8484999999999999e-05 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039372+0000", 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3337000000000003e-05 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29213 7.c 7:3e440d7d:::benchmark_data_smithi079_83551_object29212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.21931663700002, 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057673473000000003, 2024-09-17T08:17:56.497 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29213 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248530+0000", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248900+0000", 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000370066 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248909+0000", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9749999999999996e-06 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248914+0000", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8389999999999997e-06 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248950+0000", 2024-09-17T08:17:56.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.595e-05 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249061+0000", 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110703 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249260+0000", 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000199182 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252421+0000", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031614400000000002 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252454+0000", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3216999999999997e-05 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306147+0000", 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.053692036999999998 2024-09-17T08:17:56.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306179+0000", 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2332000000000001e-05 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306204+0000", 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5429999999999999e-05 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29214 7.d 7:bb0590f4:::benchmark_data_smithi079_83551_object29213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.21892220799998, 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057144951999999999, 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29214 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248925+0000", 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249567+0000", 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064241899999999998 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249580+0000", 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2741e-05 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249586+0000", 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4869999999999999e-06 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249651+0000", 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4521000000000001e-05 2024-09-17T08:17:56.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249739+0000", 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8483e-05 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249811+0000", 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1867999999999997e-05 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250845+0000", 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001034285 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251620+0000", 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00077503800000000003 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.305985+0000", 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.054364282 2024-09-17T08:17:56.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306043+0000", 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8400999999999999e-05 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306070+0000", 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7157000000000001e-05 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29221 7.c 7:3e2e93ed:::benchmark_data_smithi079_83551_object29220:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.21791919499998, 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.058164050000000002, 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29221 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249927+0000", 2024-09-17T08:17:56.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250103+0000", 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581399999999999 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250116+0000", 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3127e-05 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250124+0000", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7419999999999997e-06 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251107+0000", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098316799999999998 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251265+0000", 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015859700000000001 2024-09-17T08:17:56.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251404+0000", 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000138496 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252513+0000", 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011087200000000001 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252648+0000", 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000135081 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308041+0000", 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055393851000000001 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308072+0000", 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0790999999999997e-05 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308092+0000", 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0032000000000001e-05 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29227 7.7 7:ebe30d19:::benchmark_data_smithi079_83551_object29226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 327.21466536999998, 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.054569469000000002, 2024-09-17T08:17:56.517 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29227 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253182+0000", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253181+0000", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253277+0000", 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5808000000000002e-05 2024-09-17T08:17:56.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253286+0000", 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8939999999999999e-06 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253290+0000", 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3139999999999997e-06 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253325+0000", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4477000000000002e-05 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253406+0000", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1000000000000004e-05 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.253499+0000", 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3505999999999995e-05 2024-09-17T08:17:56.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.254519+0000", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00102016 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306321+0000", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.051802061000000003 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307680+0000", 2024-09-17T08:17:56.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013587880000000001 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307712+0000", 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1399000000000002e-05 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.307752+0000", 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9876999999999998e-05 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30807 7.7 7:e1a9b60f:::benchmark_data_smithi079_83551_object30806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44681916100001, 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15767790800000001, 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30807 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.524 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021027+0000", 2024-09-17T08:17:56.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021092+0000", 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5506000000000006e-05 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021104+0000", 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1472e-05 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021115+0000", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1080000000000001e-05 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021160+0000", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5367000000000001e-05 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021239+0000", 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8437999999999997e-05 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021321+0000", 2024-09-17T08:17:56.528 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:56.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2540999999999995e-05 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022432+0000", 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001110457 2024-09-17T08:17:56.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025758+0000", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003326552 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178623+0000", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.152864151 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178669+0000", 2024-09-17T08:17:56.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6896e-05 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178706+0000", 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6767000000000002e-05 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30809 7.d 7:b03b464e:::benchmark_data_smithi079_83551_object30808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44525294699997, 2024-09-17T08:17:56.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15486507099999999, 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30809 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022593+0000", 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022677+0000", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3442999999999994e-05 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022694+0000", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7031e-05 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022702+0000", 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.028e-06 2024-09-17T08:17:56.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022744+0000", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2491999999999999e-05 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022838+0000", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3326999999999996e-05 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022969+0000", 2024-09-17T08:17:56.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131068 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.023921+0000", 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00095237200000000005 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175140+0000", 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15121870300000001 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177383+0000", 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022433729999999999 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177423+0000", 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9823000000000001e-05 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177460+0000", 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6634999999999997e-05 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.537 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30813 7.7 7:e02af656:::benchmark_data_smithi079_83551_object30812:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.443795325, 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15475839, 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30813 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.538 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024046+0000", 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999938 2024-09-17T08:17:56.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024174+0000", 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000128133 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024192+0000", 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8287999999999998e-05 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024200+0000", 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3900000000000004e-06 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024259+0000", 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9525999999999999e-05 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024392+0000", 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132946 2024-09-17T08:17:56.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024549+0000", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015661000000000001 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178587+0000", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.154037904 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178743+0000", 2024-09-17T08:17:56.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015638800000000001 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178750+0000", 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5570000000000001e-06 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178783+0000", 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3241999999999998e-05 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178811+0000", 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7668999999999999e-05 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30816 7.d 7:bfa0a9cd:::benchmark_data_smithi079_83551_object30815:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44363467800002, 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.153399859, 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.544 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30816 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024212+0000", 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024268+0000", 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5563000000000003e-05 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.546 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024283+0000", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5176e-05 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024290+0000", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0079999999999996e-06 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024333+0000", 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2933999999999999e-05 2024-09-17T08:17:56.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024447+0000", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114708 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024706+0000", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025843399999999999 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177353+0000", 2024-09-17T08:17:56.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15264659 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177528+0000", 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581300000000001 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177538+0000", 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6139999999999998e-06 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177581+0000", 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2889999999999998e-05 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177613+0000", 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1819999999999997e-05 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44359919499999, 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084112399999999, 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024247+0000", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024253+0000", 2024-09-17T08:17:56.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3569999999999997e-06 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024263+0000", 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0562e-05 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024279+0000", 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5896000000000002e-05 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025745+0000", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001465767 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025753+0000", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9100000000000005e-06 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175089+0000", 2024-09-17T08:17:56.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.149336513 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30809 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44327627899997, 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15057779399999999, 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024570+0000", 2024-09-17T08:17:56.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024602+0000", 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1566000000000002e-05 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024612+0000", 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9510000000000001e-06 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024618+0000", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2609999999999999e-06 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175046+0000", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.150427808 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175130+0000", 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3523000000000007e-05 2024-09-17T08:17:56.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175149+0000", 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9442000000000001e-05 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30817 7.c 7:3940e3c2:::benchmark_data_smithi079_83551_object30816:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:17:56.559 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44324624900003, 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15284112799999999, 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30817 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024590+0000", 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999989 2024-09-17T08:17:56.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024982+0000", 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000392063 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024994+0000", 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1858000000000001e-05 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025005+0000", 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0288999999999999e-05 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025057+0000", 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2791999999999999e-05 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025172+0000", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011491499999999999 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025294+0000", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012128 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174906+0000", 2024-09-17T08:17:56.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961192100000001 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177330+0000", 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002424538 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177361+0000", 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1254999999999998e-05 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177406+0000", 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4196000000000002e-05 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177442+0000", 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6826999999999998e-05 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44204730000001, 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15291164700000001, 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025799+0000", 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025805+0000", 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1129999999999997e-06 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175419+0000", 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961376500000001 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175431+0000", 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2131e-05 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178612+0000", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031804939999999999 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9100000000000002e-06 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178712+0000", 2024-09-17T08:17:56.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4178999999999994e-05 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30816 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.44133521700002, 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084441900000001, 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026517+0000", 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5160000000000002e-06 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174997+0000", 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14847996599999999 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175009+0000", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1552999999999999e-05 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177340+0000", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023309110000000002 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177347+0000", 2024-09-17T08:17:56.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3560000000000004e-06 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177357+0000", 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7340000000000003e-06 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30820 7.d 7:b5459376:::benchmark_data_smithi079_83551_object30819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 326.440985216, 2024-09-17T08:17:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15420378100000001, 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30820 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026861+0000", 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026944+0000", 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3053999999999995e-05 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174921+0000", 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14797671100000001 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174933+0000", 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1926000000000001e-05 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176058+0000", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011258220000000001 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176161+0000", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102187 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176275+0000", 2024-09-17T08:17:56.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114832 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177625+0000", 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001349996 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.179664+0000", 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002038467 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.180981+0000", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013172679999999999 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181030+0000", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9128999999999997e-05 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181066+0000", 2024-09-17T08:17:56.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5720999999999999e-05 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:32607 7.7 7:e4b06c5d:::benchmark_data_smithi079_83551_object32606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 325.49601056300003, 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.05146564, 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.582 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 32607 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971837+0000", 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971836+0000", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971928+0000", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1908000000000002e-05 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971941+0000", 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3608000000000001e-05 2024-09-17T08:17:56.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971948+0000", 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5150000000000003e-06 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.971992+0000", 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3769000000000001e-05 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.972081+0000", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9649999999999997e-05 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.972166+0000", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4598000000000006e-05 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.973794+0000", 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0016286359999999999 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.974962+0000", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00116788 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023221+0000", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.048258471999999997 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023268+0000", 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7465e-05 2024-09-17T08:17:56.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.023303+0000", 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4328000000000001e-05 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.588 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.589 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 87 16 131383 131367 94.3493 72.375 0.0106093 0.0105943 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:33202 7.7 7:e6f9aec6:::benchmark_data_smithi079_83551_object33201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 325.16517358200002, 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.030856884000000001, 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:17:56.589 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 33202 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302674+0000", 2024-09-17T08:17:56.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302673+0000", 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302828+0000", 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015509399999999999 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302840+0000", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2853e-05 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.302846+0000", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0370000000000002e-06 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303493+0000", 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064765700000000005 2024-09-17T08:17:56.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303615+0000", 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012233599999999999 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.303760+0000", 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014483899999999999 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:17:56.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.304637+0000", 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00087705499999999996 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.309162+0000", 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0045242379999999999 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333437+0000", 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024274917999999999 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333488+0000", 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1175999999999999e-05 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:31.333531+0000", 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3040999999999997e-05 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:56.595 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:56.629 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:56.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:56 smithi173 ceph-mon[33187]: pgmap v342: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:17:56.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:56.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:17:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:56 smithi079 ceph-mon[29565]: pgmap v342: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:17:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:56 smithi079 ceph-mon[40649]: pgmap v342: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:17:56.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:56.937 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:56.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:56.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:57.039 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:57.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:57.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:57.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:57.209 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:57.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:57.310 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:57.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:57.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:57.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:57.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:57.499 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:17:57.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 88 16 132652 132636 94.1784 79.3125 0.0182842 0.010614 2024-09-17T08:17:57.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:57.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:57.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:57.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:57.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:57.831 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:17:57.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:57.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:57.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:57.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:57.973 INFO:tasks.thrashosds.thrasher:j is {'epoch': 192, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:17:50.827793+0000', 'last_up_change': '2024-09-17T08:09:41.129965+0000', 'last_in_change': '2024-09-17T08:15:28.332713+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 1.1699999570846558, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-17T08:16:15.207121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 165, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 165, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 383883652}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 383883652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 383883652}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 383883652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 383883652}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 383883652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 383883652}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 383883652}]}, 'public_addr': '172.21.15.79:6819/383883652', 'cluster_addr': '172.21.15.79:6821/383883652', 'heartbeat_back_addr': '172.21.15.79:6825/383883652', 'heartbeat_front_addr': '172.21.15.79:6823/383883652', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 165, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:16:39.698143+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:17:13.493701+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:16:37.627807+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:16:36.461140+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:17:07.797968+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:17:09.511414+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:17:20.362047+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:17:49.562116+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:17:57.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:17:57.998 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:57.999 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:58.032 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:58.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:58.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:58.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:58.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:58.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:17:58.321 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:58.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:17:58.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:58.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:58.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:58.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 89 16 133824 133808 93.9432 73.25 0.0136953 0.0106405 2024-09-17T08:17:58.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:58.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:58.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:58.705 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:58.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:17:58 smithi173 ceph-mon[33187]: pgmap v343: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-17T08:17:58.738 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:58.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:58.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:17:58 smithi079 ceph-mon[40649]: pgmap v343: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-17T08:17:58.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:17:58 smithi079 ceph-mon[29565]: pgmap v343: 148 pgs: 148 active+clean; 7.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-17T08:17:58.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:17:58.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:58.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:58.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:17:58.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:59.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:59.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:59.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:17:59.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:17:59.120 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:17:59.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:59.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:17:59.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:17:59.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:59.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:17:59.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:17:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:59.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:17:59.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:59.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total time run: 90.0065 2024-09-17T08:17:59.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total writes made: 135017 2024-09-17T08:17:59.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Write size: 65536 2024-09-17T08:17:59.535 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object size: 65536 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Bandwidth (MB/sec): 93.7551 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Bandwidth: 33.7105 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max bandwidth (MB/sec): 230.5 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min bandwidth (MB/sec): 53 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average IOPS: 1500 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev IOPS: 539.368 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max IOPS: 3688 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min IOPS: 848 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average Latency(s): 0.0106623 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Latency(s): 0.00841315 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max latency(s): 0.193327 2024-09-17T08:17:59.536 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min latency(s): 0.00121752 2024-09-17T08:17:59.539 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T08:17:59.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:59.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:59.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:59.693 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:59.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:135259 8.5 e192/165 8:a59d9709:::benchmark_data_smithi079_115040_object239:head v 192'8420, mlcod=192'8418)", 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:59.719821+0000", 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0055077620000000002, 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055370089999999999, 2024-09-17T08:17:59.727 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719821+0000", 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719821+0000", 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719820+0000", 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719828+0000", 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8849999999999999e-06 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:59.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719841+0000", 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3266e-05 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719848+0000", 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9979999999999997e-06 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719913+0000", 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5084000000000002e-05 2024-09-17T08:17:59.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.719932+0000", 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9300000000000002e-05 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:135265 8.6 e192/165 8:60d30f09:::benchmark_data_smithi079_115040_object245:head v 192'8409, mlcod=192'8407)", 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:59.723526+0000", 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0018028460000000001, 2024-09-17T08:17:59.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018936199999999999, 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723526+0000", 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723526+0000", 2024-09-17T08:17:59.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723525+0000", 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723533+0000", 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0069999999999997e-06 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:59.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723549+0000", 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6639000000000001e-05 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723556+0000", 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1799999999999999e-06 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723663+0000", 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010671999999999999 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723681+0000", 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8311000000000001e-05 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:135264 8.5 e192/165 8:adac0082:::benchmark_data_smithi079_115040_object244:head v 192'8421, mlcod=192'8418)", 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:59.723575+0000", 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0017532100000000001, 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001896972, 2024-09-17T08:17:59.735 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723575+0000", 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723575+0000", 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723575+0000", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723579+0000", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7000000000000002e-06 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723585+0000", 2024-09-17T08:17:59.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1090000000000003e-06 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723589+0000", 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1409999999999998e-06 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723637+0000", 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8449999999999999e-05 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:17:59.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:59.723657+0000", 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0103999999999999e-05 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 3 2024-09-17T08:17:59.739 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:17:59.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:17:59.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:17:59.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:17:59.959 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:17:59.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:17:59.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:17:59.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:00.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:00.133 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:00.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:00.235 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:00.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:00.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:00.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:00.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:00.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:00.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:00.538 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:00.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:00.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:00 smithi079 ceph-mon[40649]: pgmap v344: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-17T08:18:00.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:00 smithi079 ceph-mon[29565]: pgmap v344: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-17T08:18:00.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:00.667 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.668 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:00.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:00 smithi173 ceph-mon[33187]: pgmap v344: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-17T08:18:00.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:00.804 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:00.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.806 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:00.842 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:18:00.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:00.954 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:00.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:00.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:01.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:01.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.097 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:01.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:01.163 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:18:01.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:01.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:01.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:01.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:01.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:01.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:01.455 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:18:01.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:01.522 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:01.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:01.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:01 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:01.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:01.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:01 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:01.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:01.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:01.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:01.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:01 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:18:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:01.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:01.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:01.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:01.902 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:01.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:01.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:01.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:02.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:02.070 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:02.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:02.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:02.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:02.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:02.310 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:02.344 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:02.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:02.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:02.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:02.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:02.539 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: pgmap v345: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 972 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: pgmap v345: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 972 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:18:02.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:02.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:02.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:02.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: pgmap v345: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 972 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3778471716' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:02.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:02.738 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:02.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:02.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:02.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:02.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:02.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:03.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:03.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:03.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:03.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:03.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:03.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:03.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:03.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:03.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:18:03.316 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:18:03.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:03.391 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:18:03.391 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:18:03.391 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:18:03.391 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:18:03.391 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:18:03.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:03.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:03.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:03.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:03.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:03.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:03.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:03.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:03 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:03 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:03 smithi173 ceph-mon[33187]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:03 smithi173 ceph-mon[33187]: pgmap v347: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-17T08:18:03.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:03.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:03.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:03.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:03.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[29565]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:03.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[29565]: pgmap v347: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-17T08:18:03.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:03.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:03.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:03.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[40649]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:03.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:03 smithi079 ceph-mon[40649]: pgmap v347: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-17T08:18:03.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:03.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:03.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:03.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:03.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:03.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:04.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:04.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:04.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:04.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:04.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:04.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:04.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:18:04.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:04.498 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:04.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:04.638 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:04 smithi173 ceph-mon[33187]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T08:18:04.738 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:18:04.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:04.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:04 smithi079 ceph-mon[29565]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T08:18:04.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:04 smithi079 ceph-mon[40649]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T08:18:04.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:04.943 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:04.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:04.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:05.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:05.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.072 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:05.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:05.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:05.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:05.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:05.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:05.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:05.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:05.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:05.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:05.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:05.595 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:05.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:05.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:05 smithi173 ceph-mon[33187]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T08:18:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:05 smithi173 ceph-mon[33187]: pgmap v350: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 22 MiB/s wr, 998 op/s 2024-09-17T08:18:05.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:05.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:05.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[29565]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[29565]: pgmap v350: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 22 MiB/s wr, 998 op/s 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[40649]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T08:18:05.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:05 smithi079 ceph-mon[40649]: pgmap v350: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 22 MiB/s wr, 998 op/s 2024-09-17T08:18:05.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:05.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:05.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:05.980 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:06.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:06.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:06.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:06.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:06.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:06.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:06.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.460 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:06.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:06.561 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:06.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.596 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:06.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:06.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:06.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:06.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:06.743 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-17T08:18:01.451178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8711641366,"num_objects":133116,"num_object_clones":0,"num_object_copies":399348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133116,"num_whiteouts":0,"num_read":1068,"num_read_kb":914,"num_write":133529,"num_write_kb":8509339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":1776389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84590,"ondisk_log_size":84590,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":27234852,"kb_used_data":25320676,"kb_used_omap":12,"kb_used_meta":1914099,"kb_avail":722726364,"statfs":{"total":767960285184,"available":740071796736,"internally_reserved":0,"allocated":25928372224,"data_stored":25925939146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":1960037944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,11,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":754647040,"num_objects":11515,"num_object_clones":0,"num_object_copies":34545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11515,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11515,"num_write_kb":736960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.092716"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858809+0000","last_change":"2024-09-17T08:17:34.931931+0000","last_active":"2024-09-17T08:17:49.858809+0000","last_peered":"2024-09-17T08:17:49.858809+0000","last_clean":"2024-09-17T08:17:49.858809+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:17:49.858809+0000","last_undegraded":"2024-09-17T08:17:49.858809+0000","last_fullsized":"2024-09-17T08:17:49.858809+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.931732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.931732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.380258+0000","scrub_duration":0.0066078860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.847554+0000","last_change":"2024-09-17T08:16:42.337270+0000","last_active":"2024-09-17T08:17:49.847554+0000","last_peered":"2024-09-17T08:17:49.847554+0000","last_clean":"2024-09-17T08:17:49.847554+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:17:49.847554+0000","last_undegraded":"2024-09-17T08:17:49.847554+0000","last_fullsized":"2024-09-17T08:17:49.847554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.337157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:08.710909+0000","scrub_duration":0.001642129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851289+0000","last_change":"2024-09-17T08:14:56.372401+0000","last_active":"2024-09-17T08:17:49.851289+0000","last_peered":"2024-09-17T08:17:49.851289+0000","last_clean":"2024-09-17T08:17:49.851289+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:17:49.851289+0000","last_undegraded":"2024-09-17T08:17:49.851289+0000","last_fullsized":"2024-09-17T08:17:49.851289+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:14:56.372270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:14:56.372270+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010005975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":187,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832338+0000","last_change":"2024-09-17T08:15:52.984443+0000","last_active":"2024-09-17T08:17:49.832338+0000","last_peered":"2024-09-17T08:17:49.832338+0000","last_clean":"2024-09-17T08:17:49.832338+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:17:49.832338+0000","last_undegraded":"2024-09-17T08:17:49.832338+0000","last_fullsized":"2024-09-17T08:17:49.832338+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:52.984337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.984337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030030090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":162,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852537+0000","last_change":"2024-09-17T08:15:43.592053+0000","last_active":"2024-09-17T08:17:49.852537+0000","last_peered":"2024-09-17T08:17:49.852537+0000","last_clean":"2024-09-17T08:17:49.852537+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:17:49.852537+0000","last_undegraded":"2024-09-17T08:17:49.852537+0000","last_fullsized":"2024-09-17T08:17:49.852537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:15:43.591667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013131347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":267,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832345+0000","last_change":"2024-09-17T08:16:35.827557+0000","last_active":"2024-09-17T08:17:49.832345+0000","last_peered":"2024-09-17T08:17:49.832345+0000","last_clean":"2024-09-17T08:17:49.832345+0000","last_became_active":"2024-09-17T08:16:33.326106+0000","last_became_peered":"2024-09-17T08:16:33.326106+0000","last_unstale":"2024-09-17T08:17:49.832345+0000","last_undegraded":"2024-09-17T08:17:49.832345+0000","last_fullsized":"2024-09-17T08:17:49.832345+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:16:35.827443+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.827443+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033149960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":63,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.852678+0000","last_change":"2024-09-17T08:16:44.335176+0000","last_active":"2024-09-17T08:17:50.852678+0000","last_peered":"2024-09-17T08:17:50.852678+0000","last_clean":"2024-09-17T08:17:50.852678+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:17:50.852678+0000","last_undegraded":"2024-09-17T08:17:50.852678+0000","last_fullsized":"2024-09-17T08:17:50.852678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.335057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.335057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065690419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":196,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832728+0000","last_change":"2024-09-17T08:16:48.900627+0000","last_active":"2024-09-17T08:17:49.832728+0000","last_peered":"2024-09-17T08:17:49.832728+0000","last_clean":"2024-09-17T08:17:49.832728+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:17:49.832728+0000","last_undegraded":"2024-09-17T08:17:49.832728+0000","last_fullsized":"2024-09-17T08:17:49.832728+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.900525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.943918+0000","scrub_duration":0.0033962240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":173,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852378+0000","last_change":"2024-09-17T08:16:33.826157+0000","last_active":"2024-09-17T08:17:49.852378+0000","last_peered":"2024-09-17T08:17:49.852378+0000","last_clean":"2024-09-17T08:17:49.852378+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:17:49.852378+0000","last_undegraded":"2024-09-17T08:17:49.852378+0000","last_fullsized":"2024-09-17T08:17:49.852378+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.826006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.826006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.282923+0000","scrub_duration":0.0026989420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":213,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832836+0000","last_change":"2024-09-17T08:15:31.963456+0000","last_active":"2024-09-17T08:17:49.832836+0000","last_peered":"2024-09-17T08:17:49.832836+0000","last_clean":"2024-09-17T08:17:49.832836+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:17:49.832836+0000","last_undegraded":"2024-09-17T08:17:49.832836+0000","last_fullsized":"2024-09-17T08:17:49.832836+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:15:31.963328+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:15:31.963328+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040865249999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":198,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850493+0000","last_change":"2024-09-17T08:16:49.012348+0000","last_active":"2024-09-17T08:17:49.850493+0000","last_peered":"2024-09-17T08:17:49.850493+0000","last_clean":"2024-09-17T08:17:49.850493+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:17:49.850493+0000","last_undegraded":"2024-09-17T08:17:49.850493+0000","last_fullsized":"2024-09-17T08:17:49.850493+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.012248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070975750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":63,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.853422+0000","last_change":"2024-09-17T08:16:23.065070+0000","last_active":"2024-09-17T08:17:50.853422+0000","last_peered":"2024-09-17T08:17:50.853422+0000","last_clean":"2024-09-17T08:17:50.853422+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:17:50.853422+0000","last_undegraded":"2024-09-17T08:17:50.853422+0000","last_fullsized":"2024-09-17T08:17:50.853422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.064993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.064993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015683839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852959+0000","last_change":"2024-09-17T08:17:01.908852+0000","last_active":"2024-09-17T08:17:49.852959+0000","last_peered":"2024-09-17T08:17:49.852959+0000","last_clean":"2024-09-17T08:17:49.852959+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:17:49.852959+0000","last_undegraded":"2024-09-17T08:17:49.852959+0000","last_fullsized":"2024-09-17T08:17:49.852959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.908732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.507561+0000","scrub_duration":0.0085101169999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":198,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833501+0000","last_change":"2024-09-17T08:16:46.894268+0000","last_active":"2024-09-17T08:17:49.833501+0000","last_peered":"2024-09-17T08:17:49.833501+0000","last_clean":"2024-09-17T08:17:49.833501+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:17:49.833501+0000","last_undegraded":"2024-09-17T08:17:49.833501+0000","last_fullsized":"2024-09-17T08:17:49.833501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:16:46.894134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.894134+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053467080000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":72,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844444+0000","last_change":"2024-09-17T08:15:54.069982+0000","last_active":"2024-09-17T08:17:49.844444+0000","last_peered":"2024-09-17T08:17:49.844444+0000","last_clean":"2024-09-17T08:17:49.844444+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:17:49.844444+0000","last_undegraded":"2024-09-17T08:17:49.844444+0000","last_fullsized":"2024-09-17T08:17:49.844444+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.069886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.069886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001445294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":196,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.847108+0000","last_change":"2024-09-17T08:17:31.421060+0000","last_active":"2024-09-17T08:17:49.847108+0000","last_peered":"2024-09-17T08:17:49.847108+0000","last_clean":"2024-09-17T08:17:49.847108+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:17:49.847108+0000","last_undegraded":"2024-09-17T08:17:49.847108+0000","last_fullsized":"2024-09-17T08:17:49.847108+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.420934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.420934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.381148+0000","scrub_duration":0.0086368369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"192'8318","reported_seq":8354,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.631648+0000","last_change":"2024-09-17T08:16:16.645652+0000","last_active":"2024-09-17T08:17:57.631648+0000","last_peered":"2024-09-17T08:17:57.631648+0000","last_clean":"2024-09-17T08:17:57.631648+0000","last_became_active":"2024-09-17T08:16:16.645277+0000","last_became_peered":"2024-09-17T08:16:16.645277+0000","last_unstale":"2024-09-17T08:17:57.631648+0000","last_undegraded":"2024-09-17T08:17:57.631648+0000","last_fullsized":"2024-09-17T08:17:57.631648+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5018,"log_dups_size":0,"ondisk_log_size":5018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545128448,"num_objects":8318,"num_object_clones":0,"num_object_copies":24954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8318,"num_write_kb":532352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":253,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833432+0000","last_change":"2024-09-17T08:16:26.873516+0000","last_active":"2024-09-17T08:17:49.833432+0000","last_peered":"2024-09-17T08:17:49.833432+0000","last_clean":"2024-09-17T08:17:49.833432+0000","last_became_active":"2024-09-17T08:15:29.362201+0000","last_became_peered":"2024-09-17T08:15:29.362201+0000","last_unstale":"2024-09-17T08:17:49.833432+0000","last_undegraded":"2024-09-17T08:17:49.833432+0000","last_fullsized":"2024-09-17T08:17:49.833432+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:16:26.873402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.873402+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002741856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850984+0000","last_change":"2024-09-17T08:17:36.164173+0000","last_active":"2024-09-17T08:17:49.850984+0000","last_peered":"2024-09-17T08:17:49.850984+0000","last_clean":"2024-09-17T08:17:49.850984+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:17:49.850984+0000","last_undegraded":"2024-09-17T08:17:49.850984+0000","last_fullsized":"2024-09-17T08:17:49.850984+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.164053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.164053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:45.324794+0000","scrub_duration":0.0014889840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":74,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851084+0000","last_change":"2024-09-17T08:17:46.704576+0000","last_active":"2024-09-17T08:17:49.851084+0000","last_peered":"2024-09-17T08:17:49.851084+0000","last_clean":"2024-09-17T08:17:49.851084+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:17:49.851084+0000","last_undegraded":"2024-09-17T08:17:49.851084+0000","last_fullsized":"2024-09-17T08:17:49.851084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:46.704456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:46.704456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.759310+0000","scrub_duration":0.0093420550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":189,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.857256+0000","last_change":"2024-09-17T08:17:49.298346+0000","last_active":"2024-09-17T08:17:49.857256+0000","last_peered":"2024-09-17T08:17:49.857256+0000","last_clean":"2024-09-17T08:17:49.857256+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:17:49.857256+0000","last_undegraded":"2024-09-17T08:17:49.857256+0000","last_fullsized":"2024-09-17T08:17:49.857256+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:49.295155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:49.295155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:55.253615+0000","scrub_duration":0.010469981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"192'8252","reported_seq":8286,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.884456+0000","last_change":"2024-09-17T08:16:16.648796+0000","last_active":"2024-09-17T08:17:57.884456+0000","last_peered":"2024-09-17T08:17:57.884456+0000","last_clean":"2024-09-17T08:17:57.884456+0000","last_became_active":"2024-09-17T08:16:16.648526+0000","last_became_peered":"2024-09-17T08:16:16.648526+0000","last_unstale":"2024-09-17T08:17:57.884456+0000","last_undegraded":"2024-09-17T08:17:57.884456+0000","last_fullsized":"2024-09-17T08:17:57.884456+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4952,"log_dups_size":0,"ondisk_log_size":4952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540803072,"num_objects":8252,"num_object_clones":0,"num_object_copies":24756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":528128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":197,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833667+0000","last_change":"2024-09-17T08:16:56.688647+0000","last_active":"2024-09-17T08:17:49.833667+0000","last_peered":"2024-09-17T08:17:49.833667+0000","last_clean":"2024-09-17T08:17:49.833667+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:17:49.833667+0000","last_undegraded":"2024-09-17T08:17:49.833667+0000","last_fullsized":"2024-09-17T08:17:49.833667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:56.688373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.688373+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.791670+0000","scrub_duration":0.0062131820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852348+0000","last_change":"2024-09-17T08:17:40.659072+0000","last_active":"2024-09-17T08:17:49.852348+0000","last_peered":"2024-09-17T08:17:49.852348+0000","last_clean":"2024-09-17T08:17:49.852348+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:17:49.852348+0000","last_undegraded":"2024-09-17T08:17:49.852348+0000","last_fullsized":"2024-09-17T08:17:49.852348+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.658816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:51.813043+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.658816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:47.720191+0000","scrub_duration":0.011311672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":296,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.842410+0000","last_change":"2024-09-17T08:17:42.416017+0000","last_active":"2024-09-17T08:17:50.842410+0000","last_peered":"2024-09-17T08:17:50.842410+0000","last_clean":"2024-09-17T08:17:50.842410+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:17:50.842410+0000","last_undegraded":"2024-09-17T08:17:50.842410+0000","last_fullsized":"2024-09-17T08:17:50.842410+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:17:42.415151+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:17:42.415151+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.003958+0000","scrub_duration":0.0143199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833102+0000","last_change":"2024-09-17T08:16:30.856483+0000","last_active":"2024-09-17T08:17:49.833102+0000","last_peered":"2024-09-17T08:17:49.833102+0000","last_clean":"2024-09-17T08:17:49.833102+0000","last_became_active":"2024-09-17T08:15:29.358693+0000","last_became_peered":"2024-09-17T08:15:29.358693+0000","last_unstale":"2024-09-17T08:17:49.833102+0000","last_undegraded":"2024-09-17T08:17:49.833102+0000","last_fullsized":"2024-09-17T08:17:49.833102+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:30.856368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.856368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044042839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"192'8188","reported_seq":8224,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.639959+0000","last_change":"2024-09-17T08:16:16.648125+0000","last_active":"2024-09-17T08:17:57.639959+0000","last_peered":"2024-09-17T08:17:57.639959+0000","last_clean":"2024-09-17T08:17:57.639959+0000","last_became_active":"2024-09-17T08:16:16.647816+0000","last_became_peered":"2024-09-17T08:16:16.647816+0000","last_unstale":"2024-09-17T08:17:57.639959+0000","last_undegraded":"2024-09-17T08:17:57.639959+0000","last_fullsized":"2024-09-17T08:17:57.639959+0000","mapping_epoch":165,"log_start":"177'3100","ondisk_log_start":"177'3100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5088,"log_dups_size":0,"ondisk_log_size":5088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536608768,"num_objects":8188,"num_object_clones":0,"num_object_copies":24564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8188,"num_write_kb":524032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":57,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.857058+0000","last_change":"2024-09-17T08:16:53.116193+0000","last_active":"2024-09-17T08:17:49.857058+0000","last_peered":"2024-09-17T08:17:49.857058+0000","last_clean":"2024-09-17T08:17:49.857058+0000","last_became_active":"2024-09-17T08:16:33.328511+0000","last_became_peered":"2024-09-17T08:16:33.328511+0000","last_unstale":"2024-09-17T08:17:49.857058+0000","last_undegraded":"2024-09-17T08:17:49.857058+0000","last_fullsized":"2024-09-17T08:17:49.857058+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:16:53.115891+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:16:53.115891+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.168215+0000","scrub_duration":0.0043675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850444+0000","last_change":"2024-09-17T08:16:40.302468+0000","last_active":"2024-09-17T08:17:49.850444+0000","last_peered":"2024-09-17T08:17:49.850444+0000","last_clean":"2024-09-17T08:17:49.850444+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:17:49.850444+0000","last_undegraded":"2024-09-17T08:17:49.850444+0000","last_fullsized":"2024-09-17T08:17:49.850444+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.302235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.302235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.381029+0000","scrub_duration":0.0071342690000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":273,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.836493+0000","last_change":"2024-09-17T08:15:40.042590+0000","last_active":"2024-09-17T08:17:50.836493+0000","last_peered":"2024-09-17T08:17:50.836493+0000","last_clean":"2024-09-17T08:17:50.836493+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:17:50.836493+0000","last_undegraded":"2024-09-17T08:17:50.836493+0000","last_fullsized":"2024-09-17T08:17:50.836493+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:15:40.042507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:40.042507+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048815059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851827+0000","last_change":"2024-09-17T08:16:56.421870+0000","last_active":"2024-09-17T08:17:49.851827+0000","last_peered":"2024-09-17T08:17:49.851827+0000","last_clean":"2024-09-17T08:17:49.851827+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:17:49.851827+0000","last_undegraded":"2024-09-17T08:17:49.851827+0000","last_fullsized":"2024-09-17T08:17:49.851827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:56.421736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.421736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.576854+0000","scrub_duration":0.0048071169999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"192'8338","reported_seq":8374,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.324891+0000","last_change":"2024-09-17T08:16:16.649421+0000","last_active":"2024-09-17T08:17:57.324891+0000","last_peered":"2024-09-17T08:17:57.324891+0000","last_clean":"2024-09-17T08:17:57.324891+0000","last_became_active":"2024-09-17T08:16:16.648969+0000","last_became_peered":"2024-09-17T08:16:16.648969+0000","last_unstale":"2024-09-17T08:17:57.324891+0000","last_undegraded":"2024-09-17T08:17:57.324891+0000","last_fullsized":"2024-09-17T08:17:57.324891+0000","mapping_epoch":165,"log_start":"176'3000","ondisk_log_start":"176'3000","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":546439168,"num_objects":8338,"num_object_clones":0,"num_object_copies":25014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8338,"num_write_kb":533632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"156'16","reported_seq":237,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833367+0000","last_change":"2024-09-17T08:15:44.029509+0000","last_active":"2024-09-17T08:17:49.833367+0000","last_peered":"2024-09-17T08:17:49.833367+0000","last_clean":"2024-09-17T08:17:49.833367+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:17:49.833367+0000","last_undegraded":"2024-09-17T08:17:49.833367+0000","last_fullsized":"2024-09-17T08:17:49.833367+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'16","last_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.029355+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011380917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850737+0000","last_change":"2024-09-17T08:16:37.745942+0000","last_active":"2024-09-17T08:17:49.850737+0000","last_peered":"2024-09-17T08:17:49.850737+0000","last_clean":"2024-09-17T08:17:49.850737+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:17:49.850737+0000","last_undegraded":"2024-09-17T08:17:49.850737+0000","last_fullsized":"2024-09-17T08:17:49.850737+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.745837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:03.563608+0000","scrub_duration":0.0044495799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":73,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833081+0000","last_change":"2024-09-17T08:16:25.867376+0000","last_active":"2024-09-17T08:17:49.833081+0000","last_peered":"2024-09-17T08:17:49.833081+0000","last_clean":"2024-09-17T08:17:49.833081+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:17:49.833081+0000","last_undegraded":"2024-09-17T08:17:49.833081+0000","last_fullsized":"2024-09-17T08:17:49.833081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:25.867267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:25.867267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015224139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.855381+0000","last_change":"2024-09-17T08:17:39.909841+0000","last_active":"2024-09-17T08:17:49.855381+0000","last_peered":"2024-09-17T08:17:49.855381+0000","last_clean":"2024-09-17T08:17:49.855381+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:17:49.855381+0000","last_undegraded":"2024-09-17T08:17:49.855381+0000","last_fullsized":"2024-09-17T08:17:49.855381+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.909586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.909586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.911382+0000","scrub_duration":0.001911444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"192'8269","reported_seq":8460,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.287590+0000","last_change":"2024-09-17T08:17:41.995541+0000","last_active":"2024-09-17T08:17:57.287590+0000","last_peered":"2024-09-17T08:17:57.287590+0000","last_clean":"2024-09-17T08:17:57.287590+0000","last_became_active":"2024-09-17T08:16:16.648937+0000","last_became_peered":"2024-09-17T08:16:16.648937+0000","last_unstale":"2024-09-17T08:17:57.287590+0000","last_undegraded":"2024-09-17T08:17:57.287590+0000","last_fullsized":"2024-09-17T08:17:57.287590+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'7187","last_scrub_stamp":"2024-09-17T08:17:41.995394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.995394+0000","objects_scrubbed":7135,"log_size":4969,"log_dups_size":0,"ondisk_log_size":4969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:50.208844+0000","scrub_duration":1.840802743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541917184,"num_objects":8269,"num_object_clones":0,"num_object_copies":24807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8269,"num_write_kb":529216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":192,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832715+0000","last_change":"2024-09-17T08:16:27.874195+0000","last_active":"2024-09-17T08:17:49.832715+0000","last_peered":"2024-09-17T08:17:49.832715+0000","last_clean":"2024-09-17T08:17:49.832715+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:17:49.832715+0000","last_undegraded":"2024-09-17T08:17:49.832715+0000","last_fullsized":"2024-09-17T08:17:49.832715+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:16:27.874083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.874083+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022658510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":135,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.213103+0000","last_change":"2024-09-17T08:17:50.213103+0000","last_active":"2024-09-17T08:17:50.213103+0000","last_peered":"2024-09-17T08:17:50.213103+0000","last_clean":"2024-09-17T08:17:50.213103+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:17:50.213103+0000","last_undegraded":"2024-09-17T08:17:50.213103+0000","last_fullsized":"2024-09-17T08:17:50.213103+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:50.212978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:17:50.212978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.476553+0000","scrub_duration":0.0039563619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":305,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.837942+0000","last_change":"2024-09-17T08:16:48.678053+0000","last_active":"2024-09-17T08:17:50.837942+0000","last_peered":"2024-09-17T08:17:50.837942+0000","last_clean":"2024-09-17T08:17:50.837942+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:17:50.837942+0000","last_undegraded":"2024-09-17T08:17:50.837942+0000","last_fullsized":"2024-09-17T08:17:50.837942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.677913+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.360519+0000","scrub_duration":0.032798792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851047+0000","last_change":"2024-09-17T08:16:47.964627+0000","last_active":"2024-09-17T08:17:49.851047+0000","last_peered":"2024-09-17T08:17:49.851047+0000","last_clean":"2024-09-17T08:17:49.851047+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:17:49.851047+0000","last_undegraded":"2024-09-17T08:17:49.851047+0000","last_fullsized":"2024-09-17T08:17:49.851047+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.964546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.603179+0000","scrub_duration":0.0015627130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"192'8218","reported_seq":8254,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:58.105217+0000","last_change":"2024-09-17T08:16:16.404499+0000","last_active":"2024-09-17T08:17:58.105217+0000","last_peered":"2024-09-17T08:17:58.105217+0000","last_clean":"2024-09-17T08:17:58.105217+0000","last_became_active":"2024-09-17T08:16:16.404227+0000","last_became_peered":"2024-09-17T08:16:16.404227+0000","last_unstale":"2024-09-17T08:17:58.105217+0000","last_undegraded":"2024-09-17T08:17:58.105217+0000","last_fullsized":"2024-09-17T08:17:58.105217+0000","mapping_epoch":165,"log_start":"176'2500","ondisk_log_start":"176'2500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538574848,"num_objects":8218,"num_object_clones":0,"num_object_copies":24654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":525952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":203,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.853290+0000","last_change":"2024-09-17T08:15:52.770015+0000","last_active":"2024-09-17T08:17:49.853290+0000","last_peered":"2024-09-17T08:17:49.853290+0000","last_clean":"2024-09-17T08:17:49.853290+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:17:49.853290+0000","last_undegraded":"2024-09-17T08:17:49.853290+0000","last_fullsized":"2024-09-17T08:17:49.853290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:15:52.769743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.769743+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.023040108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851487+0000","last_change":"2024-09-17T08:17:02.482908+0000","last_active":"2024-09-17T08:17:49.851487+0000","last_peered":"2024-09-17T08:17:49.851487+0000","last_clean":"2024-09-17T08:17:49.851487+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:17:49.851487+0000","last_undegraded":"2024-09-17T08:17:49.851487+0000","last_fullsized":"2024-09-17T08:17:49.851487+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:02.482796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.482796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:24.312024+0000","scrub_duration":0.006488972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":169,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832608+0000","last_change":"2024-09-17T08:15:48.938396+0000","last_active":"2024-09-17T08:17:49.832608+0000","last_peered":"2024-09-17T08:17:49.832608+0000","last_clean":"2024-09-17T08:17:49.832608+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:17:49.832608+0000","last_undegraded":"2024-09-17T08:17:49.832608+0000","last_fullsized":"2024-09-17T08:17:49.832608+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.938247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.938247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00524202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832995+0000","last_change":"2024-09-17T08:15:35.024756+0000","last_active":"2024-09-17T08:17:49.832995+0000","last_peered":"2024-09-17T08:17:49.832995+0000","last_clean":"2024-09-17T08:17:49.832995+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:17:49.832995+0000","last_undegraded":"2024-09-17T08:17:49.832995+0000","last_fullsized":"2024-09-17T08:17:49.832995+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:35.024623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:35.024623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010817126999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"192'8283","reported_seq":8319,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.830417+0000","last_change":"2024-09-17T08:16:16.647250+0000","last_active":"2024-09-17T08:17:57.830417+0000","last_peered":"2024-09-17T08:17:57.830417+0000","last_clean":"2024-09-17T08:17:57.830417+0000","last_became_active":"2024-09-17T08:16:16.646899+0000","last_became_peered":"2024-09-17T08:16:16.646899+0000","last_unstale":"2024-09-17T08:17:57.830417+0000","last_undegraded":"2024-09-17T08:17:57.830417+0000","last_fullsized":"2024-09-17T08:17:57.830417+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":542834688,"num_objects":8283,"num_object_clones":0,"num_object_copies":24849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8283,"num_write_kb":530112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":218,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844371+0000","last_change":"2024-09-17T08:16:48.235582+0000","last_active":"2024-09-17T08:17:49.844371+0000","last_peered":"2024-09-17T08:17:49.844371+0000","last_clean":"2024-09-17T08:17:49.844371+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:17:49.844371+0000","last_undegraded":"2024-09-17T08:17:49.844371+0000","last_fullsized":"2024-09-17T08:17:49.844371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:16:48.235424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.235424+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.405506+0000","scrub_duration":0.0061761419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":189,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852454+0000","last_change":"2024-09-17T08:17:35.141768+0000","last_active":"2024-09-17T08:17:49.852454+0000","last_peered":"2024-09-17T08:17:49.852454+0000","last_clean":"2024-09-17T08:17:49.852454+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:17:49.852454+0000","last_undegraded":"2024-09-17T08:17:49.852454+0000","last_fullsized":"2024-09-17T08:17:49.852454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:17:35.141551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.141551+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.706752+0000","scrub_duration":0.0071014449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":136,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844443+0000","last_change":"2024-09-17T08:17:44.468391+0000","last_active":"2024-09-17T08:17:49.844443+0000","last_peered":"2024-09-17T08:17:49.844443+0000","last_clean":"2024-09-17T08:17:49.844443+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:17:49.844443+0000","last_undegraded":"2024-09-17T08:17:49.844443+0000","last_fullsized":"2024-09-17T08:17:49.844443+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:17:44.468114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:12.768554+0000","scrub_duration":0.003219296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":74,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850575+0000","last_change":"2024-09-17T08:17:44.618063+0000","last_active":"2024-09-17T08:17:49.850575+0000","last_peered":"2024-09-17T08:17:49.850575+0000","last_clean":"2024-09-17T08:17:49.850575+0000","last_became_active":"2024-09-17T08:15:29.372791+0000","last_became_peered":"2024-09-17T08:15:29.372791+0000","last_unstale":"2024-09-17T08:17:49.850575+0000","last_undegraded":"2024-09-17T08:17:49.850575+0000","last_fullsized":"2024-09-17T08:17:49.850575+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:44.617613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:44.617613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.250750+0000","scrub_duration":0.0062292040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"192'8313","reported_seq":8348,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:58.118693+0000","last_change":"2024-09-17T08:16:16.645635+0000","last_active":"2024-09-17T08:17:58.118693+0000","last_peered":"2024-09-17T08:17:58.118693+0000","last_clean":"2024-09-17T08:17:58.118693+0000","last_became_active":"2024-09-17T08:16:16.645315+0000","last_became_peered":"2024-09-17T08:16:16.645315+0000","last_unstale":"2024-09-17T08:17:58.118693+0000","last_undegraded":"2024-09-17T08:17:58.118693+0000","last_fullsized":"2024-09-17T08:17:58.118693+0000","mapping_epoch":165,"log_start":"176'2600","ondisk_log_start":"176'2600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5713,"log_dups_size":0,"ondisk_log_size":5713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544800768,"num_objects":8313,"num_object_clones":0,"num_object_copies":24939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8313,"num_write_kb":532032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"156'15","reported_seq":80,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850830+0000","last_change":"2024-09-17T08:15:44.010651+0000","last_active":"2024-09-17T08:17:49.850830+0000","last_peered":"2024-09-17T08:17:49.850830+0000","last_clean":"2024-09-17T08:17:49.850830+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:17:49.850830+0000","last_undegraded":"2024-09-17T08:17:49.850830+0000","last_fullsized":"2024-09-17T08:17:49.850830+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'15","last_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.010126+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012291415999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852232+0000","last_change":"2024-09-17T08:17:37.532454+0000","last_active":"2024-09-17T08:17:49.852232+0000","last_peered":"2024-09-17T08:17:49.852232+0000","last_clean":"2024-09-17T08:17:49.852232+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:17:49.852232+0000","last_undegraded":"2024-09-17T08:17:49.852232+0000","last_fullsized":"2024-09-17T08:17:49.852232+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.532313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.532313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:43.540050+0000","scrub_duration":0.012255401000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":285,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:52.920226+0000","last_change":"2024-09-17T08:17:00.237348+0000","last_active":"2024-09-17T08:17:52.920226+0000","last_peered":"2024-09-17T08:17:52.920226+0000","last_clean":"2024-09-17T08:17:52.920226+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:17:52.920226+0000","last_undegraded":"2024-09-17T08:17:52.920226+0000","last_fullsized":"2024-09-17T08:17:52.920226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:17:00.237229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.237229+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:10.239107+0000","scrub_duration":0.004406877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.847001+0000","last_change":"2024-09-17T08:16:35.335671+0000","last_active":"2024-09-17T08:17:49.847001+0000","last_peered":"2024-09-17T08:17:49.847001+0000","last_clean":"2024-09-17T08:17:49.847001+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:17:49.847001+0000","last_undegraded":"2024-09-17T08:17:49.847001+0000","last_fullsized":"2024-09-17T08:17:49.847001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.335551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.335551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:50.895726+0000","scrub_duration":0.007005336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"192'8408","reported_seq":8444,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.872271+0000","last_change":"2024-09-17T08:16:16.648773+0000","last_active":"2024-09-17T08:17:57.872271+0000","last_peered":"2024-09-17T08:17:57.872271+0000","last_clean":"2024-09-17T08:17:57.872271+0000","last_became_active":"2024-09-17T08:16:16.648491+0000","last_became_peered":"2024-09-17T08:16:16.648491+0000","last_unstale":"2024-09-17T08:17:57.872271+0000","last_undegraded":"2024-09-17T08:17:57.872271+0000","last_fullsized":"2024-09-17T08:17:57.872271+0000","mapping_epoch":165,"log_start":"177'3400","ondisk_log_start":"177'3400","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":551026688,"num_objects":8408,"num_object_clones":0,"num_object_copies":25224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8408,"num_write_kb":538112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"152'30","reported_seq":115,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.848629+0000","last_change":"2024-09-17T08:17:41.611847+0000","last_active":"2024-09-17T08:17:50.848629+0000","last_peered":"2024-09-17T08:17:50.848629+0000","last_clean":"2024-09-17T08:17:50.848629+0000","last_became_active":"2024-09-17T08:15:29.385461+0000","last_became_peered":"2024-09-17T08:15:29.385461+0000","last_unstale":"2024-09-17T08:17:50.848629+0000","last_undegraded":"2024-09-17T08:17:50.848629+0000","last_fullsized":"2024-09-17T08:17:50.848629+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'30","last_scrub_stamp":"2024-09-17T08:17:41.611678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.611678+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.598787+0000","scrub_duration":0.006094777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851001+0000","last_change":"2024-09-17T08:15:53.067473+0000","last_active":"2024-09-17T08:17:49.851001+0000","last_peered":"2024-09-17T08:17:49.851001+0000","last_clean":"2024-09-17T08:17:49.851001+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:17:49.851001+0000","last_undegraded":"2024-09-17T08:17:49.851001+0000","last_fullsized":"2024-09-17T08:17:49.851001+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:15:53.067191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081478090000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":198,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844742+0000","last_change":"2024-09-17T08:17:48.455405+0000","last_active":"2024-09-17T08:17:49.844742+0000","last_peered":"2024-09-17T08:17:49.844742+0000","last_clean":"2024-09-17T08:17:49.844742+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:17:49.844742+0000","last_undegraded":"2024-09-17T08:17:49.844742+0000","last_fullsized":"2024-09-17T08:17:49.844742+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.455290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.494744+0000","scrub_duration":0.0015673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852310+0000","last_change":"2024-09-17T08:15:44.574330+0000","last_active":"2024-09-17T08:17:49.852310+0000","last_peered":"2024-09-17T08:17:49.852310+0000","last_clean":"2024-09-17T08:17:49.852310+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:17:49.852310+0000","last_undegraded":"2024-09-17T08:17:49.852310+0000","last_fullsized":"2024-09-17T08:17:49.852310+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:44.574209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.574209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050670539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"192'8271","reported_seq":8305,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.863229+0000","last_change":"2024-09-17T08:16:16.402968+0000","last_active":"2024-09-17T08:17:57.863229+0000","last_peered":"2024-09-17T08:17:57.863229+0000","last_clean":"2024-09-17T08:17:57.863229+0000","last_became_active":"2024-09-17T08:16:16.402701+0000","last_became_peered":"2024-09-17T08:16:16.402701+0000","last_unstale":"2024-09-17T08:17:57.863229+0000","last_undegraded":"2024-09-17T08:17:57.863229+0000","last_fullsized":"2024-09-17T08:17:57.863229+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":4971,"log_dups_size":0,"ondisk_log_size":4971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":542048256,"num_objects":8271,"num_object_clones":0,"num_object_copies":24813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8271,"num_write_kb":529344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":75,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.872349+0000","last_change":"2024-09-17T08:16:33.206426+0000","last_active":"2024-09-17T08:17:49.872349+0000","last_peered":"2024-09-17T08:17:49.872349+0000","last_clean":"2024-09-17T08:17:49.872349+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:17:49.872349+0000","last_undegraded":"2024-09-17T08:17:49.872349+0000","last_fullsized":"2024-09-17T08:17:49.872349+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:16:33.206032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.206032+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0092541819999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":215,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:55.832667+0000","last_change":"2024-09-17T08:15:39.964221+0000","last_active":"2024-09-17T08:17:55.832667+0000","last_peered":"2024-09-17T08:17:55.832667+0000","last_clean":"2024-09-17T08:17:55.832667+0000","last_became_active":"2024-09-17T08:15:29.370817+0000","last_became_peered":"2024-09-17T08:15:29.370817+0000","last_unstale":"2024-09-17T08:17:55.832667+0000","last_undegraded":"2024-09-17T08:17:55.832667+0000","last_fullsized":"2024-09-17T08:17:55.832667+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:15:39.964058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:39.964058+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015935661,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":154,"num_read_kb":149,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.861653+0000","last_change":"2024-09-17T08:17:48.327422+0000","last_active":"2024-09-17T08:17:49.861653+0000","last_peered":"2024-09-17T08:17:49.861653+0000","last_clean":"2024-09-17T08:17:49.861653+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:17:49.861653+0000","last_undegraded":"2024-09-17T08:17:49.861653+0000","last_fullsized":"2024-09-17T08:17:49.861653+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.327064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.327064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.338047+0000","scrub_duration":0.0096668850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858239+0000","last_change":"2024-09-17T08:16:52.152636+0000","last_active":"2024-09-17T08:17:49.858239+0000","last_peered":"2024-09-17T08:17:49.858239+0000","last_clean":"2024-09-17T08:17:49.858239+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:17:49.858239+0000","last_undegraded":"2024-09-17T08:17:49.858239+0000","last_fullsized":"2024-09-17T08:17:49.858239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:52.152367+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.152367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.124044+0000","scrub_duration":0.0020852959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852008+0000","last_change":"2024-09-17T08:16:44.868243+0000","last_active":"2024-09-17T08:17:49.852008+0000","last_peered":"2024-09-17T08:17:49.852008+0000","last_clean":"2024-09-17T08:17:49.852008+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:17:49.852008+0000","last_undegraded":"2024-09-17T08:17:49.852008+0000","last_fullsized":"2024-09-17T08:17:49.852008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.868012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.868012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:06.438510+0000","scrub_duration":0.0025635359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"192'8457","reported_seq":8493,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.629190+0000","last_change":"2024-09-17T08:16:16.648374+0000","last_active":"2024-09-17T08:17:57.629190+0000","last_peered":"2024-09-17T08:17:57.629190+0000","last_clean":"2024-09-17T08:17:57.629190+0000","last_became_active":"2024-09-17T08:16:16.647832+0000","last_became_peered":"2024-09-17T08:16:16.647832+0000","last_unstale":"2024-09-17T08:17:57.629190+0000","last_undegraded":"2024-09-17T08:17:57.629190+0000","last_fullsized":"2024-09-17T08:17:57.629190+0000","mapping_epoch":165,"log_start":"177'3400","ondisk_log_start":"177'3400","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554237952,"num_objects":8457,"num_object_clones":0,"num_object_copies":25371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8457,"num_write_kb":541248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":252,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.845127+0000","last_change":"2024-09-17T08:17:02.332268+0000","last_active":"2024-09-17T08:17:49.845127+0000","last_peered":"2024-09-17T08:17:49.845127+0000","last_clean":"2024-09-17T08:17:49.845127+0000","last_became_active":"2024-09-17T08:15:29.379763+0000","last_became_peered":"2024-09-17T08:15:29.379763+0000","last_unstale":"2024-09-17T08:17:49.845127+0000","last_undegraded":"2024-09-17T08:17:49.845127+0000","last_fullsized":"2024-09-17T08:17:49.845127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:17:02.331862+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.331862+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.727563+0000","scrub_duration":0.0094874130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.846277+0000","last_change":"2024-09-17T08:16:50.181218+0000","last_active":"2024-09-17T08:17:49.846277+0000","last_peered":"2024-09-17T08:17:49.846277+0000","last_clean":"2024-09-17T08:17:49.846277+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:17:49.846277+0000","last_undegraded":"2024-09-17T08:17:49.846277+0000","last_fullsized":"2024-09-17T08:17:49.846277+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:16:50.181067+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:56.169610+0000","scrub_duration":0.0098447959999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.840333+0000","last_change":"2024-09-17T08:16:35.781222+0000","last_active":"2024-09-17T08:17:50.840333+0000","last_peered":"2024-09-17T08:17:50.840333+0000","last_clean":"2024-09-17T08:17:50.840333+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:17:50.840333+0000","last_undegraded":"2024-09-17T08:17:50.840333+0000","last_fullsized":"2024-09-17T08:17:50.840333+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:35.781112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.781112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040414090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.873095+0000","last_change":"2024-09-17T08:15:47.973188+0000","last_active":"2024-09-17T08:17:49.873095+0000","last_peered":"2024-09-17T08:17:49.873095+0000","last_clean":"2024-09-17T08:17:49.873095+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:17:49.873095+0000","last_undegraded":"2024-09-17T08:17:49.873095+0000","last_fullsized":"2024-09-17T08:17:49.873095+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:47.973067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:47.973067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046832360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":63,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852105+0000","last_change":"2024-09-17T08:16:42.309390+0000","last_active":"2024-09-17T08:17:49.852105+0000","last_peered":"2024-09-17T08:17:49.852105+0000","last_clean":"2024-09-17T08:17:49.852105+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:17:49.852105+0000","last_undegraded":"2024-09-17T08:17:49.852105+0000","last_fullsized":"2024-09-17T08:17:49.852105+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.309103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.309103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057864980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"192'8323","reported_seq":8358,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:58.075291+0000","last_change":"2024-09-17T08:16:16.648023+0000","last_active":"2024-09-17T08:17:58.075291+0000","last_peered":"2024-09-17T08:17:58.075291+0000","last_clean":"2024-09-17T08:17:58.075291+0000","last_became_active":"2024-09-17T08:16:16.647748+0000","last_became_peered":"2024-09-17T08:16:16.647748+0000","last_unstale":"2024-09-17T08:17:58.075291+0000","last_undegraded":"2024-09-17T08:17:58.075291+0000","last_fullsized":"2024-09-17T08:17:58.075291+0000","mapping_epoch":165,"log_start":"176'2600","ondisk_log_start":"176'2600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":545456128,"num_objects":8323,"num_object_clones":0,"num_object_copies":24969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8323,"num_write_kb":532672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"139'21","reported_seq":166,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833296+0000","last_change":"2024-09-17T08:17:37.634899+0000","last_active":"2024-09-17T08:17:49.833296+0000","last_peered":"2024-09-17T08:17:49.833296+0000","last_clean":"2024-09-17T08:17:49.833296+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:17:49.833296+0000","last_undegraded":"2024-09-17T08:17:49.833296+0000","last_fullsized":"2024-09-17T08:17:49.833296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'21","last_scrub_stamp":"2024-09-17T08:17:37.634598+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.634598+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.942132+0000","scrub_duration":0.0074777940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851864+0000","last_change":"2024-09-17T08:17:36.901495+0000","last_active":"2024-09-17T08:17:49.851864+0000","last_peered":"2024-09-17T08:17:49.851864+0000","last_clean":"2024-09-17T08:17:49.851864+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:17:49.851864+0000","last_undegraded":"2024-09-17T08:17:49.851864+0000","last_fullsized":"2024-09-17T08:17:49.851864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:17:36.901360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.901360+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:41.027333+0000","scrub_duration":0.002432104,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":73,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856435+0000","last_change":"2024-09-17T08:16:43.234496+0000","last_active":"2024-09-17T08:17:49.856435+0000","last_peered":"2024-09-17T08:17:49.856435+0000","last_clean":"2024-09-17T08:17:49.856435+0000","last_became_active":"2024-09-17T08:14:58.717890+0000","last_became_peered":"2024-09-17T08:14:58.717890+0000","last_unstale":"2024-09-17T08:17:49.856435+0000","last_undegraded":"2024-09-17T08:17:49.856435+0000","last_fullsized":"2024-09-17T08:17:49.856435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:16:43.234376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.329605+0000","scrub_duration":0.0026197199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":446,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850983+0000","last_change":"2024-09-17T08:17:47.212851+0000","last_active":"2024-09-17T08:17:49.850983+0000","last_peered":"2024-09-17T08:17:49.850983+0000","last_clean":"2024-09-17T08:17:49.850983+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:17:49.850983+0000","last_undegraded":"2024-09-17T08:17:49.850983+0000","last_fullsized":"2024-09-17T08:17:49.850983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.209749+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:11.287491+0000","scrub_duration":0.014148275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851448+0000","last_change":"2024-09-17T08:17:48.223952+0000","last_active":"2024-09-17T08:17:49.851448+0000","last_peered":"2024-09-17T08:17:49.851448+0000","last_clean":"2024-09-17T08:17:49.851448+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:17:49.851448+0000","last_undegraded":"2024-09-17T08:17:49.851448+0000","last_fullsized":"2024-09-17T08:17:49.851448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.223800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.223800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:17.257739+0000","scrub_duration":0.003735787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":72,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852959+0000","last_change":"2024-09-17T08:16:33.884267+0000","last_active":"2024-09-17T08:17:49.852959+0000","last_peered":"2024-09-17T08:17:49.852959+0000","last_clean":"2024-09-17T08:17:49.852959+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:17:49.852959+0000","last_undegraded":"2024-09-17T08:17:49.852959+0000","last_fullsized":"2024-09-17T08:17:49.852959+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.884139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.884139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"192'8292","reported_seq":8328,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:58.105697+0000","last_change":"2024-09-17T08:16:16.646700+0000","last_active":"2024-09-17T08:17:58.105697+0000","last_peered":"2024-09-17T08:17:58.105697+0000","last_clean":"2024-09-17T08:17:58.105697+0000","last_became_active":"2024-09-17T08:16:16.646368+0000","last_became_peered":"2024-09-17T08:16:16.646368+0000","last_unstale":"2024-09-17T08:17:58.105697+0000","last_undegraded":"2024-09-17T08:17:58.105697+0000","last_fullsized":"2024-09-17T08:17:58.105697+0000","mapping_epoch":165,"log_start":"176'2600","ondisk_log_start":"176'2600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543424512,"num_objects":8292,"num_object_clones":0,"num_object_copies":24876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8292,"num_write_kb":530688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":94,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851083+0000","last_change":"2024-09-17T08:17:49.689057+0000","last_active":"2024-09-17T08:17:49.851083+0000","last_peered":"2024-09-17T08:17:49.851083+0000","last_clean":"2024-09-17T08:17:49.851083+0000","last_became_active":"2024-09-17T08:15:29.360821+0000","last_became_peered":"2024-09-17T08:15:29.360821+0000","last_unstale":"2024-09-17T08:17:49.851083+0000","last_undegraded":"2024-09-17T08:17:49.851083+0000","last_fullsized":"2024-09-17T08:17:49.851083+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:17:49.688552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:49.688552+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:10.450394+0000","scrub_duration":0.0066490250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.867076+0000","last_change":"2024-09-17T08:17:40.401393+0000","last_active":"2024-09-17T08:17:49.867076+0000","last_peered":"2024-09-17T08:17:49.867076+0000","last_clean":"2024-09-17T08:17:49.867076+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:17:49.867076+0000","last_undegraded":"2024-09-17T08:17:49.867076+0000","last_fullsized":"2024-09-17T08:17:49.867076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.401262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.401262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:53.178795+0000","scrub_duration":0.0066464740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856361+0000","last_change":"2024-09-17T08:17:36.372735+0000","last_active":"2024-09-17T08:17:49.856361+0000","last_peered":"2024-09-17T08:17:49.856361+0000","last_clean":"2024-09-17T08:17:49.856361+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:17:49.856361+0000","last_undegraded":"2024-09-17T08:17:49.856361+0000","last_fullsized":"2024-09-17T08:17:49.856361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.372603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.372603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.970558+0000","scrub_duration":0.010983404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850898+0000","last_change":"2024-09-17T08:17:34.135923+0000","last_active":"2024-09-17T08:17:49.850898+0000","last_peered":"2024-09-17T08:17:49.850898+0000","last_clean":"2024-09-17T08:17:49.850898+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:17:49.850898+0000","last_undegraded":"2024-09-17T08:17:49.850898+0000","last_fullsized":"2024-09-17T08:17:49.850898+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.135505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.135505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.113488+0000","scrub_duration":0.01331624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"192'8385","reported_seq":8421,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.650393+0000","last_change":"2024-09-17T08:16:16.650552+0000","last_active":"2024-09-17T08:17:57.650393+0000","last_peered":"2024-09-17T08:17:57.650393+0000","last_clean":"2024-09-17T08:17:57.650393+0000","last_became_active":"2024-09-17T08:16:16.650307+0000","last_became_peered":"2024-09-17T08:16:16.650307+0000","last_unstale":"2024-09-17T08:17:57.650393+0000","last_undegraded":"2024-09-17T08:17:57.650393+0000","last_fullsized":"2024-09-17T08:17:57.650393+0000","mapping_epoch":165,"log_start":"177'3300","ondisk_log_start":"177'3300","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549519360,"num_objects":8385,"num_object_clones":0,"num_object_copies":25155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":536640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":236,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844298+0000","last_change":"2024-09-17T08:17:47.420671+0000","last_active":"2024-09-17T08:17:49.844298+0000","last_peered":"2024-09-17T08:17:49.844298+0000","last_clean":"2024-09-17T08:17:49.844298+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:17:49.844298+0000","last_undegraded":"2024-09-17T08:17:49.844298+0000","last_fullsized":"2024-09-17T08:17:49.844298+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:47.420536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.420536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.300439+0000","scrub_duration":0.010761165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":74,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851695+0000","last_change":"2024-09-17T08:17:45.666143+0000","last_active":"2024-09-17T08:17:49.851695+0000","last_peered":"2024-09-17T08:17:49.851695+0000","last_clean":"2024-09-17T08:17:49.851695+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:17:49.851695+0000","last_undegraded":"2024-09-17T08:17:49.851695+0000","last_fullsized":"2024-09-17T08:17:49.851695+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:45.666044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:17:45.666044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:56.542414+0000","scrub_duration":0.0048391390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.857939+0000","last_change":"2024-09-17T08:17:31.297792+0000","last_active":"2024-09-17T08:17:49.857939+0000","last_peered":"2024-09-17T08:17:49.857939+0000","last_clean":"2024-09-17T08:17:49.857939+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:17:49.857939+0000","last_undegraded":"2024-09-17T08:17:49.857939+0000","last_fullsized":"2024-09-17T08:17:49.857939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.297237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.297237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.099318+0000","scrub_duration":0.0044744609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.846201+0000","last_change":"2024-09-17T08:16:54.185896+0000","last_active":"2024-09-17T08:17:49.846201+0000","last_peered":"2024-09-17T08:17:49.846201+0000","last_clean":"2024-09-17T08:17:49.846201+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:17:49.846201+0000","last_undegraded":"2024-09-17T08:17:49.846201+0000","last_fullsized":"2024-09-17T08:17:49.846201+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:54.185311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:54.185311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:16.656330+0000","scrub_duration":0.0041021820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"192'8367","reported_seq":8403,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.310994+0000","last_change":"2024-09-17T08:16:16.649082+0000","last_active":"2024-09-17T08:17:57.310994+0000","last_peered":"2024-09-17T08:17:57.310994+0000","last_clean":"2024-09-17T08:17:57.310994+0000","last_became_active":"2024-09-17T08:16:16.648709+0000","last_became_peered":"2024-09-17T08:16:16.648709+0000","last_unstale":"2024-09-17T08:17:57.310994+0000","last_undegraded":"2024-09-17T08:17:57.310994+0000","last_fullsized":"2024-09-17T08:17:57.310994+0000","mapping_epoch":165,"log_start":"176'3100","ondisk_log_start":"176'3100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5267,"log_dups_size":0,"ondisk_log_size":5267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548339712,"num_objects":8367,"num_object_clones":0,"num_object_copies":25101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8367,"num_write_kb":535488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"140'22","reported_seq":209,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850428+0000","last_change":"2024-09-17T08:16:57.000722+0000","last_active":"2024-09-17T08:17:49.850428+0000","last_peered":"2024-09-17T08:17:49.850428+0000","last_clean":"2024-09-17T08:17:49.850428+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:17:49.850428+0000","last_undegraded":"2024-09-17T08:17:49.850428+0000","last_fullsized":"2024-09-17T08:17:49.850428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'22","last_scrub_stamp":"2024-09-17T08:16:57.000450+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:14:50.155143+0000","last_clean_scrub_stamp":"2024-09-17T08:16:57.000450+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.273416+0000","scrub_duration":0.0060780779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858432+0000","last_change":"2024-09-17T08:16:40.246174+0000","last_active":"2024-09-17T08:17:49.858432+0000","last_peered":"2024-09-17T08:17:49.858432+0000","last_clean":"2024-09-17T08:17:49.858432+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:17:49.858432+0000","last_undegraded":"2024-09-17T08:17:49.858432+0000","last_fullsized":"2024-09-17T08:17:49.858432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.246057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.246057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.996076+0000","scrub_duration":0.0035572160000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":273,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.838366+0000","last_change":"2024-09-17T08:16:24.936605+0000","last_active":"2024-09-17T08:17:50.838366+0000","last_peered":"2024-09-17T08:17:50.838366+0000","last_clean":"2024-09-17T08:17:50.838366+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:17:50.838366+0000","last_undegraded":"2024-09-17T08:17:50.838366+0000","last_fullsized":"2024-09-17T08:17:50.838366+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:16:24.936485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:24.936485+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.936007+0000","scrub_duration":0.0033469289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":83,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844372+0000","last_change":"2024-09-17T08:16:58.243174+0000","last_active":"2024-09-17T08:17:49.844372+0000","last_peered":"2024-09-17T08:17:49.844372+0000","last_clean":"2024-09-17T08:17:49.844372+0000","last_became_active":"2024-09-17T08:14:58.717856+0000","last_became_peered":"2024-09-17T08:14:58.717856+0000","last_unstale":"2024-09-17T08:17:49.844372+0000","last_undegraded":"2024-09-17T08:17:49.844372+0000","last_fullsized":"2024-09-17T08:17:49.844372+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.242877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.242877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:02.573195+0000","scrub_duration":0.0056167969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"192'8238","reported_seq":8274,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:57.338593+0000","last_change":"2024-09-17T08:16:16.647722+0000","last_active":"2024-09-17T08:17:57.338593+0000","last_peered":"2024-09-17T08:17:57.338593+0000","last_clean":"2024-09-17T08:17:57.338593+0000","last_became_active":"2024-09-17T08:16:16.647324+0000","last_became_peered":"2024-09-17T08:16:16.647324+0000","last_unstale":"2024-09-17T08:17:57.338593+0000","last_undegraded":"2024-09-17T08:17:57.338593+0000","last_fullsized":"2024-09-17T08:17:57.338593+0000","mapping_epoch":165,"log_start":"176'2900","ondisk_log_start":"176'2900","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539885568,"num_objects":8238,"num_object_clones":0,"num_object_copies":24714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8238,"num_write_kb":527232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":247,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832700+0000","last_change":"2024-09-17T08:17:00.600357+0000","last_active":"2024-09-17T08:17:49.832700+0000","last_peered":"2024-09-17T08:17:49.832700+0000","last_clean":"2024-09-17T08:17:49.832700+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:17:49.832700+0000","last_undegraded":"2024-09-17T08:17:49.832700+0000","last_fullsized":"2024-09-17T08:17:49.832700+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:17:00.600225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.600225+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.375848+0000","scrub_duration":0.0071635179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833337+0000","last_change":"2024-09-17T08:16:39.838818+0000","last_active":"2024-09-17T08:17:49.833337+0000","last_peered":"2024-09-17T08:17:49.833337+0000","last_clean":"2024-09-17T08:17:49.833337+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:17:49.833337+0000","last_undegraded":"2024-09-17T08:17:49.833337+0000","last_fullsized":"2024-09-17T08:17:49.833337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:16:39.838667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.838667+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050598739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":209,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.855904+0000","last_change":"2024-09-17T08:17:43.435620+0000","last_active":"2024-09-17T08:17:49.855904+0000","last_peered":"2024-09-17T08:17:49.855904+0000","last_clean":"2024-09-17T08:17:49.855904+0000","last_became_active":"2024-09-17T08:15:29.374092+0000","last_became_peered":"2024-09-17T08:15:29.374092+0000","last_unstale":"2024-09-17T08:17:49.855904+0000","last_undegraded":"2024-09-17T08:17:49.855904+0000","last_fullsized":"2024-09-17T08:17:49.855904+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:17:43.435490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.510920+0000","scrub_duration":0.0037096490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":185,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856133+0000","last_change":"2024-09-17T08:16:23.763256+0000","last_active":"2024-09-17T08:17:49.856133+0000","last_peered":"2024-09-17T08:17:49.856133+0000","last_clean":"2024-09-17T08:17:49.856133+0000","last_became_active":"2024-09-17T08:15:29.384203+0000","last_became_peered":"2024-09-17T08:15:29.384203+0000","last_unstale":"2024-09-17T08:17:49.856133+0000","last_undegraded":"2024-09-17T08:17:49.856133+0000","last_fullsized":"2024-09-17T08:17:49.856133+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.763173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.763173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001814333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.845038+0000","last_change":"2024-09-17T08:16:36.341678+0000","last_active":"2024-09-17T08:17:49.845038+0000","last_peered":"2024-09-17T08:17:49.845038+0000","last_clean":"2024-09-17T08:17:49.845038+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:17:49.845038+0000","last_undegraded":"2024-09-17T08:17:49.845038+0000","last_fullsized":"2024-09-17T08:17:49.845038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:16:36.341539+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:36.341539+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011421961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":202,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856042+0000","last_change":"2024-09-17T08:16:49.193851+0000","last_active":"2024-09-17T08:17:49.856042+0000","last_peered":"2024-09-17T08:17:49.856042+0000","last_clean":"2024-09-17T08:17:49.856042+0000","last_became_active":"2024-09-17T08:15:29.373114+0000","last_became_peered":"2024-09-17T08:15:29.373114+0000","last_unstale":"2024-09-17T08:17:49.856042+0000","last_undegraded":"2024-09-17T08:17:49.856042+0000","last_fullsized":"2024-09-17T08:17:49.856042+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.193338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.193338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:59.549312+0000","scrub_duration":0.0089744739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":176,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.853625+0000","last_change":"2024-09-17T08:15:33.385747+0000","last_active":"2024-09-17T08:17:49.853625+0000","last_peered":"2024-09-17T08:17:49.853625+0000","last_clean":"2024-09-17T08:17:49.853625+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:17:49.853625+0000","last_undegraded":"2024-09-17T08:17:49.853625+0000","last_fullsized":"2024-09-17T08:17:49.853625+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:33.385285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:33.385285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077031670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858797+0000","last_change":"2024-09-17T08:16:48.222810+0000","last_active":"2024-09-17T08:17:49.858797+0000","last_peered":"2024-09-17T08:17:49.858797+0000","last_clean":"2024-09-17T08:17:49.858797+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:17:49.858797+0000","last_undegraded":"2024-09-17T08:17:49.858797+0000","last_fullsized":"2024-09-17T08:17:49.858797+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.222304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.222304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.070451+0000","scrub_duration":0.003060929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":142,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832774+0000","last_change":"2024-09-17T08:16:29.839526+0000","last_active":"2024-09-17T08:17:49.832774+0000","last_peered":"2024-09-17T08:17:49.832774+0000","last_clean":"2024-09-17T08:17:49.832774+0000","last_became_active":"2024-09-17T08:15:29.358382+0000","last_became_peered":"2024-09-17T08:15:29.358382+0000","last_unstale":"2024-09-17T08:17:49.832774+0000","last_undegraded":"2024-09-17T08:17:49.832774+0000","last_fullsized":"2024-09-17T08:17:49.832774+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:29.839416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.839416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002703015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":214,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.853645+0000","last_change":"2024-09-17T08:16:30.802344+0000","last_active":"2024-09-17T08:17:49.853645+0000","last_peered":"2024-09-17T08:17:49.853645+0000","last_clean":"2024-09-17T08:17:49.853645+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:17:49.853645+0000","last_undegraded":"2024-09-17T08:17:49.853645+0000","last_fullsized":"2024-09-17T08:17:49.853645+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:16:30.800480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.800480+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":90,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.171756+0000","last_change":"2024-09-17T08:17:50.171756+0000","last_active":"2024-09-17T08:17:50.171756+0000","last_peered":"2024-09-17T08:17:50.171756+0000","last_clean":"2024-09-17T08:17:50.171756+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:17:50.171756+0000","last_undegraded":"2024-09-17T08:17:50.171756+0000","last_fullsized":"2024-09-17T08:17:50.171756+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:50.171624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:50.171624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:53.779301+0000","scrub_duration":0.01783852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851931+0000","last_change":"2024-09-17T08:16:51.020489+0000","last_active":"2024-09-17T08:17:49.851931+0000","last_peered":"2024-09-17T08:17:49.851931+0000","last_clean":"2024-09-17T08:17:49.851931+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:17:49.851931+0000","last_undegraded":"2024-09-17T08:17:49.851931+0000","last_fullsized":"2024-09-17T08:17:49.851931+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.020318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.020318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.433650+0000","scrub_duration":0.0019765820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":228,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.855751+0000","last_change":"2024-09-17T08:16:45.629362+0000","last_active":"2024-09-17T08:17:49.855751+0000","last_peered":"2024-09-17T08:17:49.855751+0000","last_clean":"2024-09-17T08:17:49.855751+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:17:49.855751+0000","last_undegraded":"2024-09-17T08:17:49.855751+0000","last_fullsized":"2024-09-17T08:17:49.855751+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:45.626489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:45.626489+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.848535+0000","scrub_duration":0.0060751379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":115,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851750+0000","last_change":"2024-09-17T08:17:47.134565+0000","last_active":"2024-09-17T08:17:49.851750+0000","last_peered":"2024-09-17T08:17:49.851750+0000","last_clean":"2024-09-17T08:17:49.851750+0000","last_became_active":"2024-09-17T08:14:58.720104+0000","last_became_peered":"2024-09-17T08:14:58.720104+0000","last_unstale":"2024-09-17T08:17:49.851750+0000","last_undegraded":"2024-09-17T08:17:49.851750+0000","last_fullsized":"2024-09-17T08:17:49.851750+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:47.134436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.134436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.627771+0000","scrub_duration":0.0068223679999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"152'11","reported_seq":97,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:52.920172+0000","last_change":"2024-09-17T08:15:52.034956+0000","last_active":"2024-09-17T08:17:52.920172+0000","last_peered":"2024-09-17T08:17:52.920172+0000","last_clean":"2024-09-17T08:17:52.920172+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:17:52.920172+0000","last_undegraded":"2024-09-17T08:17:52.920172+0000","last_fullsized":"2024-09-17T08:17:52.920172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'11","last_scrub_stamp":"2024-09-17T08:15:52.034662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.034662+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038045739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844700+0000","last_change":"2024-09-17T08:17:33.406086+0000","last_active":"2024-09-17T08:17:49.844700+0000","last_peered":"2024-09-17T08:17:49.844700+0000","last_clean":"2024-09-17T08:17:49.844700+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:17:49.844700+0000","last_undegraded":"2024-09-17T08:17:49.844700+0000","last_fullsized":"2024-09-17T08:17:49.844700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:33.405939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.405939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.635332+0000","scrub_duration":0.0072126600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"177'13","reported_seq":116,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852210+0000","last_change":"2024-09-17T08:17:00.945597+0000","last_active":"2024-09-17T08:17:49.852210+0000","last_peered":"2024-09-17T08:17:49.852210+0000","last_clean":"2024-09-17T08:17:49.852210+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:17:49.852210+0000","last_undegraded":"2024-09-17T08:17:49.852210+0000","last_fullsized":"2024-09-17T08:17:49.852210+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"177'13","last_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.945463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:07.215485+0000","scrub_duration":0.016379540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844515+0000","last_change":"2024-09-17T08:16:46.264262+0000","last_active":"2024-09-17T08:17:49.844515+0000","last_peered":"2024-09-17T08:17:49.844515+0000","last_clean":"2024-09-17T08:17:49.844515+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:17:49.844515+0000","last_undegraded":"2024-09-17T08:17:49.844515+0000","last_fullsized":"2024-09-17T08:17:49.844515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:46.264138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.264138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:04.163330+0000","scrub_duration":0.0046054340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":274,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.839256+0000","last_change":"2024-09-17T08:17:35.880572+0000","last_active":"2024-09-17T08:17:50.839256+0000","last_peered":"2024-09-17T08:17:50.839256+0000","last_clean":"2024-09-17T08:17:50.839256+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:17:50.839256+0000","last_undegraded":"2024-09-17T08:17:50.839256+0000","last_fullsized":"2024-09-17T08:17:50.839256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:17:35.880439+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.880439+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.829845+0000","scrub_duration":0.0069173250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833060+0000","last_change":"2024-09-17T08:16:44.899582+0000","last_active":"2024-09-17T08:17:49.833060+0000","last_peered":"2024-09-17T08:17:49.833060+0000","last_clean":"2024-09-17T08:17:49.833060+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:17:49.833060+0000","last_undegraded":"2024-09-17T08:17:49.833060+0000","last_fullsized":"2024-09-17T08:17:49.833060+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.899473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.899473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:58.638190+0000","scrub_duration":0.0048647389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":73,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.844221+0000","last_change":"2024-09-17T08:17:31.426355+0000","last_active":"2024-09-17T08:17:50.844221+0000","last_peered":"2024-09-17T08:17:50.844221+0000","last_clean":"2024-09-17T08:17:50.844221+0000","last_became_active":"2024-09-17T08:15:29.364047+0000","last_became_peered":"2024-09-17T08:15:29.364047+0000","last_unstale":"2024-09-17T08:17:50.844221+0000","last_undegraded":"2024-09-17T08:17:50.844221+0000","last_fullsized":"2024-09-17T08:17:50.844221+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:31.426185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.426185+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.778192+0000","scrub_duration":0.028345339000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.855827+0000","last_change":"2024-09-17T08:15:54.838921+0000","last_active":"2024-09-17T08:17:49.855827+0000","last_peered":"2024-09-17T08:17:49.855827+0000","last_clean":"2024-09-17T08:17:49.855827+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:17:49.855827+0000","last_undegraded":"2024-09-17T08:17:49.855827+0000","last_fullsized":"2024-09-17T08:17:49.855827+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:54.838847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:54.838847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":187,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856195+0000","last_change":"2024-09-17T08:17:32.933012+0000","last_active":"2024-09-17T08:17:49.856195+0000","last_peered":"2024-09-17T08:17:49.856195+0000","last_clean":"2024-09-17T08:17:49.856195+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:17:49.856195+0000","last_undegraded":"2024-09-17T08:17:49.856195+0000","last_fullsized":"2024-09-17T08:17:49.856195+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.932889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.932889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.988363+0000","scrub_duration":0.017331688000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":196,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.846932+0000","last_change":"2024-09-17T08:17:37.431486+0000","last_active":"2024-09-17T08:17:49.846932+0000","last_peered":"2024-09-17T08:17:49.846932+0000","last_clean":"2024-09-17T08:17:49.846932+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:17:49.846932+0000","last_undegraded":"2024-09-17T08:17:49.846932+0000","last_fullsized":"2024-09-17T08:17:49.846932+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.431366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.431366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.282544+0000","scrub_duration":0.0073929879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":243,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832932+0000","last_change":"2024-09-17T08:16:59.607476+0000","last_active":"2024-09-17T08:17:49.832932+0000","last_peered":"2024-09-17T08:17:49.832932+0000","last_clean":"2024-09-17T08:17:49.832932+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:17:49.832932+0000","last_undegraded":"2024-09-17T08:17:49.832932+0000","last_fullsized":"2024-09-17T08:17:49.832932+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:16:59.607325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.607325+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.411643+0000","scrub_duration":0.0083300879999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":63,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.850255+0000","last_change":"2024-09-17T08:17:01.469617+0000","last_active":"2024-09-17T08:17:50.850255+0000","last_peered":"2024-09-17T08:17:50.850255+0000","last_clean":"2024-09-17T08:17:50.850255+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:17:50.850255+0000","last_undegraded":"2024-09-17T08:17:50.850255+0000","last_fullsized":"2024-09-17T08:17:50.850255+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.469473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.469473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:22.713180+0000","scrub_duration":0.0032550880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":63,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.844697+0000","last_change":"2024-09-17T08:17:32.455271+0000","last_active":"2024-09-17T08:17:50.844697+0000","last_peered":"2024-09-17T08:17:50.844697+0000","last_clean":"2024-09-17T08:17:50.844697+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:17:50.844697+0000","last_undegraded":"2024-09-17T08:17:50.844697+0000","last_fullsized":"2024-09-17T08:17:50.844697+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.455128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.455128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.037261+0000","scrub_duration":0.0091779470000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851043+0000","last_change":"2024-09-17T08:16:39.766499+0000","last_active":"2024-09-17T08:17:49.851043+0000","last_peered":"2024-09-17T08:17:49.851043+0000","last_clean":"2024-09-17T08:17:49.851043+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:17:49.851043+0000","last_undegraded":"2024-09-17T08:17:49.851043+0000","last_fullsized":"2024-09-17T08:17:49.851043+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:39.766371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.766371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:51.034351+0000","scrub_duration":0.003380568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":222,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.853216+0000","last_change":"2024-09-17T08:17:42.964511+0000","last_active":"2024-09-17T08:17:49.853216+0000","last_peered":"2024-09-17T08:17:49.853216+0000","last_clean":"2024-09-17T08:17:49.853216+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:17:49.853216+0000","last_undegraded":"2024-09-17T08:17:49.853216+0000","last_fullsized":"2024-09-17T08:17:49.853216+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:17:42.964394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:42.964394+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.742246+0000","scrub_duration":0.0020163619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":84,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858364+0000","last_change":"2024-09-17T08:17:37.395358+0000","last_active":"2024-09-17T08:17:49.858364+0000","last_peered":"2024-09-17T08:17:49.858364+0000","last_clean":"2024-09-17T08:17:49.858364+0000","last_became_active":"2024-09-17T08:14:58.715288+0000","last_became_peered":"2024-09-17T08:14:58.715288+0000","last_unstale":"2024-09-17T08:17:49.858364+0000","last_undegraded":"2024-09-17T08:17:49.858364+0000","last_fullsized":"2024-09-17T08:17:49.858364+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.394936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.394936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.208429+0000","scrub_duration":0.0060254940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":136,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832621+0000","last_change":"2024-09-17T08:17:31.626868+0000","last_active":"2024-09-17T08:17:49.832621+0000","last_peered":"2024-09-17T08:17:49.832621+0000","last_clean":"2024-09-17T08:17:49.832621+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:17:49.832621+0000","last_undegraded":"2024-09-17T08:17:49.832621+0000","last_fullsized":"2024-09-17T08:17:49.832621+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.626749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:52.132279+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.626749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.963940+0000","scrub_duration":0.002332518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":178,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850417+0000","last_change":"2024-09-17T08:16:52.066187+0000","last_active":"2024-09-17T08:17:49.850417+0000","last_peered":"2024-09-17T08:17:49.850417+0000","last_clean":"2024-09-17T08:17:49.850417+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:17:49.850417+0000","last_undegraded":"2024-09-17T08:17:49.850417+0000","last_fullsized":"2024-09-17T08:17:49.850417+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.066058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.066058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.573732+0000","scrub_duration":0.0023799279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":190,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851213+0000","last_change":"2024-09-17T08:17:33.932414+0000","last_active":"2024-09-17T08:17:49.851213+0000","last_peered":"2024-09-17T08:17:49.851213+0000","last_clean":"2024-09-17T08:17:49.851213+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:17:49.851213+0000","last_undegraded":"2024-09-17T08:17:49.851213+0000","last_fullsized":"2024-09-17T08:17:49.851213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:33.932279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.932279+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:03.370870+0000","scrub_duration":0.0031072140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":110,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.857338+0000","last_change":"2024-09-17T08:16:27.299767+0000","last_active":"2024-09-17T08:17:49.857338+0000","last_peered":"2024-09-17T08:17:49.857338+0000","last_clean":"2024-09-17T08:17:49.857338+0000","last_became_active":"2024-09-17T08:14:58.716813+0000","last_became_peered":"2024-09-17T08:14:58.716813+0000","last_unstale":"2024-09-17T08:17:49.857338+0000","last_undegraded":"2024-09-17T08:17:49.857338+0000","last_fullsized":"2024-09-17T08:17:49.857338+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:16:27.299675+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.299675+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:54.173560+0000","scrub_duration":0.0018966499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850195+0000","last_change":"2024-09-17T08:17:39.165283+0000","last_active":"2024-09-17T08:17:49.850195+0000","last_peered":"2024-09-17T08:17:49.850195+0000","last_clean":"2024-09-17T08:17:49.850195+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:17:49.850195+0000","last_undegraded":"2024-09-17T08:17:49.850195+0000","last_fullsized":"2024-09-17T08:17:49.850195+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.164442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.164442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:07.439327+0000","scrub_duration":0.0024479319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":156,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.856064+0000","last_change":"2024-09-17T08:15:51.743705+0000","last_active":"2024-09-17T08:17:49.856064+0000","last_peered":"2024-09-17T08:17:49.856064+0000","last_clean":"2024-09-17T08:17:49.856064+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:17:49.856064+0000","last_undegraded":"2024-09-17T08:17:49.856064+0000","last_fullsized":"2024-09-17T08:17:49.856064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:51.721883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:51.721883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082713839999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":169,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.850845+0000","last_change":"2024-09-17T08:16:40.783169+0000","last_active":"2024-09-17T08:17:49.850845+0000","last_peered":"2024-09-17T08:17:49.850845+0000","last_clean":"2024-09-17T08:17:49.850845+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:17:49.850845+0000","last_undegraded":"2024-09-17T08:17:49.850845+0000","last_fullsized":"2024-09-17T08:17:49.850845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:16:40.783041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.783041+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030603710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":84,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.832422+0000","last_change":"2024-09-17T08:17:46.783050+0000","last_active":"2024-09-17T08:17:49.832422+0000","last_peered":"2024-09-17T08:17:49.832422+0000","last_clean":"2024-09-17T08:17:49.832422+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:17:49.832422+0000","last_undegraded":"2024-09-17T08:17:49.832422+0000","last_fullsized":"2024-09-17T08:17:49.832422+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:46.782915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:46.782915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:50.927390+0000","scrub_duration":0.0019436189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":198,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.857189+0000","last_change":"2024-09-17T08:16:47.247472+0000","last_active":"2024-09-17T08:17:50.857189+0000","last_peered":"2024-09-17T08:17:50.857189+0000","last_clean":"2024-09-17T08:17:50.857189+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:17:50.857189+0000","last_undegraded":"2024-09-17T08:17:50.857189+0000","last_fullsized":"2024-09-17T08:17:50.857189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:47.247339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.247339+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:15.653964+0000","scrub_duration":0.0072034789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":209,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-17T08:17:50.836399+0000","last_change":"2024-09-17T08:16:52.823084+0000","last_active":"2024-09-17T08:17:50.836399+0000","last_peered":"2024-09-17T08:17:50.836399+0000","last_clean":"2024-09-17T08:17:50.836399+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:17:50.836399+0000","last_undegraded":"2024-09-17T08:17:50.836399+0000","last_fullsized":"2024-09-17T08:17:50.836399+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:16:52.822979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.822979+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:18.218032+0000","scrub_duration":0.0016070290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":73,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851757+0000","last_change":"2024-09-17T08:17:48.711279+0000","last_active":"2024-09-17T08:17:49.851757+0000","last_peered":"2024-09-17T08:17:49.851757+0000","last_clean":"2024-09-17T08:17:49.851757+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:17:49.851757+0000","last_undegraded":"2024-09-17T08:17:49.851757+0000","last_fullsized":"2024-09-17T08:17:49.851757+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.711144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.711144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.543032+0000","scrub_duration":0.0072007140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":173,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851495+0000","last_change":"2024-09-17T08:16:58.658509+0000","last_active":"2024-09-17T08:17:49.851495+0000","last_peered":"2024-09-17T08:17:49.851495+0000","last_clean":"2024-09-17T08:17:49.851495+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:17:49.851495+0000","last_undegraded":"2024-09-17T08:17:49.851495+0000","last_fullsized":"2024-09-17T08:17:49.851495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.658382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.658382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:14.307934+0000","scrub_duration":0.0056271569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":75,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.871924+0000","last_change":"2024-09-17T08:17:38.572088+0000","last_active":"2024-09-17T08:17:49.871924+0000","last_peered":"2024-09-17T08:17:49.871924+0000","last_clean":"2024-09-17T08:17:49.871924+0000","last_became_active":"2024-09-17T08:15:29.361714+0000","last_became_peered":"2024-09-17T08:15:29.361714+0000","last_unstale":"2024-09-17T08:17:49.871924+0000","last_undegraded":"2024-09-17T08:17:49.871924+0000","last_fullsized":"2024-09-17T08:17:49.871924+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:38.571885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:38.571885+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:56.028615+0000","scrub_duration":0.0076271409999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.844900+0000","last_change":"2024-09-17T08:16:49.210341+0000","last_active":"2024-09-17T08:17:49.844900+0000","last_peered":"2024-09-17T08:17:49.844900+0000","last_clean":"2024-09-17T08:17:49.844900+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:17:49.844900+0000","last_undegraded":"2024-09-17T08:17:49.844900+0000","last_fullsized":"2024-09-17T08:17:49.844900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:49.210217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:16:49.210217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:53.183066+0000","scrub_duration":0.0066617630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":167,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833188+0000","last_change":"2024-09-17T08:16:37.835578+0000","last_active":"2024-09-17T08:17:49.833188+0000","last_peered":"2024-09-17T08:17:49.833188+0000","last_clean":"2024-09-17T08:17:49.833188+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:17:49.833188+0000","last_undegraded":"2024-09-17T08:17:49.833188+0000","last_fullsized":"2024-09-17T08:17:49.833188+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.835368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.835368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001605574,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":135,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833523+0000","last_change":"2024-09-17T08:17:36.583106+0000","last_active":"2024-09-17T08:17:49.833523+0000","last_peered":"2024-09-17T08:17:49.833523+0000","last_clean":"2024-09-17T08:17:49.833523+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:17:49.833523+0000","last_undegraded":"2024-09-17T08:17:49.833523+0000","last_fullsized":"2024-09-17T08:17:49.833523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.582852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.582852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.756203+0000","scrub_duration":0.001816591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":289,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.866882+0000","last_change":"2024-09-17T08:16:59.092346+0000","last_active":"2024-09-17T08:17:49.866882+0000","last_peered":"2024-09-17T08:17:49.866882+0000","last_clean":"2024-09-17T08:17:49.866882+0000","last_became_active":"2024-09-17T08:15:29.388191+0000","last_became_peered":"2024-09-17T08:15:29.388191+0000","last_unstale":"2024-09-17T08:17:49.866882+0000","last_undegraded":"2024-09-17T08:17:49.866882+0000","last_fullsized":"2024-09-17T08:17:49.866882+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:16:59.092016+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.092016+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:12.568034+0000","scrub_duration":0.0088076999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":62,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.851889+0000","last_change":"2024-09-17T08:16:51.343826+0000","last_active":"2024-09-17T08:17:49.851889+0000","last_peered":"2024-09-17T08:17:49.851889+0000","last_clean":"2024-09-17T08:17:49.851889+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:17:49.851889+0000","last_undegraded":"2024-09-17T08:17:49.851889+0000","last_fullsized":"2024-09-17T08:17:49.851889+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.343712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.343712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:05.929418+0000","scrub_duration":0.010532788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":156,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.853521+0000","last_change":"2024-09-17T08:15:34.433273+0000","last_active":"2024-09-17T08:17:49.853521+0000","last_peered":"2024-09-17T08:17:49.853521+0000","last_clean":"2024-09-17T08:17:49.853521+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:17:49.853521+0000","last_undegraded":"2024-09-17T08:17:49.853521+0000","last_fullsized":"2024-09-17T08:17:49.853521+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:34.432898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:15:34.432898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012758472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":173,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.858898+0000","last_change":"2024-09-17T08:15:48.646475+0000","last_active":"2024-09-17T08:17:49.858898+0000","last_peered":"2024-09-17T08:17:49.858898+0000","last_clean":"2024-09-17T08:17:49.858898+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:17:49.858898+0000","last_undegraded":"2024-09-17T08:17:49.858898+0000","last_fullsized":"2024-09-17T08:17:49.858898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:15:48.646329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:15:48.646329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076865340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":236,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.852054+0000","last_change":"2024-09-17T08:16:29.751857+0000","last_active":"2024-09-17T08:17:49.852054+0000","last_peered":"2024-09-17T08:17:49.852054+0000","last_clean":"2024-09-17T08:17:49.852054+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:17:49.852054+0000","last_undegraded":"2024-09-17T08:17:49.852054+0000","last_fullsized":"2024-09-17T08:17:49.852054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:16:29.751693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.751693+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054470619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":180,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-17T08:17:49.833102+0000","last_change":"2024-09-17T08:17:43.726427+0000","last_active":"2024-09-17T08:17:49.833102+0000","last_peered":"2024-09-17T08:17:49.833102+0000","last_clean":"2024-09-17T08:17:49.833102+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:17:49.833102+0000","last_undegraded":"2024-09-17T08:17:49.833102+0000","last_fullsized":"2024-09-17T08:17:49.833102+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:43.726340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:43.726340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:11.278692+0000","scrub_duration":0.0012672689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":8711045120,"num_objects":132920,"num_object_clones":0,"num_object_copies":398760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132920,"num_write_kb":8506880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26094600192,"data_stored":26094600192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83920,"ondisk_log_size":83920,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":690,"num_read_kb":447,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":451,"ondisk_log_size":451,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":149,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070566,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2865592,"kb_used_data":2635952,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":90879560,"statfs":{"total":95995035648,"available":93060669440,"internally_reserved":0,"allocated":2699214848,"data_stored":2698907867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":235141579},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4279999999999999,"15min":0.85999999999999999},"min":{"1min":0.23200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.415,"5min":27.751999999999999,"15min":27.751999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.583,"15min":0.92800000000000005},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":5.1589999999999998,"5min":37.954000000000001,"15min":37.954000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6990000000000001,"5min":1.012,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.255,"15min":0.23699999999999999},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.401,"5min":0.91100000000000003,"15min":0.68400000000000005},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":5.1340000000000003,"5min":5.1340000000000003,"15min":5.1340000000000003},"last":0.88400000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.117,"15min":0.74399999999999999},"min":{"1min":0.33800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.0359999999999996,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.089,"15min":0.76900000000000002},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":5.4550000000000001,"5min":5.4550000000000001,"15min":5.4550000000000001},"last":0.505}]},{"osd":3,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":1.3029999999999999,"15min":0.80700000000000005},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.3420000000000001,"5min":11.298,"15min":11.298},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.4430000000000001,"15min":0.90700000000000003},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.391,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.57599999999999996,"15min":0.49399999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.2249999999999996,"5min":5.2249999999999996,"15min":5.2249999999999996},"last":2.278},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.52700000000000002,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.232}]},{"osd":5,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.78800000000000003,"15min":0.59099999999999997},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":5.367,"5min":16.808,"15min":16.808},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.056,"5min":0.71699999999999997,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.7400000000000002,"5min":11.842000000000001,"15min":11.842000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.67200000000000004,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.0949999999999998,"5min":5.0949999999999998,"15min":5.0949999999999998},"last":0.995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.6659999999999999,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":1.3200000000000001}]}]},{"osd":6,"up_from":50,"seq":214748364907,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2842472,"kb_used_data":2618592,"kb_used_omap":1,"kb_used_meta":223870,"kb_avail":90902680,"statfs":{"total":95995035648,"available":93084344320,"internally_reserved":0,"allocated":2681438208,"data_stored":2681146871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":229243335},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":3.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.248,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":36.784999999999997,"5min":73.757999999999996,"15min":73.757999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":2.7719999999999998,"5min":3.6709999999999998,"15min":1.5489999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":37.073,"5min":73.811000000000007,"15min":73.811000000000007},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.6829999999999998,"5min":1.5509999999999999,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":37.256999999999998,"5min":37.256999999999998,"15min":37.256999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":2.9670000000000001,"5min":1.708,"15min":0.92300000000000004},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":37.415999999999997,"5min":37.415999999999997,"15min":37.415999999999997},"last":0.66900000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":1.5569999999999999,"15min":0.871},"min":{"1min":0.25700000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.6539999999999999,"5min":68.165000000000006,"15min":68.165000000000006},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.821,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4359999999999999,"5min":67.997,"15min":67.997},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.633,"5min":2.6829999999999998,"15min":1.2649999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":39.198999999999998,"5min":77.010999999999996,"15min":77.010999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":2.605,"5min":2.6259999999999999,"15min":1.236},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":36.804000000000002,"5min":76.375,"15min":76.375},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.629,"15min":0.45900000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.98699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.63300000000000001,"15min":0.47399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51000000000000001,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":2.5880000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.66100000000000003,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":2.7959999999999998}]}]},{"osd":5,"up_from":41,"seq":176093659248,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4441896,"kb_used_data":4212256,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":89303256,"statfs":{"total":95995035648,"available":91446534144,"internally_reserved":0,"allocated":4313350144,"data_stored":4313058212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":235141574},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":2.1000000000000001,"15min":1.0389999999999999},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":5.3079999999999998,"5min":63.098999999999997,"15min":63.098999999999997},"last":2.1640000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.948,"15min":0.97199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.0899999999999999,"5min":62.567,"15min":62.567},"last":0.24199999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.91400000000000003,"15min":0.64000000000000001},"min":{"1min":0.33300000000000002,"5min":0.222,"15min":0.222},"max":{"1min":2.532,"5min":5.056,"15min":5.056},"last":2.5819999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.107,"15min":0.70299999999999996},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":2.4580000000000002,"5min":8.8119999999999994,"15min":8.8119999999999994},"last":2.3610000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.014,"15min":0.68899999999999995},"min":{"1min":0.247,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.589,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.89300000000000002,"15min":0.63300000000000001},"min":{"1min":0.315,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.141,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":2.5379999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":2.149,"15min":1.05},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.6259999999999999,"5min":75.286000000000001,"15min":75.286000000000001},"last":5.1100000000000003},{"interface":"front","average":{"1min":0.999,"5min":2.089,"15min":1.05},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.29499999999999998},"max":{"1min":5.8300000000000001,"5min":77.206999999999994,"15min":77.206999999999994},"last":2.5150000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.5,"15min":0.40699999999999997},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.19},"max":{"1min":1.1899999999999999,"5min":3.544,"15min":3.544},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.752}]},{"osd":6,"last update":"Tue Sep 17 08:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.2709999999999999,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.623,"15min":0.48699999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.254},"max":{"1min":5.1779999999999999,"5min":5.1779999999999999,"15min":5.1779999999999999},"last":2.5609999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.60199999999999998,"15min":0.51600000000000001},"min":{"1min":0.38100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3330000000000002,"5min":3.645,"15min":3.645},"last":2.4830000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.49199999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":1.0309999999999999,"5min":3.613,"15min":3.613},"last":0.55100000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953589,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2807220,"kb_used_data":2637484,"kb_used_omap":1,"kb_used_meta":169726,"kb_avail":90937932,"statfs":{"total":95995035648,"available":93120442368,"internally_reserved":0,"allocated":2700783616,"data_stored":2700480025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":173799878},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.371,"15min":0.80800000000000005},"min":{"1min":0.245,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.3819999999999997,"5min":23.786000000000001,"15min":23.786000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.405,"15min":0.81699999999999995},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":4.0030000000000001,"5min":23.245000000000001,"15min":23.245000000000001},"last":2.0550000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":0.74099999999999999},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":4.4790000000000001,"5min":23.135000000000002,"15min":23.135000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.8140000000000001,"5min":1.347,"15min":0.80100000000000005},"min":{"1min":0.30599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":7.6180000000000003,"5min":23.434999999999999,"15min":23.434999999999999},"last":1.556}]},{"osd":2,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":0.95399999999999996,"15min":0.66900000000000004},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.4119999999999999,"5min":4.4119999999999999,"15min":4.4119999999999999},"last":1.589},{"interface":"front","average":{"1min":1.492,"5min":1.0129999999999999,"15min":0.67300000000000004},"min":{"1min":0.38,"5min":0.27100000000000002,"15min":0.23799999999999999},"max":{"1min":7.0140000000000002,"5min":7.0140000000000002,"15min":7.0140000000000002},"last":2.161}]},{"osd":3,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":3.4849999999999999,"15min":1.52},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.6469999999999998,"5min":154.137,"15min":154.137},"last":2.0830000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":3.2679999999999998,"15min":1.4490000000000001},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.27400000000000002},"max":{"1min":3.351,"5min":153.54400000000001,"15min":153.54400000000001},"last":0.442}]},{"osd":5,"last update":"Tue Sep 17 08:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.4970000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.504,"15min":0.432},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":2.1030000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.9079999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.623,"15min":0.505},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.8660000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":0.68700000000000006,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.1260000000000003,"5min":4.1260000000000003,"15min":4.1260000000000003},"last":2.125},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.60899999999999999,"15min":0.51200000000000001},"min":{"1min":0.38700000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":1.5369999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117113,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3461568,"kb_used_data":3169592,"kb_used_omap":1,"kb_used_meta":291966,"kb_avail":90283584,"statfs":{"total":95995035648,"available":92450390016,"internally_reserved":0,"allocated":3245662208,"data_stored":3245345806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":298973640},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.851,"5min":1.9790000000000001,"15min":0.90300000000000002},"min":{"1min":0.23799999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":5.1879999999999997,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.8140000000000001,"15min":0.85099999999999998},"min":{"1min":0.28999999999999998,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":3.3940000000000001,"5min":10.362,"15min":10.362},"last":2.2730000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.825,"15min":0.879},"min":{"1min":0.25800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":4.7990000000000004,"5min":8.4410000000000007,"15min":8.4410000000000007},"last":3.577},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.867,"15min":0.89400000000000002},"min":{"1min":0.374,"5min":0.188,"15min":0.188},"max":{"1min":3.0539999999999998,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.9319999999999999,"15min":0.91000000000000003},"min":{"1min":0.26400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":2.976,"5min":10.786,"15min":10.786},"last":2.911},{"interface":"front","average":{"1min":1.8140000000000001,"5min":2.2669999999999999,"15min":1.0189999999999999},"min":{"1min":0.30299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.46,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":6.2530000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.4610000000000001,"15min":0.84999999999999998},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":3.8159999999999998,"5min":7.742,"15min":7.742},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.097,"5min":1.2969999999999999,"15min":0.81799999999999995},"min":{"1min":0.308,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.4340000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":1.468}]},{"osd":5,"last update":"Tue Sep 17 08:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.4490000000000001,"15min":0.86199999999999999},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.4870000000000001,"5min":8.2240000000000002,"15min":8.2240000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.724,"5min":1.482,"15min":0.89000000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.4359999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.393,"15min":0.85299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4870000000000001,"5min":10.945,"15min":10.945},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.796,"5min":1.6339999999999999,"15min":0.92900000000000005},"min":{"1min":0.41699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":5.1360000000000001,"5min":5.798,"15min":5.798},"last":1.3660000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4470000000000001,"15min":0.90600000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":4.7069999999999999,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.405,"15min":0.88500000000000001},"min":{"1min":0.34100000000000003,"5min":0.316,"15min":0.28999999999999998},"max":{"1min":4.3440000000000003,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":0.57099999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411452,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3421356,"kb_used_data":3187620,"kb_used_omap":1,"kb_used_meta":233726,"kb_avail":90323796,"statfs":{"total":95995035648,"available":92491567104,"internally_reserved":0,"allocated":3264122880,"data_stored":3263827250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":239335878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":2.0459999999999998,"15min":0.90200000000000002},"min":{"1min":0.20999999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":7.1600000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.163,"5min":2.226,"15min":0.97899999999999998},"min":{"1min":0.19,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":7.0419999999999998,"5min":23.341000000000001,"15min":23.341000000000001},"last":0.25600000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.575,"15min":0.77000000000000002},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":3.2829999999999999,"5min":6.1360000000000001,"15min":6.1360000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.4059999999999999,"15min":0.72799999999999998},"min":{"1min":0.22500000000000001,"5min":0.184,"15min":0.184},"max":{"1min":4.7930000000000001,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.42199999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":2.899,"5min":2.2370000000000001,"15min":1.0129999999999999},"min":{"1min":0.246,"5min":0.23499999999999999,"15min":0.20300000000000001},"max":{"1min":36.792999999999999,"5min":36.792999999999999,"15min":36.792999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":6.7859999999999996,"5min":2.9940000000000002,"15min":1.288},"min":{"1min":0.251,"5min":0.219,"15min":0.219},"max":{"1min":56.533000000000001,"5min":56.533000000000001,"15min":56.533000000000001},"last":0.30099999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.3899999999999999,"15min":0.83699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.5939999999999999,"5min":13.58,"15min":13.58},"last":0.505},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.427,"15min":0.85199999999999998},"min":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.3820000000000001,"5min":13.818,"15min":13.818},"last":0.47499999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":1.1639999999999999,"15min":0.752},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.8280000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"last":0.435},{"interface":"front","average":{"1min":0.72899999999999998,"5min":1.2090000000000001,"15min":0.76100000000000001},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9910000000000001,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Sep 17 08:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.349,"15min":0.83599999999999997},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.9379999999999999,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3169999999999999,"15min":0.84699999999999998},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.4399999999999999,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":2.4049999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.206,"15min":0.77700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6989999999999998,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":1.212,"15min":0.77400000000000002},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7210000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":0.92500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3957448,"kb_used_data":3683520,"kb_used_omap":1,"kb_used_meta":273918,"kb_avail":89787704,"statfs":{"total":95995035648,"available":91942608896,"internally_reserved":0,"allocated":3771924480,"data_stored":3771600476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280492486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":2.1219999999999999,"15min":0.995},"min":{"1min":0.193,"5min":0.16,"15min":0.16},"max":{"1min":4.9470000000000001,"5min":31.585999999999999,"15min":31.585999999999999},"last":3.1200000000000001},{"interface":"front","average":{"1min":2.1749999999999998,"5min":2.4260000000000002,"15min":1.1000000000000001},"min":{"1min":0.32900000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":6.2119999999999997,"5min":35.807000000000002,"15min":35.807000000000002},"last":1.7290000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:17:02 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.5169999999999999,"15min":0.73699999999999999},"min":{"1min":0.23999999999999999,"5min":0.20100000000000001,"15min":0.19700000000000001},"max":{"1min":4.7009999999999996,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":2.0539999999999998},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.675,"15min":0.81899999999999995},"min":{"1min":0.28899999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.1740000000000004,"5min":10.638,"15min":10.638},"last":1.4319999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8460000000000001,"5min":1.6890000000000001,"15min":0.82699999999999996},"min":{"1min":0.26100000000000001,"5min":0.23300000000000001,"15min":0.20300000000000001},"max":{"1min":5.2759999999999998,"5min":7.8499999999999996,"15min":7.8499999999999996},"last":2.286},{"interface":"front","average":{"1min":1.696,"5min":1.6399999999999999,"15min":0.81499999999999995},"min":{"1min":0.35499999999999998,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":6.2759999999999998,"5min":7.5519999999999996,"15min":7.5519999999999996},"last":4.6440000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:17:54 2024","interfaces":[{"interface":"back","average":{"1min":2.218,"5min":1.518,"15min":0.86699999999999999},"min":{"1min":0.42299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.8959999999999999,"5min":7.4859999999999998,"15min":7.4859999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.1910000000000001,"15min":0.78900000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.867,"5min":10.208,"15min":10.208},"last":3.23}]},{"osd":5,"last update":"Tue Sep 17 08:17:07 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.169,"15min":0.746},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.0090000000000003,"5min":5.9249999999999998,"15min":5.9249999999999998},"last":1.994},{"interface":"front","average":{"1min":1.3,"5min":1.3300000000000001,"15min":0.80900000000000005},"min":{"1min":0.45600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.383,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":1.5409999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:17:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.242,"15min":0.76700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.919,"5min":10.253,"15min":10.253},"last":3.0169999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.3109999999999999,"15min":0.82699999999999996},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2590000000000003,"5min":7.2590000000000003,"15min":7.2590000000000003},"last":2.0299999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.3069999999999999,"15min":0.81599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.6859999999999999,"5min":10.066000000000001,"15min":10.066000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.2569999999999999,"15min":0.81499999999999995},"min":{"1min":0.43099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":3.0339999999999998,"5min":10.279,"15min":10.279},"last":3.1680000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3437300,"kb_used_data":3175660,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":90307852,"statfs":{"total":95995035648,"available":92475240448,"internally_reserved":0,"allocated":3251875840,"data_stored":3251572639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":5.6630000000000003,"5min":2.3660000000000001,"15min":1.024},"min":{"1min":0.30499999999999999,"5min":0.21299999999999999,"15min":0.19900000000000001},"max":{"1min":61.649000000000001,"5min":61.649000000000001,"15min":61.649000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":5.452,"5min":2.6859999999999999,"15min":1.1399999999999999},"min":{"1min":0.183,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":61.281999999999996,"5min":61.281999999999996,"15min":61.281999999999996},"last":0.28299999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.6799999999999999,"15min":0.81000000000000005},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.8100000000000001,"5min":12.355,"15min":12.355},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.6759999999999999,"15min":0.82599999999999996},"min":{"1min":0.216,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":8.0489999999999995,"5min":8.0489999999999995,"15min":8.0489999999999995},"last":0.85899999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:17:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6879999999999997,"5min":2.903,"15min":1.254},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.20999999999999999},"max":{"1min":61.926000000000002,"5min":61.926000000000002,"15min":61.926000000000002},"last":0.754},{"interface":"front","average":{"1min":5.9459999999999997,"5min":2.6840000000000002,"15min":1.194},"min":{"1min":0.32500000000000001,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":61.616,"5min":61.616,"15min":61.616},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Sep 17 08:17:44 2024","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":1.728,"15min":0.95499999999999996},"min":{"1min":0.38200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":37.963999999999999,"5min":37.963999999999999,"15min":37.963999999999999},"last":2.6070000000000002},{"interface":"front","average":{"1min":2.944,"5min":1.629,"15min":0.96699999999999997},"min":{"1min":0.29699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":37.731999999999999,"5min":37.731999999999999,"15min":37.731999999999999},"last":2.653}]},{"osd":5,"last update":"Tue Sep 17 08:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1619999999999999,"15min":0.76200000000000001},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":5.5499999999999998,"5min":7.835,"15min":7.835},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":1.3720000000000001,"15min":0.83899999999999997},"min":{"1min":0.435,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":6.8789999999999996,"15min":6.8789999999999996},"last":1.399}]},{"osd":6,"last update":"Tue Sep 17 08:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.228,"15min":0.79900000000000004},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.4800000000000004,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.4159999999999999,"15min":0.86699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.4529999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.99199999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.9990000000000001,"5min":1.9890000000000001,"15min":1.109},"min":{"1min":0.42899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":37.704999999999998,"5min":37.704999999999998,"15min":37.704999999999998},"last":1.673},{"interface":"front","average":{"1min":3.0880000000000001,"5min":1.9359999999999999,"15min":1.0640000000000001},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":37.878,"5min":37.878,"15min":37.878},"last":1.6379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3270049792,"data_stored":3270049792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3808100352,"data_stored":3808100352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3278569472,"data_stored":3278569472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3264282624,"data_stored":3264282624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2715353088,"data_stored":2715353088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4352507904,"data_stored":4352507904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2702573568,"data_stored":2702573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2703163392,"data_stored":2703163392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:18:06.751 INFO:tasks.thrashosds.thrasher:Setting 6.b to [3, 2, 1, 6, 5, 4] 2024-09-17T08:18:06.751 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '6.b', '3', '2', '1', '6', '5', '4'] 2024-09-17T08:18:06.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 6.b 3 2 1 6 5 4 2024-09-17T08:18:06.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:06 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:06.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:06 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:06.788 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:06 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:06.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:06 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:06.804 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:06 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:06.804 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:06 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:06.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:06.877 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:06.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:06.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:06.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:07.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.019 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:07.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:07.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:07.142 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:07.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:07.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:07.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:07.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:07.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:07.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:07.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:07.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:18:07.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:07.530 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:07 smithi173 ceph-mon[33187]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T08:18:07.530 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:07 smithi173 ceph-mon[33187]: pgmap v352: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-09-17T08:18:07.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:07.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:07.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:07.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:07.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:18:07.706 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:18:07.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:18:07.706 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:18:07.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30826 7.5 7:a9498751:::benchmark_data_smithi079_83551_object30825:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.028920+0000", 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 337.67451573900001, 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14601156100000001, 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30826 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028920+0000", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028920+0000", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028919+0000", 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028984+0000", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5059000000000002e-05 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028992+0000", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7819999999999994e-06 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.028999+0000", 2024-09-17T08:18:07.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2919999999999998e-06 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029032+0000", 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2808000000000002e-05 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029117+0000", 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4691000000000006e-05 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029208+0000", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1435999999999997e-05 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035262+0000", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0060537109999999998 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035313+0000", 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0868999999999997e-05 2024-09-17T08:18:07.711 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174818+0000", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.139505304 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174892+0000", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3909e-05 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174932+0000", 2024-09-17T08:18:07.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9854999999999997e-05 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30834 7.5 7:a7572fd4:::benchmark_data_smithi079_83551_object30833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.035117+0000", 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 337.66831937500001, 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14157809499999999, 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.713 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30834 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035117+0000", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035117+0000", 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035115+0000", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035165+0000", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9579000000000001e-05 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035181+0000", 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6101000000000001e-05 2024-09-17T08:18:07.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035188+0000", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7939999999999997e-06 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035337+0000", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014974199999999999 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035480+0000", 2024-09-17T08:18:07.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014221700000000001 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035621+0000", 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014156800000000001 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176550+0000", 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14092914600000001 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.717 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176570+0000", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0013999999999998e-05 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176606+0000", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6130999999999997e-05 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176656+0000", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0096000000000003e-05 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176695+0000", 2024-09-17T08:18:07.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8436000000000003e-05 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30835 7.5 7:a9066964:::benchmark_data_smithi079_83551_object30834:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.035360+0000", 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 337.668076418, 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.147715869, 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.719 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.720 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.720 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.720 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30835 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035360+0000", 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035360+0000", 2024-09-17T08:18:07.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035359+0000", 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035424+0000", 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5690000000000003e-05 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035435+0000", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0393e-05 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035446+0000", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.096e-05 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175068+0000", 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13962180699999999 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175234+0000", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016660399999999999 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175408+0000", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000173487 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176818+0000", 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001410002 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176947+0000", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012901899999999999 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.182989+0000", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0060419280000000002 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.183037+0000", 2024-09-17T08:18:07.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7922e-05 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.183076+0000", 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9078000000000001e-05 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30834 7.5 e98/91 ondisk, result = 0)", 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.037138+0000", 2024-09-17T08:18:07.726 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 337.666298213, 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13941862299999999, 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037138+0000", 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037138+0000", 2024-09-17T08:18:07.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037137+0000", 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037142+0000", 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2930000000000002e-06 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174947+0000", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.137804975 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174954+0000", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5870000000000004e-06 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176537+0000", 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015828330000000001 2024-09-17T08:18:07.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176544+0000", 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1439999999999997e-06 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176557+0000", 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2449999999999999e-05 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30834 7.5 e98/91 ondisk, result = 0)", 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.037626+0000", 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 337.66581034400002, 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13907634699999999, 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037626+0000", 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037626+0000", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037625+0000", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037629+0000", 2024-09-17T08:18:07.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2899999999999996e-06 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174620+0000", 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13699086299999999 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174644+0000", 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3475000000000001e-05 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176597+0000", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001952676 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176602+0000", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3319999999999999e-06 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176702+0000", 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100382 2024-09-17T08:18:07.734 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67128 7.5 7:a1db3a8e:::benchmark_data_smithi079_83551_object67127:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.546055+0000", 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 314.15738089000001, 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.094852083000000004, 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.735 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67128 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546055+0000", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546055+0000", 2024-09-17T08:18:07.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546054+0000", 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546104+0000", 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9988999999999999e-05 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546122+0000", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7799999999999999e-05 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546129+0000", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3180000000000001e-06 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546172+0000", 2024-09-17T08:18:07.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3467999999999998e-05 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546331+0000", 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015862800000000001 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546470+0000", 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013906899999999999 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.550303+0000", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038328780000000001 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.550316+0000", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2969999999999999e-05 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.640800+0000", 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.090484113000000005 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.640853+0000", 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3380000000000001e-05 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.640907+0000", 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4051999999999997e-05 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:67129 7.5 7:a78af858:::benchmark_data_smithi079_83551_object67128:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:53.546238+0000", 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 314.15719823699999, 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.097436186999999994, 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 67129 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546238+0000", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546238+0000", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546237+0000", 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546322+0000", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4319000000000004e-05 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546350+0000", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8639000000000001e-05 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.546357+0000", 2024-09-17T08:18:07.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1250000000000004e-06 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.548556+0000", 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021987769999999998 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.548676+0000", 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120407 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.548790+0000", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000113843 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.550349+0000", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015585029999999999 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.550384+0000", 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5641e-05 2024-09-17T08:18:07.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.643598+0000", 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093213685000000004 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.643647+0000", 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9322e-05 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:53.643674+0000", 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6706000000000001e-05 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:123284 7.5 7:a4a80332:::benchmark_data_smithi079_83551_object123283:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:44.312651+0000", 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 263.39078532500002, 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11825376899999999, 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.748 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 123284 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312651+0000", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312651+0000", 2024-09-17T08:18:07.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312649+0000", 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312678+0000", 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8246e-05 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.750 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312688+0000", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0445000000000001e-05 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312694+0000", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0349999999999998e-06 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312747+0000", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2553999999999999e-05 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.312934+0000", 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018774300000000001 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313085+0000", 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000150631 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317464+0000", 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043791300000000002 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.320099+0000", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026344440000000001 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.430795+0000", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110696527 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.430847+0000", 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1505000000000002e-05 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.430905+0000", 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8019000000000001e-05 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:123285 7.5 7:ae87d071:::benchmark_data_smithi079_83551_object123284:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:44.313285+0000", 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 263.390150887, 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.117725249, 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.754 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 123285 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313285+0000", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313285+0000", 2024-09-17T08:18:07.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313284+0000", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313314+0000", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9529000000000001e-05 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313328+0000", 2024-09-17T08:18:07.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4472e-05 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.313335+0000", 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9469999999999997e-06 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.315411+0000", 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020752180000000002 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.757 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.315528+0000", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117026 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.315623+0000", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5565999999999999e-05 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.319470+0000", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0038466559999999999 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.758 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.320136+0000", 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00066577800000000001 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.430940+0000", 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110804337 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.430979+0000", 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9035999999999997e-05 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.431011+0000", 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1624000000000001e-05 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:123290 7.5 7:affd1176:::benchmark_data_smithi079_83551_object123289:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:13:44.317172+0000", 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 263.386263789, 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.113941327, 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.760 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 123290 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317172+0000", 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317172+0000", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317171+0000", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317242+0000", 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1401000000000003e-05 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317254+0000", 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1948e-05 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317260+0000", 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.144e-06 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317486+0000", 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00022554000000000001 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317601+0000", 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115253 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.317757+0000", 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015637100000000001 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.319564+0000", 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018064209999999999 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.320154+0000", 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000590356 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.431043+0000", 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110888326 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.431080+0000", 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7249000000000002e-05 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:13:44.431114+0000", 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3924000000000002e-05 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:212057 7.5 7:a03fac0e:::benchmark_data_smithi079_83551_object84657:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-17T08:18:07.767 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:09.612355+0000", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 178.09108168200001, 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10127380699999999, 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212057 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612355+0000", 2024-09-17T08:18:07.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612355+0000", 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612353+0000", 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612359+0000", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6439999999999997e-06 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612370+0000", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1069e-05 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612377+0000", 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3560000000000004e-06 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612428+0000", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0633000000000001e-05 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612561+0000", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013312099999999999 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.612691+0000", 2024-09-17T08:18:07.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012974300000000001 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.615040+0000", 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023494029999999999 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.620576+0000", 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0055362639999999999 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.713564+0000", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.092987313000000002 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.713599+0000", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5228000000000002e-05 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:09.713628+0000", 2024-09-17T08:18:07.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9348000000000001e-05 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:223013 7.5 7:ae718cad:::benchmark_data_smithi079_83551_object95613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:22.007765+0000", 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 165.69567101499999, 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11144543599999999, 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.774 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 223013 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007765+0000", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007765+0000", 2024-09-17T08:18:07.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007764+0000", 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007769+0000", 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5289999999999999e-06 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007781+0000", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1083999999999999e-05 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007786+0000", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.989e-06 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007818+0000", 2024-09-17T08:18:07.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2914999999999999e-05 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.007951+0000", 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013219999999999999 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.008083+0000", 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131996 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.010419+0000", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023366889999999999 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.028037+0000", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017617237000000001 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119153+0000", 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091116232000000005 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119186+0000", 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2700999999999997e-05 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119211+0000", 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5111000000000001e-05 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:223020 7.5 7:a2e8ffcd:::benchmark_data_smithi079_83551_object95620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:22.015193+0000", 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 165.68824348800001, 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10411187700000001, 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 223020 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.781 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015193+0000", 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015193+0000", 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015192+0000", 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015197+0000", 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5269999999999998e-06 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015208+0000", 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0661e-05 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015215+0000", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4279999999999999e-06 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015256+0000", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0419e-05 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015379+0000", 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012337200000000001 2024-09-17T08:18:07.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.015502+0000", 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012298499999999999 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.017773+0000", 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00227108 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.028059+0000", 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010285506999999999 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119249+0000", 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091190461 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119280+0000", 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0778000000000002e-05 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119305+0000", 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4729e-05 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:223024 7.5 7:a2b21256:::benchmark_data_smithi079_83551_object95624:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:22.022728+0000", 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 165.680708049, 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096642247000000001, 2024-09-17T08:18:07.787 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 223024 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022728+0000", 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022728+0000", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022726+0000", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022733+0000", 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4819999999999999e-06 2024-09-17T08:18:07.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022745+0000", 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2361000000000001e-05 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022752+0000", 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6549999999999998e-06 2024-09-17T08:18:07.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022824+0000", 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2758999999999999e-05 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.022937+0000", 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011260900000000001 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.023043+0000", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010572 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.025593+0000", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025500150000000001 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.032620+0000", 2024-09-17T08:18:07.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0070273690000000003 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119328+0000", 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.086708230999999997 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119352+0000", 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3779000000000001e-05 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.119370+0000", 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8207999999999999e-05 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:223025 7.5 7:a7f30dbe:::benchmark_data_smithi079_83551_object95625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:22.026631+0000", 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 165.676804721, 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.098109978, 2024-09-17T08:18:07.794 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 223025 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026631+0000", 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026631+0000", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026630+0000", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026635+0000", 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1540000000000004e-06 2024-09-17T08:18:07.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026644+0000", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8600000000000005e-06 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026651+0000", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.776e-06 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026677+0000", 2024-09-17T08:18:07.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5976e-05 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026753+0000", 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6210000000000004e-05 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.026821+0000", 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7633999999999997e-05 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.028130+0000", 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013088869999999999 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.032677+0000", 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0045472960000000002 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.124655+0000", 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091977324999999999 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.124703+0000", 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8087000000000002e-05 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:22.124741+0000", 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8803000000000001e-05 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:243142 7.5 7:aebba6bd:::benchmark_data_smithi079_83551_object115742:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:42.917212+0000", 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.78622456900001, 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099976871999999994, 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 243142 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917212+0000", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917212+0000", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917211+0000", 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917215+0000", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4240000000000003e-06 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917226+0000", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0607e-05 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.917232+0000", 2024-09-17T08:18:07.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1999999999999999e-06 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.921635+0000", 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044033850000000001 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.921759+0000", 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012374200000000001 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.921883+0000", 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012389000000000001 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.923459+0000", 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001575543 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.925511+0000", 2024-09-17T08:18:07.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020522330000000001 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.017132+0000", 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091620684999999993 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.017163+0000", 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1402999999999997e-05 2024-09-17T08:18:07.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.017189+0000", 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5548999999999999e-05 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:243149 7.5 7:aa8caad0:::benchmark_data_smithi079_83551_object115749:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:42.929018+0000", 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.77441833099999, 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096554763000000002, 2024-09-17T08:18:07.807 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:07.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:07 smithi079 ceph-mon[40649]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T08:18:07.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:07 smithi079 ceph-mon[40649]: pgmap v352: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-09-17T08:18:07.809 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:07 smithi079 ceph-mon[29565]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T08:18:07.809 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:07 smithi079 ceph-mon[29565]: pgmap v352: 148 pgs: 148 active+clean; 8.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-09-17T08:18:07.809 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 243149 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929018+0000", 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929018+0000", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929017+0000", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929022+0000", 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.259e-06 2024-09-17T08:18:07.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929032+0000", 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3749999999999992e-06 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929040+0000", 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6989999999999992e-06 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929134+0000", 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3696999999999994e-05 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929254+0000", 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120357 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.929381+0000", 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012644699999999999 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.934338+0000", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0049574450000000004 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.936943+0000", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026042970000000002 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025474+0000", 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.088531548000000002 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025529+0000", 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5359000000000003e-05 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025573+0000", 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3204000000000002e-05 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.815 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:243157 7.5 7:ae2da9c8:::benchmark_data_smithi079_83551_object115757:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:42.931259+0000", 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 144.77217736399999, 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.094453956000000006, 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 243157 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931259+0000", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931259+0000", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931258+0000", 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:18:07.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931263+0000", 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8489999999999997e-06 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931274+0000", 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1846e-05 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931280+0000", 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7440000000000004e-06 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.931920+0000", 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064012900000000002 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.932108+0000", 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018728199999999999 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.932217+0000", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010996 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.934402+0000", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021840430000000001 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:42.937035+0000", 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026330300000000002 2024-09-17T08:18:07.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025637+0000", 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.088602677000000005 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025680+0000", 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2574999999999998e-05 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.821 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:43.025713+0000", 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2997000000000003e-05 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:245262 7.5 7:abac4ec9:::benchmark_data_smithi079_83551_object117862:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:15:44.892584+0000", 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 142.810852393, 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.090591992999999996, 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.822 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 245262 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892584+0000", 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892584+0000", 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892582+0000", 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892588+0000", 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2369999999999997e-06 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892600+0000", 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2095e-05 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892606+0000", 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7989999999999999e-06 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892652+0000", 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5457999999999997e-05 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892779+0000", 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127506 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,2", 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.892899+0000", 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011971 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.897147+0000", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004248028 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.899169+0000", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020217270000000001 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.983119+0000", 2024-09-17T08:18:07.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.083950156999999997 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.983151+0000", 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.18e-05 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:15:44.983176+0000", 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5286000000000002e-05 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.828 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:114262 8.d 8:b6f0b089:::benchmark_data_smithi079_115040_object114261:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:17:41.626131+0000", 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.077304957999999, 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091366571999999993, 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 114262 2024-09-17T08:18:07.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626131+0000", 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626131+0000", 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626131+0000", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626231+0000", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010010599999999999 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626243+0000", 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2839e-05 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.626249+0000", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6200000000000004e-06 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.687169+0000", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.060920368000000003 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.687329+0000", 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015919900000000001 2024-09-17T08:18:07.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.687452+0000", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012313600000000001 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.694448+0000", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0069962729999999999 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.698230+0000", 2024-09-17T08:18:07.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037822189999999999 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.717401+0000", 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019170717 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.717455+0000", 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4268999999999998e-05 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:18:07.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:17:41.717498+0000", 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2536000000000001e-05 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:18:07.835 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:18:07.841 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:07.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:07.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:07.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:07.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:07.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:08.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:08.104 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:08.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:08.206 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:08.230 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:08.287 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:08.288 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:18:08.288 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:18:08.293 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:08.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:08.338 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:18:08.338 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:18:08.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:08.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.358 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:08.378 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:18:08.378 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:18:08.405 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:18:08.406 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:18:08.439 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:18:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:18:08.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:08.473 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:18:08.473 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:18:08.505 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:18:08.505 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:18:08.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.511 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:08.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:08.544 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:18:08.544 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:18:08.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3297807563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:08.599 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3297807563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:08.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:08.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.653 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:08.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3297807563' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:08.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:08.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:18:08.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.796 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:08.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:08.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:08.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:08.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:08.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:09.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:09.073 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:09.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:09.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:09.199 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:09.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.200 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:09.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:09.337 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:09.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.338 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:09.440 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:09.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:09.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:09.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:09.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:09.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:09.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:09.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:09.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:09.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:09 smithi079 ceph-mon[29565]: pgmap v353: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:09 smithi079 ceph-mon[40649]: pgmap v353: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:09.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:09.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:09.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:09 smithi173 ceph-mon[33187]: pgmap v353: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:09.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:10.011 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:10.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:10.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:10.138 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:10.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:10.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:10.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:10.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:10.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:10.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:10.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:10.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:10.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:10.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:10.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:10.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:10.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:10.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:10.812 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:10.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:10.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:10.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:10.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:10.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:11.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:11.081 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:11.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.082 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:11.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:11.220 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:11.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:11.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:11.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.363 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:11.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:11.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:11.508 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:11.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:11.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:11.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.650 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:11.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:11.751 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:11.786 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:11.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:11.887 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:11.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:11.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:11.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:12.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:12.052 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:12.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:12.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:12.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:12.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:12.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:12.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:12.331 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:12.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:12.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:12.470 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:12.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:12.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:12.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:12.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:12.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:12 smithi079 ceph-mon[29565]: pgmap v354: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 649 B/s rd, 0 B/s wr, 794 op/s 2024-09-17T08:18:12.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:12 smithi079 ceph-mon[40649]: pgmap v354: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 649 B/s rd, 0 B/s wr, 794 op/s 2024-09-17T08:18:12.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:12 smithi173 ceph-mon[33187]: pgmap v354: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 649 B/s rd, 0 B/s wr, 794 op/s 2024-09-17T08:18:12.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:12.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:12.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:12.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:12.894 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:12.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:12.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:12.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:13.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:13.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:13.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:13.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:13.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:13.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:13.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:18:13.309 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:13.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:13.343 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:13.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:13.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:13.455 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:18:13.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:13.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:13.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3477377004' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3477377004' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:13.715 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:13.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3477377004' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:18:13.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:13.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:13.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:13.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:13.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:14.004 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:14.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:14.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:14.134 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:14.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:14.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:14.268 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:14.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:14.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:14.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:14.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:14.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:14.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:14.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:14.551 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:14.551 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: pgmap v355: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: pgmap v355: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:14.552 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:18:14.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T08:18:14.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:18:14.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:18:14.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:18:14.553 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:14 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:18:14.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:14.677 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:14.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:14.719 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:18:14.719 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:18:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: pgmap v355: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:18:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:14 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:18:14.779 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:14.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:14.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:14.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:14.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:14.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:15.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:15.085 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:15.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:15.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:15.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.360 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:15.445 INFO:teuthology.orchestra.run.smithi079.stderr:set 6.b pg_upmap_items mapping to [3->2,1->6,5->4] 2024-09-17T08:18:15.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:15.506 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:15.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:15.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:18:15.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:18:15.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2798103706' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:18:15.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:18:15.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2798103706' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:15 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:15.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:15 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:18:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:15 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:18:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2798103706' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:15 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]: dispatch 2024-09-17T08:18:15.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:15.783 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:15.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:15.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:15.905 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:15.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:15.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:16.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:16.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.046 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:16.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:16.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:16.174 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:16.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:16.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:16.313 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:16.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:16.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:16.435 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:16.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.436 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:16.537 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:16.575 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:16.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:16.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:16.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:16.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:18:16.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: pgmap v357: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]': finished 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:18:16.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:16 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:18:16.807 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:16.841 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:16.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:16.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:18:16.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:18:16.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: pgmap v357: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]': finished 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:18:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:18:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: pgmap v357: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:18:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.b", "id": [3, 2, 1, 6, 5, 4]}]': finished 2024-09-17T08:18:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T08:18:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:18:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:16 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:18:16.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:16.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:16.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:16.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:17.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:17.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:17.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:17.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:17.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:17.313 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:17.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:17.400 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:17.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:17.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:17.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:17.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:17.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:17.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:17.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:17.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:17.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:17.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:17.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:18:17.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[29565]: pgmap v360: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T08:18:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:17.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:17 smithi079 ceph-mon[40649]: pgmap v360: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:18:17.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:17.927 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:17.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:17.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:17 smithi173 ceph-mon[33187]: pgmap v360: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:18:18.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:18.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:18.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:18.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:18.202 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:18.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:18.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:18.338 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:18.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:18.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:18.476 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:18.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:18.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:18.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:18.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:18.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:18 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:18:18.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:18 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:18:18.611 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:18 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:18:18.611 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:18 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:18:18.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.754 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:18.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:18:18.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:18.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:18 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:18:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:18 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:18:18.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:18.937 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:18.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:19.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:19.066 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:19.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:19.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:18:19.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:19.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:19.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:19.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:19.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:19.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:19.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:19.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:19.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:19.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:19.650 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:18:19.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:18:19.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: 8.6 scrub starts 2024-09-17T08:18:19.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:18:19.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:18:19.651 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:19 smithi173 ceph-mon[33187]: pgmap v361: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:19.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:19.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.657 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:19.756 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: 8.6 scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[40649]: pgmap v361: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: 8.6 scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:18:19.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:18:19.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:19 smithi079 ceph-mon[29565]: pgmap v361: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:18:19.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:19.794 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:19.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:19.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:19.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:19.937 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:19.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:20.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:20.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:20.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:20.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:20.196 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:20.198 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:20.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:20.246 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:20.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:20.325 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:20.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:20.427 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:20.461 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:20.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:20.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:20.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:20.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:20.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:20.739 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:20.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.740 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:20.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:20.850 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:18:20.850 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:18:20.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:20 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:18:20.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:20.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:20.889 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:18:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:18:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:18:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:18:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:18:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:20 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:18:20.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:21.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:21.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:21.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:21.167 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:21.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.168 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:21.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:21.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:21.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:21.385 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:18:21.386 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:18:21.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:21.434 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:18:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:18:21.456 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:21.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:21.468 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:18:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:18:21.503 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:18:21.503 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:18:21.543 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:18:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:18:21.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:21.575 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:18:21.575 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:18:21.589 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:21.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:21.606 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:18:21.606 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:18:21.641 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:18:21.641 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2952763181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[40649]: pgmap v362: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 645 B/s rd, 0 B/s wr, 786 op/s 2024-09-17T08:18:21.642 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2952763181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:21.643 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:21 smithi079 ceph-mon[29565]: pgmap v362: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 645 B/s rd, 0 B/s wr, 786 op/s 2024-09-17T08:18:21.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:21.721 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:21.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:21.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:21.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:21.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:21.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2952763181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:21 smithi173 ceph-mon[33187]: pgmap v362: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 645 B/s rd, 0 B/s wr, 786 op/s 2024-09-17T08:18:21.990 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:21.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:21.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:22.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:22.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:22.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.276 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:22.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:22.415 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:22.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:22.550 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:22.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:22.686 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:22.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:22.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:22.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:22.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 8.6 scrub ok 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 8.e scrub starts 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:18:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:18:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:18:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:18:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 8.6 scrub ok 2024-09-17T08:18:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:22.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 8.e scrub starts 2024-09-17T08:18:22.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:18:22.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:22 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:18:22.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:22.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:22.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:22.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:22.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 8.6 scrub ok 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 8.e scrub starts 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:18:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:22 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:18:23.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:23.096 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:23.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:23.197 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:23.226 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:23.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:23.327 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:23.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:23.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:23.481 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:23.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:23.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:23.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:23.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:23.651 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:18:23.651 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:18:23.652 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_1 2024-09-17T08:18:23.652 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_1 by 10 min 8 2024-09-17T08:18:23.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:18:23.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:23.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:23.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:23.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:18:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:18:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: 8.3 scrub starts 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[40649]: pgmap v364: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:18:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:18:23.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T08:18:23.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:23.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: 8.3 scrub starts 2024-09-17T08:18:23.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:23 smithi079 ceph-mon[29565]: pgmap v364: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-17T08:18:23.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:23.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:23.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: 8.3 scrub starts 2024-09-17T08:18:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:23 smithi173 ceph-mon[33187]: pgmap v364: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-17T08:18:23.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:24.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:24.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:24.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:24.164 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:24.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:24.302 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:24.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:24.404 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:24.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:24.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:24.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:24.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:24.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:24.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:24.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:24.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:24.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:24.818 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:24.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:24.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:18:24.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 8.9 scrub starts 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[29565]: 8.7 scrub starts 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:18:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:18:24.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 8.9 scrub starts 2024-09-17T08:18:24.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:24 smithi079 ceph-mon[40649]: 8.7 scrub starts 2024-09-17T08:18:24.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:24.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:24.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:24.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:18:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:18:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:18:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:18:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 8.9 scrub starts 2024-09-17T08:18:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:24 smithi173 ceph-mon[33187]: 8.7 scrub starts 2024-09-17T08:18:25.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:25.083 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:25.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:25.185 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:25.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:25.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:25.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:25.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:25.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:25.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:25.492 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:25.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:25.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:25.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:25.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:25.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:25.754 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:25.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:25.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:18:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:18:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 8.e scrub ok 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 8.c scrub starts 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: 8.3 scrub ok 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[40649]: pgmap v365: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 8.e scrub ok 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 8.c scrub starts 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: 8.3 scrub ok 2024-09-17T08:18:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:25 smithi079 ceph-mon[29565]: pgmap v365: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:18:25.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:25.883 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:25.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:18:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:18:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 8.e scrub ok 2024-09-17T08:18:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:18:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 8.c scrub starts 2024-09-17T08:18:25.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: 8.3 scrub ok 2024-09-17T08:18:25.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:25 smithi173 ceph-mon[33187]: pgmap v365: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:18:25.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:26.016 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:26.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:26.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:26.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.143 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:26.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:26.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:26.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.286 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:26.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:26.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:26.417 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:26.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:26.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:26.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:18:26.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:18:26.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:18:26.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:18:26.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:18:26.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:18:26.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:18:26.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:26 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:18:26.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.805 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:26.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:26.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:26.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:26.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:26 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:18:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:26 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:18:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:26 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:18:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:26 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:18:27.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:27.079 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:27.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:27.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:27.224 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:27.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:27.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:27.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:27.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:27.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:27.478 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:27.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:27.553 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:18:27.580 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:27.606 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:27.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:27.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:27.728 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:27.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:27.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:27.868 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:27.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:27.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:27.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: 8.9 scrub ok 2024-09-17T08:18:27.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: 8.b scrub starts 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: 8.7 scrub ok 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: 8.c scrub ok 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3068285129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[29565]: pgmap v366: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: 8.9 scrub ok 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: 8.b scrub starts 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: 8.7 scrub ok 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: 8.c scrub ok 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3068285129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:27 smithi079 ceph-mon[40649]: pgmap v366: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:27.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: 8.9 scrub ok 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: 8.b scrub starts 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: 8.7 scrub ok 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: 8.c scrub ok 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3068285129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:27 smithi173 ceph-mon[33187]: pgmap v366: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:28.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:28.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:28.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:28.152 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:28.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:28.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:28.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:28.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:28.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:28.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:28.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:28.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:18:28.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:28.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:28.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:28.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:28.700 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:28.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:28.796 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:28.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:18:28.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:28 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:18:28.801 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:28.834 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:28.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:28.936 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:28.965 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:28.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:28.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:18:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:28 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:18:29.067 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:29.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.096 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:29.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:29.197 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:29.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:29.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:29.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:29.355 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:29.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:29.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:29.483 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:29.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.485 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:29.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:29.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:29.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:29.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:29.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:29.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:29.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[40649]: 8.b scrub ok 2024-09-17T08:18:29.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[40649]: 5.15 deep-scrub starts 2024-09-17T08:18:29.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[40649]: 5.15 deep-scrub ok 2024-09-17T08:18:29.809 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[40649]: pgmap v368: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:18:29.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[29565]: 8.b scrub ok 2024-09-17T08:18:29.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[29565]: 5.15 deep-scrub starts 2024-09-17T08:18:29.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[29565]: 5.15 deep-scrub ok 2024-09-17T08:18:29.810 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:29 smithi079 ceph-mon[29565]: pgmap v368: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:18:29.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:29.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:29.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:29.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:29 smithi173 ceph-mon[33187]: 8.b scrub ok 2024-09-17T08:18:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:29 smithi173 ceph-mon[33187]: 5.15 deep-scrub starts 2024-09-17T08:18:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:29 smithi173 ceph-mon[33187]: 5.15 deep-scrub ok 2024-09-17T08:18:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:29 smithi173 ceph-mon[33187]: pgmap v368: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-17T08:18:29.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:30.023 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:30.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:30.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:30.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:30.276 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:18:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:18:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:18:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:18:30.276 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:18:30.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.301 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:30.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:30.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.441 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:30.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:30.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:30.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:30.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:30.709 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:30.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:30.834 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:30 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:18:30.834 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:30 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:18:30.834 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:30 smithi173 ceph-mon[33187]: 8.a scrub starts 2024-09-17T08:18:30.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:30.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:18:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:18:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[40649]: 8.a scrub starts 2024-09-17T08:18:30.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:18:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:18:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:30 smithi079 ceph-mon[29565]: 8.a scrub starts 2024-09-17T08:18:30.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:30.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:30.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:30.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:31.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:31.113 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:31.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:31.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:18:31.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:31.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:31.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:31.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:31.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.371 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:31.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:31.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:31.512 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:31.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:31.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:31.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:31.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:31.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:31.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:31.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:18:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:18:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:18:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[40649]: pgmap v369: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:31 smithi079 ceph-mon[29565]: pgmap v369: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:31.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:31.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:31.924 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:31.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:31 smithi173 ceph-mon[33187]: pgmap v369: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:32.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:32.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:32.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:32.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:32.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.187 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:32.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:32.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:32.315 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:32.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:32.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:32.424 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:32.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.484 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:32.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:32.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:32.632 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:32.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:32.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:32.757 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:32.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:32.860 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:18:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:18:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:18:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[40649]: 8.1 deep-scrub starts 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T08:18:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:32.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:32.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:32 smithi079 ceph-mon[29565]: 8.1 deep-scrub starts 2024-09-17T08:18:32.889 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:32.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:32.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:32.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:18:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:18:32.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:18:32.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:18:32.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T08:18:32.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:32.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:32.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:32 smithi173 ceph-mon[33187]: 8.1 deep-scrub starts 2024-09-17T08:18:32.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:33.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:33.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:33.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:33.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:33.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:33.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:33.534 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:33.559 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:33.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:33.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:33.681 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:33.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:33.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:33.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[40649]: 8.a scrub ok 2024-09-17T08:18:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[40649]: pgmap v371: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:18:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[29565]: 8.a scrub ok 2024-09-17T08:18:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:33 smithi079 ceph-mon[29565]: pgmap v371: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:18:33.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:33.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:33.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:33.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:33 smithi173 ceph-mon[33187]: 8.a scrub ok 2024-09-17T08:18:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:33 smithi173 ceph-mon[33187]: pgmap v371: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:18:34.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:34.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:34.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:34.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:34.218 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:34.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:34.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:34.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.353 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:34.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:34.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:34.482 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:34.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:34.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:34.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:34.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:34.713 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:34.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:34.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:34.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 5.6 deep-scrub starts 2024-09-17T08:18:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 5.6 deep-scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[40649]: 8.4 scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 5.6 deep-scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 5.6 deep-scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:18:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:34 smithi079 ceph-mon[29565]: 8.4 scrub starts 2024-09-17T08:18:34.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:34.892 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:34.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 5.6 deep-scrub starts 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 5.6 deep-scrub ok 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:18:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:34 smithi173 ceph-mon[33187]: 8.4 scrub starts 2024-09-17T08:18:34.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:35.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:35.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:35.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:35.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.165 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:35.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:35.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:35.302 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:35.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:35.404 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:35.431 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:35.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:35.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:35.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:18:35.591 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:35.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:35.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:35.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:35.720 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:35.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:35.822 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:35.852 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:35.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:35.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[29565]: pgmap v373: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:18:35.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:35 smithi079 ceph-mon[40649]: pgmap v373: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:35.954 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:18:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:35 smithi173 ceph-mon[33187]: pgmap v373: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:18:35.985 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:35.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:35.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:36.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:36.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.127 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:36.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:36.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:36.273 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:36.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:36.375 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:36.402 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:36.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.403 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:36.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:36.527 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:36.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:36.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:36.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.672 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:36.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:36.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[29565]: 8.4 scrub ok 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[29565]: 8.0 scrub starts 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[29565]: 8.2 scrub starts 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[40649]: 8.4 scrub ok 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[40649]: 8.0 scrub starts 2024-09-17T08:18:36.787 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:18:36.788 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:36 smithi079 ceph-mon[40649]: 8.2 scrub starts 2024-09-17T08:18:36.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:36.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:36.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:36.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:36.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:36.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:36 smithi173 ceph-mon[33187]: 8.4 scrub ok 2024-09-17T08:18:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:36 smithi173 ceph-mon[33187]: 8.0 scrub starts 2024-09-17T08:18:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:36 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:18:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:36 smithi173 ceph-mon[33187]: 8.2 scrub starts 2024-09-17T08:18:37.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:37.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:37.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:37.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:37.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:37.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:37.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:37.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:37.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:37.570 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:37.594 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:37.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:37.734 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:37.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:37.836 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:37.861 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 8.f scrub starts 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:18:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[29565]: pgmap v374: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 742 op/s 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 8.f scrub starts 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:18:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:37 smithi079 ceph-mon[40649]: pgmap v374: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 742 op/s 2024-09-17T08:18:37.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 8.f scrub starts 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:18:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:37 smithi173 ceph-mon[33187]: pgmap v374: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 742 op/s 2024-09-17T08:18:37.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:37.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:37.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:38.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:38.106 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:38.147 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:38.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:38.249 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:38.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:38.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:38.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:38.406 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:38.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:38.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:38.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:38.628 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:38.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:38.657 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:38.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:38.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:38.786 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:18:38.790 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:18:38.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:38.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:38.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 8.2 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 8.0 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 4.8 deep-scrub starts 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[29565]: 4.8 deep-scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 8.2 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 8.0 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 4.8 deep-scrub starts 2024-09-17T08:18:38.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:38 smithi079 ceph-mon[40649]: 4.8 deep-scrub ok 2024-09-17T08:18:38.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:38.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:38.945 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:38.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 8.2 scrub ok 2024-09-17T08:18:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 8.0 scrub ok 2024-09-17T08:18:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:18:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:18:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 4.8 deep-scrub starts 2024-09-17T08:18:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:38 smithi173 ceph-mon[33187]: 4.8 deep-scrub ok 2024-09-17T08:18:39.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:39.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:39.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:39.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:39.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:39.370 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:39.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:39.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:39.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:39.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.811 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: 8.f scrub ok 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[40649]: pgmap v375: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: 8.f scrub ok 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:18:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:18:39.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:39 smithi079 ceph-mon[29565]: pgmap v375: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:18:39.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:39.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:39.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:39.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: 8.f scrub ok 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:18:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:39 smithi173 ceph-mon[33187]: pgmap v375: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:18:40.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:40.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:40.179 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:40.202 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:40.304 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:40.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:40.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:40.465 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:40.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:40.592 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:40.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:40.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:40.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:40.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:40 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:18:40.843 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:40 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:18:40.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:40 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:18:40.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:40 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:18:40.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:40.870 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:40.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:40.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:40 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:18:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:40 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:18:41.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:41.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:41.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:41.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.147 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:41.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:41.248 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:41.277 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:41.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:41.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:41.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:41.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:41.412 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-17T08:18:37.380300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6257842478,"num_objects":95675,"num_object_clones":0,"num_object_copies":287025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95675,"num_whiteouts":0,"num_read":1104,"num_read_kb":950,"num_write":175166,"num_write_kb":8643548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":1776389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85352,"ondisk_log_size":85352,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20456812,"kb_used_data":18354460,"kb_used_omap":12,"kb_used_meta":2102003,"kb_avail":729504404,"statfs":{"total":767960285184,"available":747012509696,"internally_reserved":0,"allocated":18794967040,"data_stored":18792542047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":2152451640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,5,7,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-389218304,"num_objects":-5939,"num_object_clones":0,"num_object_copies":-17817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5939,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003958"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":177,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550669+0000","last_change":"2024-09-17T08:17:34.931931+0000","last_active":"2024-09-17T08:18:27.550669+0000","last_peered":"2024-09-17T08:18:27.550669+0000","last_clean":"2024-09-17T08:18:27.550669+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:18:27.550669+0000","last_undegraded":"2024-09-17T08:18:27.550669+0000","last_fullsized":"2024-09-17T08:18:27.550669+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.931732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.931732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.380258+0000","scrub_duration":0.0066078860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":195,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554621+0000","last_change":"2024-09-17T08:16:42.337270+0000","last_active":"2024-09-17T08:18:27.554621+0000","last_peered":"2024-09-17T08:18:27.554621+0000","last_clean":"2024-09-17T08:18:27.554621+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:18:27.554621+0000","last_undegraded":"2024-09-17T08:18:27.554621+0000","last_fullsized":"2024-09-17T08:18:27.554621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.337157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001642129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"72'15","reported_seq":206,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550099+0000","last_change":"2024-09-17T08:18:14.214347+0000","last_active":"2024-09-17T08:18:27.550099+0000","last_peered":"2024-09-17T08:18:27.550099+0000","last_clean":"2024-09-17T08:18:27.550099+0000","last_became_active":"2024-09-17T08:09:56.504223+0000","last_became_peered":"2024-09-17T08:09:56.504223+0000","last_unstale":"2024-09-17T08:18:27.550099+0000","last_undegraded":"2024-09-17T08:18:27.550099+0000","last_fullsized":"2024-09-17T08:18:27.550099+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'15","last_scrub_stamp":"2024-09-17T08:18:14.214228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:18:14.214228+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:21.065343+0000","scrub_duration":0.0096835010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":208,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547182+0000","last_change":"2024-09-17T08:18:23.654934+0000","last_active":"2024-09-17T08:18:27.547182+0000","last_peered":"2024-09-17T08:18:27.547182+0000","last_clean":"2024-09-17T08:18:27.547182+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:18:27.547182+0000","last_undegraded":"2024-09-17T08:18:27.547182+0000","last_fullsized":"2024-09-17T08:18:27.547182+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:23.654810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:23.654810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:26.074851+0000","scrub_duration":0.0028269580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":184,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555302+0000","last_change":"2024-09-17T08:18:20.301265+0000","last_active":"2024-09-17T08:18:27.555302+0000","last_peered":"2024-09-17T08:18:27.555302+0000","last_clean":"2024-09-17T08:18:27.555302+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:18:27.555302+0000","last_undegraded":"2024-09-17T08:18:27.555302+0000","last_fullsized":"2024-09-17T08:18:27.555302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:20.301115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:18:20.301115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:20.943367+0000","scrub_duration":0.036110046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"87'13","reported_seq":277,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547256+0000","last_change":"2024-09-17T08:16:35.827557+0000","last_active":"2024-09-17T08:18:27.547256+0000","last_peered":"2024-09-17T08:18:27.547256+0000","last_clean":"2024-09-17T08:18:27.547256+0000","last_became_active":"2024-09-17T08:16:33.326106+0000","last_became_peered":"2024-09-17T08:16:33.326106+0000","last_unstale":"2024-09-17T08:18:27.547256+0000","last_undegraded":"2024-09-17T08:18:27.547256+0000","last_fullsized":"2024-09-17T08:18:27.547256+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'13","last_scrub_stamp":"2024-09-17T08:16:35.827443+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:16:35.827443+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033149960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545530+0000","last_change":"2024-09-17T08:16:44.335176+0000","last_active":"2024-09-17T08:18:27.545530+0000","last_peered":"2024-09-17T08:18:27.545530+0000","last_clean":"2024-09-17T08:18:27.545530+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:18:27.545530+0000","last_undegraded":"2024-09-17T08:18:27.545530+0000","last_fullsized":"2024-09-17T08:18:27.545530+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.335057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.335057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065690419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":206,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547923+0000","last_change":"2024-09-17T08:16:48.900627+0000","last_active":"2024-09-17T08:18:27.547923+0000","last_peered":"2024-09-17T08:18:27.547923+0000","last_clean":"2024-09-17T08:18:27.547923+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:18:27.547923+0000","last_undegraded":"2024-09-17T08:18:27.547923+0000","last_fullsized":"2024-09-17T08:18:27.547923+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.900525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033962240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":195,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.920208+0000","last_change":"2024-09-17T08:18:32.920208+0000","last_active":"2024-09-17T08:18:32.920208+0000","last_peered":"2024-09-17T08:18:32.920208+0000","last_clean":"2024-09-17T08:18:32.920208+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:18:32.920208+0000","last_undegraded":"2024-09-17T08:18:32.920208+0000","last_fullsized":"2024-09-17T08:18:32.920208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:32.920123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:32.920123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.946961+0000","scrub_duration":0.0032487929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"85'7","reported_seq":234,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548247+0000","last_change":"2024-09-17T08:18:19.647765+0000","last_active":"2024-09-17T08:18:27.548247+0000","last_peered":"2024-09-17T08:18:27.548247+0000","last_clean":"2024-09-17T08:18:27.548247+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:18:27.548247+0000","last_undegraded":"2024-09-17T08:18:27.548247+0000","last_fullsized":"2024-09-17T08:18:27.548247+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'7","last_scrub_stamp":"2024-09-17T08:18:19.647688+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:18:19.647688+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:44.374748+0000","scrub_duration":0.0050351679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":220,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.595913+0000","last_change":"2024-09-17T08:18:20.872085+0000","last_active":"2024-09-17T08:18:31.595913+0000","last_peered":"2024-09-17T08:18:31.595913+0000","last_clean":"2024-09-17T08:18:31.595913+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:18:31.595913+0000","last_undegraded":"2024-09-17T08:18:31.595913+0000","last_fullsized":"2024-09-17T08:18:31.595913+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:20.871982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:49.012248+0000","last_clean_scrub_stamp":"2024-09-17T08:18:20.871982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:40.838812+0000","scrub_duration":0.002490683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545350+0000","last_change":"2024-09-17T08:16:23.065070+0000","last_active":"2024-09-17T08:18:27.545350+0000","last_peered":"2024-09-17T08:18:27.545350+0000","last_clean":"2024-09-17T08:18:27.545350+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:18:27.545350+0000","last_undegraded":"2024-09-17T08:18:27.545350+0000","last_fullsized":"2024-09-17T08:18:27.545350+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:23.064993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:23.064993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015683839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":206,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T08:18:35.012638+0000","last_change":"2024-09-17T08:18:35.012737+0000","last_active":"2024-09-17T08:18:35.012638+0000","last_peered":"2024-09-17T08:18:35.012638+0000","last_clean":"2024-09-17T08:18:35.012638+0000","last_became_active":"2024-09-17T08:09:54.502056+0000","last_became_peered":"2024-09-17T08:09:54.502056+0000","last_unstale":"2024-09-17T08:18:35.012638+0000","last_undegraded":"2024-09-17T08:18:35.012638+0000","last_fullsized":"2024-09-17T08:18:35.012638+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:35.012616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:18:35.012616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:54.281670+0000","scrub_duration":0.0028422709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"72'9","reported_seq":208,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548173+0000","last_change":"2024-09-17T08:16:46.894268+0000","last_active":"2024-09-17T08:18:27.548173+0000","last_peered":"2024-09-17T08:18:27.548173+0000","last_clean":"2024-09-17T08:18:27.548173+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:18:27.548173+0000","last_undegraded":"2024-09-17T08:18:27.548173+0000","last_fullsized":"2024-09-17T08:18:27.548173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:16:46.894134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.894134+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053467080000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":93,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554368+0000","last_change":"2024-09-17T08:18:13.676803+0000","last_active":"2024-09-17T08:18:27.554368+0000","last_peered":"2024-09-17T08:18:27.554368+0000","last_clean":"2024-09-17T08:18:27.554368+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:18:27.554368+0000","last_undegraded":"2024-09-17T08:18:27.554368+0000","last_fullsized":"2024-09-17T08:18:27.554368+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:13.676676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:13.676676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:31.121454+0000","scrub_duration":0.006122415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":206,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554020+0000","last_change":"2024-09-17T08:17:31.421060+0000","last_active":"2024-09-17T08:18:27.554020+0000","last_peered":"2024-09-17T08:18:27.554020+0000","last_clean":"2024-09-17T08:18:27.554020+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:18:27.554020+0000","last_undegraded":"2024-09-17T08:18:27.554020+0000","last_fullsized":"2024-09-17T08:18:27.554020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.420934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.420934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.381148+0000","scrub_duration":0.0086368369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"202'11052","reported_seq":11090,"reported_epoch":202,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-17T08:18:32.783195+0000","last_change":"2024-09-17T08:18:32.009873+0000","last_active":"2024-09-17T08:18:32.783195+0000","last_peered":"2024-09-17T08:18:32.783195+0000","last_clean":"2024-09-17T08:18:32.783195+0000","last_became_active":"2024-09-17T08:16:16.645277+0000","last_became_peered":"2024-09-17T08:16:16.645277+0000","last_unstale":"2024-09-17T08:18:32.783195+0000","last_undegraded":"2024-09-17T08:18:32.783195+0000","last_fullsized":"2024-09-17T08:18:32.783195+0000","mapping_epoch":165,"log_start":"188'5900","ondisk_log_start":"188'5900","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":124,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392953856,"num_objects":5996,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10928,"num_write_kb":541568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"72'17","reported_seq":263,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.549115+0000","last_change":"2024-09-17T08:16:26.873516+0000","last_active":"2024-09-17T08:18:27.549115+0000","last_peered":"2024-09-17T08:18:27.549115+0000","last_clean":"2024-09-17T08:18:27.549115+0000","last_became_active":"2024-09-17T08:15:29.362201+0000","last_became_peered":"2024-09-17T08:15:29.362201+0000","last_unstale":"2024-09-17T08:18:27.549115+0000","last_undegraded":"2024-09-17T08:18:27.549115+0000","last_fullsized":"2024-09-17T08:18:27.549115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'17","last_scrub_stamp":"2024-09-17T08:16:26.873402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:26.873402+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002741856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":195,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596370+0000","last_change":"2024-09-17T08:17:36.164173+0000","last_active":"2024-09-17T08:18:31.596370+0000","last_peered":"2024-09-17T08:18:31.596370+0000","last_clean":"2024-09-17T08:18:31.596370+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:18:31.596370+0000","last_undegraded":"2024-09-17T08:18:31.596370+0000","last_fullsized":"2024-09-17T08:18:31.596370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.164053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.164053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:45.324794+0000","scrub_duration":0.0014889840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546078+0000","last_change":"2024-09-17T08:17:46.704576+0000","last_active":"2024-09-17T08:18:27.546078+0000","last_peered":"2024-09-17T08:18:27.546078+0000","last_clean":"2024-09-17T08:18:27.546078+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:18:27.546078+0000","last_undegraded":"2024-09-17T08:18:27.546078+0000","last_fullsized":"2024-09-17T08:18:27.546078+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:46.704456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:46.704456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.759310+0000","scrub_duration":0.0093420550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":199,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548660+0000","last_change":"2024-09-17T08:17:49.298346+0000","last_active":"2024-09-17T08:18:27.548660+0000","last_peered":"2024-09-17T08:18:27.548660+0000","last_clean":"2024-09-17T08:18:27.548660+0000","last_became_active":"2024-09-17T08:10:00.521168+0000","last_became_peered":"2024-09-17T08:10:00.521168+0000","last_unstale":"2024-09-17T08:18:27.548660+0000","last_undegraded":"2024-09-17T08:18:27.548660+0000","last_fullsized":"2024-09-17T08:18:27.548660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:49.295155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:49.295155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:55.253615+0000","scrub_duration":0.010469981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"202'10867","reported_seq":10914,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.888240+0000","last_change":"2024-09-17T08:16:16.648796+0000","last_active":"2024-09-17T08:18:32.888240+0000","last_peered":"2024-09-17T08:18:32.888240+0000","last_clean":"2024-09-17T08:18:32.888240+0000","last_became_active":"2024-09-17T08:16:16.648526+0000","last_became_peered":"2024-09-17T08:16:16.648526+0000","last_unstale":"2024-09-17T08:18:32.888240+0000","last_undegraded":"2024-09-17T08:18:32.888240+0000","last_fullsized":"2024-09-17T08:18:32.888240+0000","mapping_epoch":165,"log_start":"188'5800","ondisk_log_start":"188'5800","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387383296,"num_objects":5911,"num_object_clones":0,"num_object_copies":17733,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10867,"num_write_kb":536896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"72'10","reported_seq":207,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548422+0000","last_change":"2024-09-17T08:16:56.688647+0000","last_active":"2024-09-17T08:18:27.548422+0000","last_peered":"2024-09-17T08:18:27.548422+0000","last_clean":"2024-09-17T08:18:27.548422+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:18:27.548422+0000","last_undegraded":"2024-09-17T08:18:27.548422+0000","last_fullsized":"2024-09-17T08:18:27.548422+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:56.688373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.688373+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062131820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546040+0000","last_change":"2024-09-17T08:17:40.659072+0000","last_active":"2024-09-17T08:18:27.546040+0000","last_peered":"2024-09-17T08:18:27.546040+0000","last_clean":"2024-09-17T08:18:27.546040+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:18:27.546040+0000","last_undegraded":"2024-09-17T08:18:27.546040+0000","last_fullsized":"2024-09-17T08:18:27.546040+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.658816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:51.813043+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.658816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:47.720191+0000","scrub_duration":0.011311672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"84'9","reported_seq":313,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.932056+0000","last_change":"2024-09-17T08:17:42.416017+0000","last_active":"2024-09-17T08:18:27.932056+0000","last_peered":"2024-09-17T08:18:27.932056+0000","last_clean":"2024-09-17T08:18:27.932056+0000","last_became_active":"2024-09-17T08:09:58.512641+0000","last_became_peered":"2024-09-17T08:09:58.512641+0000","last_unstale":"2024-09-17T08:18:27.932056+0000","last_undegraded":"2024-09-17T08:18:27.932056+0000","last_fullsized":"2024-09-17T08:18:27.932056+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-17T08:17:42.415151+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:17:42.415151+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.003958+0000","scrub_duration":0.0143199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":195,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548527+0000","last_change":"2024-09-17T08:16:30.856483+0000","last_active":"2024-09-17T08:18:27.548527+0000","last_peered":"2024-09-17T08:18:27.548527+0000","last_clean":"2024-09-17T08:18:27.548527+0000","last_became_active":"2024-09-17T08:15:29.358693+0000","last_became_peered":"2024-09-17T08:15:29.358693+0000","last_unstale":"2024-09-17T08:18:27.548527+0000","last_undegraded":"2024-09-17T08:18:27.548527+0000","last_fullsized":"2024-09-17T08:18:27.548527+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:30.856368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.856368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044042839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"202'10761","reported_seq":10947,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.765912+0000","last_change":"2024-09-17T08:18:25.118827+0000","last_active":"2024-09-17T08:18:32.765912+0000","last_peered":"2024-09-17T08:18:32.765912+0000","last_clean":"2024-09-17T08:18:32.765912+0000","last_became_active":"2024-09-17T08:16:16.647816+0000","last_became_peered":"2024-09-17T08:16:16.647816+0000","last_unstale":"2024-09-17T08:18:32.765912+0000","last_undegraded":"2024-09-17T08:18:32.765912+0000","last_fullsized":"2024-09-17T08:18:32.765912+0000","mapping_epoch":165,"log_start":"187'5600","ondisk_log_start":"187'5600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10246","last_scrub_stamp":"2024-09-17T08:18:25.118613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:25.118613+0000","objects_scrubbed":6473,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:31.815722+0000","scrub_duration":2.3375071890000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386072576,"num_objects":5891,"num_object_clones":0,"num_object_copies":17673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10761,"num_write_kb":532864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"105'22","reported_seq":78,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547412+0000","last_change":"2024-09-17T08:18:21.891394+0000","last_active":"2024-09-17T08:18:27.547412+0000","last_peered":"2024-09-17T08:18:27.547412+0000","last_clean":"2024-09-17T08:18:27.547412+0000","last_became_active":"2024-09-17T08:16:33.328511+0000","last_became_peered":"2024-09-17T08:16:33.328511+0000","last_unstale":"2024-09-17T08:18:27.547412+0000","last_undegraded":"2024-09-17T08:18:27.547412+0000","last_fullsized":"2024-09-17T08:18:27.547412+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'22","last_scrub_stamp":"2024-09-17T08:18:21.890844+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:18:21.890844+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.221051+0000","scrub_duration":0.013990831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545454+0000","last_change":"2024-09-17T08:16:40.302468+0000","last_active":"2024-09-17T08:18:27.545454+0000","last_peered":"2024-09-17T08:18:27.545454+0000","last_clean":"2024-09-17T08:18:27.545454+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:18:27.545454+0000","last_undegraded":"2024-09-17T08:18:27.545454+0000","last_fullsized":"2024-09-17T08:18:27.545454+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:40.302235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:40.302235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071342690000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"72'8","reported_seq":301,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:29.675693+0000","last_change":"2024-09-17T08:18:29.675693+0000","last_active":"2024-09-17T08:18:29.675693+0000","last_peered":"2024-09-17T08:18:29.675693+0000","last_clean":"2024-09-17T08:18:29.675693+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:18:29.675693+0000","last_undegraded":"2024-09-17T08:18:29.675693+0000","last_fullsized":"2024-09-17T08:18:29.675693+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:18:29.675555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:29.675555+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:54.870903+0000","scrub_duration":0.001956682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546642+0000","last_change":"2024-09-17T08:16:56.421870+0000","last_active":"2024-09-17T08:18:27.546642+0000","last_peered":"2024-09-17T08:18:27.546642+0000","last_clean":"2024-09-17T08:18:27.546642+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:18:27.546642+0000","last_undegraded":"2024-09-17T08:18:27.546642+0000","last_fullsized":"2024-09-17T08:18:27.546642+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:56.421736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:56.421736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048071169999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"202'10946","reported_seq":10993,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.358669+0000","last_change":"2024-09-17T08:16:16.649421+0000","last_active":"2024-09-17T08:18:32.358669+0000","last_peered":"2024-09-17T08:18:32.358669+0000","last_clean":"2024-09-17T08:18:32.358669+0000","last_became_active":"2024-09-17T08:16:16.648969+0000","last_became_peered":"2024-09-17T08:16:16.648969+0000","last_unstale":"2024-09-17T08:18:32.358669+0000","last_undegraded":"2024-09-17T08:18:32.358669+0000","last_fullsized":"2024-09-17T08:18:32.358669+0000","mapping_epoch":165,"log_start":"186'5600","ondisk_log_start":"186'5600","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395968512,"num_objects":6042,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10946,"num_write_kb":543616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"156'16","reported_seq":248,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548623+0000","last_change":"2024-09-17T08:18:27.548623+0000","last_active":"2024-09-17T08:18:27.548623+0000","last_peered":"2024-09-17T08:18:27.548623+0000","last_clean":"2024-09-17T08:18:27.548623+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:18:27.548623+0000","last_undegraded":"2024-09-17T08:18:27.548623+0000","last_fullsized":"2024-09-17T08:18:27.548623+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'16","last_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:15:44.029355+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011380917000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":214,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T08:18:37.035238+0000","last_change":"2024-09-17T08:18:37.035238+0000","last_active":"2024-09-17T08:18:37.035238+0000","last_peered":"2024-09-17T08:18:37.035238+0000","last_clean":"2024-09-17T08:18:37.035238+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:18:37.035238+0000","last_undegraded":"2024-09-17T08:18:37.035238+0000","last_fullsized":"2024-09-17T08:18:37.035238+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:37.035116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:37.035116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:04.794350+0000","scrub_duration":0.003989411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":83,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547783+0000","last_change":"2024-09-17T08:16:25.867376+0000","last_active":"2024-09-17T08:18:27.547783+0000","last_peered":"2024-09-17T08:18:27.547783+0000","last_clean":"2024-09-17T08:18:27.547783+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:18:27.547783+0000","last_undegraded":"2024-09-17T08:18:27.547783+0000","last_fullsized":"2024-09-17T08:18:27.547783+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:25.867267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:25.867267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015224139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":177,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.549821+0000","last_change":"2024-09-17T08:17:39.909841+0000","last_active":"2024-09-17T08:18:27.549821+0000","last_peered":"2024-09-17T08:18:27.549821+0000","last_clean":"2024-09-17T08:18:27.549821+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:18:27.549821+0000","last_undegraded":"2024-09-17T08:18:27.549821+0000","last_fullsized":"2024-09-17T08:18:27.549821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.909586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.909586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.911382+0000","scrub_duration":0.001911444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"203'11130","reported_seq":11334,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T08:18:37.336812+0000","last_change":"2024-09-17T08:17:41.995541+0000","last_active":"2024-09-17T08:18:37.336812+0000","last_peered":"2024-09-17T08:18:37.336812+0000","last_clean":"2024-09-17T08:18:37.336812+0000","last_became_active":"2024-09-17T08:16:16.648937+0000","last_became_peered":"2024-09-17T08:16:16.648937+0000","last_unstale":"2024-09-17T08:18:37.336812+0000","last_undegraded":"2024-09-17T08:18:37.336812+0000","last_fullsized":"2024-09-17T08:18:37.336812+0000","mapping_epoch":165,"log_start":"188'6200","ondisk_log_start":"188'6200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'7187","last_scrub_stamp":"2024-09-17T08:17:41.995394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.995394+0000","objects_scrubbed":7135,"log_size":4930,"log_dups_size":0,"ondisk_log_size":4930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:50.208844+0000","scrub_duration":1.840802743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376504344,"num_objects":5746,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11130,"num_write_kb":539969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1","version":"72'14","reported_seq":203,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547655+0000","last_change":"2024-09-17T08:16:27.874195+0000","last_active":"2024-09-17T08:18:27.547655+0000","last_peered":"2024-09-17T08:18:27.547655+0000","last_clean":"2024-09-17T08:18:27.547655+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:18:27.547655+0000","last_undegraded":"2024-09-17T08:18:27.547655+0000","last_fullsized":"2024-09-17T08:18:27.547655+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'14","last_scrub_stamp":"2024-09-17T08:16:27.874083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:27.874083+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022658510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":146,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596518+0000","last_change":"2024-09-17T08:17:50.213103+0000","last_active":"2024-09-17T08:18:31.596518+0000","last_peered":"2024-09-17T08:18:31.596518+0000","last_clean":"2024-09-17T08:18:31.596518+0000","last_became_active":"2024-09-17T08:12:46.185345+0000","last_became_peered":"2024-09-17T08:12:46.185345+0000","last_unstale":"2024-09-17T08:18:31.596518+0000","last_undegraded":"2024-09-17T08:18:31.596518+0000","last_fullsized":"2024-09-17T08:18:31.596518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:50.212978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:17:50.212978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.476553+0000","scrub_duration":0.0039563619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":324,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.593190+0000","last_change":"2024-09-17T08:16:48.678053+0000","last_active":"2024-09-17T08:18:31.593190+0000","last_peered":"2024-09-17T08:18:31.593190+0000","last_clean":"2024-09-17T08:18:31.593190+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:18:31.593190+0000","last_undegraded":"2024-09-17T08:18:31.593190+0000","last_fullsized":"2024-09-17T08:18:31.593190+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.677913+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032798792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":190,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596115+0000","last_change":"2024-09-17T08:16:47.964627+0000","last_active":"2024-09-17T08:18:31.596115+0000","last_peered":"2024-09-17T08:18:31.596115+0000","last_clean":"2024-09-17T08:18:31.596115+0000","last_became_active":"2024-09-17T08:10:00.598063+0000","last_became_peered":"2024-09-17T08:10:00.598063+0000","last_unstale":"2024-09-17T08:18:31.596115+0000","last_undegraded":"2024-09-17T08:18:31.596115+0000","last_fullsized":"2024-09-17T08:18:31.596115+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.964546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015627130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"202'10804","reported_seq":10991,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:33.210111+0000","last_change":"2024-09-17T08:18:26.785074+0000","last_active":"2024-09-17T08:18:33.210111+0000","last_peered":"2024-09-17T08:18:33.210111+0000","last_clean":"2024-09-17T08:18:33.210111+0000","last_became_active":"2024-09-17T08:16:16.404227+0000","last_became_peered":"2024-09-17T08:16:16.404227+0000","last_unstale":"2024-09-17T08:18:33.210111+0000","last_undegraded":"2024-09-17T08:18:33.210111+0000","last_fullsized":"2024-09-17T08:18:33.210111+0000","mapping_epoch":165,"log_start":"184'5100","ondisk_log_start":"184'5100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10380","last_scrub_stamp":"2024-09-17T08:18:26.784887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:26.784887+0000","objects_scrubbed":6315,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:38.203778+0000","scrub_duration":2.1258120050000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381157376,"num_objects":5816,"num_object_clones":0,"num_object_copies":17448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10804,"num_write_kb":531840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"72'18","reported_seq":214,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.551312+0000","last_change":"2024-09-17T08:15:52.770015+0000","last_active":"2024-09-17T08:18:27.551312+0000","last_peered":"2024-09-17T08:18:27.551312+0000","last_clean":"2024-09-17T08:18:27.551312+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:18:27.551312+0000","last_undegraded":"2024-09-17T08:18:27.551312+0000","last_fullsized":"2024-09-17T08:18:27.551312+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'18","last_scrub_stamp":"2024-09-17T08:15:52.769743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:15:52.769743+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.023040108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547101+0000","last_change":"2024-09-17T08:17:02.482908+0000","last_active":"2024-09-17T08:18:27.547101+0000","last_peered":"2024-09-17T08:18:27.547101+0000","last_clean":"2024-09-17T08:18:27.547101+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:18:27.547101+0000","last_undegraded":"2024-09-17T08:18:27.547101+0000","last_fullsized":"2024-09-17T08:18:27.547101+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:02.482796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.482796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006488972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":191,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547500+0000","last_change":"2024-09-17T08:18:15.727574+0000","last_active":"2024-09-17T08:18:27.547500+0000","last_peered":"2024-09-17T08:18:27.547500+0000","last_clean":"2024-09-17T08:18:27.547500+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:18:27.547500+0000","last_undegraded":"2024-09-17T08:18:27.547500+0000","last_fullsized":"2024-09-17T08:18:27.547500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:15.727466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:18:15.727466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:30.482704+0000","scrub_duration":0.023735761000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":189,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548096+0000","last_change":"2024-09-17T08:18:21.660430+0000","last_active":"2024-09-17T08:18:27.548096+0000","last_peered":"2024-09-17T08:18:27.548096+0000","last_clean":"2024-09-17T08:18:27.548096+0000","last_became_active":"2024-09-17T08:10:00.524411+0000","last_became_peered":"2024-09-17T08:10:00.524411+0000","last_unstale":"2024-09-17T08:18:27.548096+0000","last_undegraded":"2024-09-17T08:18:27.548096+0000","last_fullsized":"2024-09-17T08:18:27.548096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:21.660330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:21.660330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:44.114406+0000","scrub_duration":0.0040630429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"202'10806","reported_seq":10852,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.949860+0000","last_change":"2024-09-17T08:16:16.647250+0000","last_active":"2024-09-17T08:18:32.949860+0000","last_peered":"2024-09-17T08:18:32.949860+0000","last_clean":"2024-09-17T08:18:32.949860+0000","last_became_active":"2024-09-17T08:16:16.646899+0000","last_became_peered":"2024-09-17T08:16:16.646899+0000","last_unstale":"2024-09-17T08:18:32.949860+0000","last_undegraded":"2024-09-17T08:18:32.949860+0000","last_fullsized":"2024-09-17T08:18:32.949860+0000","mapping_epoch":165,"log_start":"188'5800","ondisk_log_start":"188'5800","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":5952,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10806,"num_write_kb":536256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"72'19","reported_seq":229,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554338+0000","last_change":"2024-09-17T08:16:48.235582+0000","last_active":"2024-09-17T08:18:27.554338+0000","last_peered":"2024-09-17T08:18:27.554338+0000","last_clean":"2024-09-17T08:18:27.554338+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:18:27.554338+0000","last_undegraded":"2024-09-17T08:18:27.554338+0000","last_fullsized":"2024-09-17T08:18:27.554338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:16:48.235424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:48.235424+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061761419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":201,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.600900+0000","last_change":"2024-09-17T08:17:35.141768+0000","last_active":"2024-09-17T08:18:31.600900+0000","last_peered":"2024-09-17T08:18:31.600900+0000","last_clean":"2024-09-17T08:18:31.600900+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:18:31.600900+0000","last_undegraded":"2024-09-17T08:18:31.600900+0000","last_fullsized":"2024-09-17T08:18:31.600900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:17:35.141551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.141551+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.706752+0000","scrub_duration":0.0071014449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":146,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555794+0000","last_change":"2024-09-17T08:17:44.468391+0000","last_active":"2024-09-17T08:18:27.555794+0000","last_peered":"2024-09-17T08:18:27.555794+0000","last_clean":"2024-09-17T08:18:27.555794+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:18:27.555794+0000","last_undegraded":"2024-09-17T08:18:27.555794+0000","last_fullsized":"2024-09-17T08:18:27.555794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:17:44.468114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:12.768554+0000","scrub_duration":0.003219296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545613+0000","last_change":"2024-09-17T08:17:44.618063+0000","last_active":"2024-09-17T08:18:27.545613+0000","last_peered":"2024-09-17T08:18:27.545613+0000","last_clean":"2024-09-17T08:18:27.545613+0000","last_became_active":"2024-09-17T08:15:29.372791+0000","last_became_peered":"2024-09-17T08:15:29.372791+0000","last_unstale":"2024-09-17T08:18:27.545613+0000","last_undegraded":"2024-09-17T08:18:27.545613+0000","last_fullsized":"2024-09-17T08:18:27.545613+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:44.617613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:44.617613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:05.250750+0000","scrub_duration":0.0062292040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"202'10842","reported_seq":11034,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:33.194095+0000","last_change":"2024-09-17T08:18:24.288753+0000","last_active":"2024-09-17T08:18:33.194095+0000","last_peered":"2024-09-17T08:18:33.194095+0000","last_clean":"2024-09-17T08:18:33.194095+0000","last_became_active":"2024-09-17T08:16:16.645315+0000","last_became_peered":"2024-09-17T08:16:16.645315+0000","last_unstale":"2024-09-17T08:18:33.194095+0000","last_undegraded":"2024-09-17T08:18:33.194095+0000","last_fullsized":"2024-09-17T08:18:33.194095+0000","mapping_epoch":165,"log_start":"184'5100","ondisk_log_start":"184'5100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10213","last_scrub_stamp":"2024-09-17T08:18:24.288567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:24.288567+0000","objects_scrubbed":6702,"log_size":5742,"log_dups_size":0,"ondisk_log_size":5742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:30.648027+0000","scrub_duration":2.679609535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393084928,"num_objects":5998,"num_object_clones":0,"num_object_copies":17994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10842,"num_write_kb":538880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"156'15","reported_seq":101,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545737+0000","last_change":"2024-09-17T08:18:18.769473+0000","last_active":"2024-09-17T08:18:27.545737+0000","last_peered":"2024-09-17T08:18:27.545737+0000","last_clean":"2024-09-17T08:18:27.545737+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:18:27.545737+0000","last_undegraded":"2024-09-17T08:18:27.545737+0000","last_fullsized":"2024-09-17T08:18:27.545737+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'15","last_scrub_stamp":"2024-09-17T08:18:18.769332+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:18:18.769332+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.731224+0000","scrub_duration":0.013677144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547339+0000","last_change":"2024-09-17T08:17:37.532454+0000","last_active":"2024-09-17T08:18:27.547339+0000","last_peered":"2024-09-17T08:18:27.547339+0000","last_clean":"2024-09-17T08:18:27.547339+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:18:27.547339+0000","last_undegraded":"2024-09-17T08:18:27.547339+0000","last_fullsized":"2024-09-17T08:18:27.547339+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.532313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.532313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:43.540050+0000","scrub_duration":0.012255401000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"72'8","reported_seq":303,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.583862+0000","last_change":"2024-09-17T08:17:00.237348+0000","last_active":"2024-09-17T08:18:31.583862+0000","last_peered":"2024-09-17T08:18:31.583862+0000","last_clean":"2024-09-17T08:18:31.583862+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:18:31.583862+0000","last_undegraded":"2024-09-17T08:18:31.583862+0000","last_fullsized":"2024-09-17T08:18:31.583862+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:17:00.237229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.237229+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004406877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":189,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.638129+0000","last_change":"2024-09-17T08:18:27.638129+0000","last_active":"2024-09-17T08:18:27.638129+0000","last_peered":"2024-09-17T08:18:27.638129+0000","last_clean":"2024-09-17T08:18:27.638129+0000","last_became_active":"2024-09-17T08:10:00.516177+0000","last_became_peered":"2024-09-17T08:10:00.516177+0000","last_unstale":"2024-09-17T08:18:27.638129+0000","last_undegraded":"2024-09-17T08:18:27.638129+0000","last_fullsized":"2024-09-17T08:18:27.638129+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:27.638023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:27.638023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.651800+0000","scrub_duration":0.0042694100000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"202'10966","reported_seq":11013,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.932812+0000","last_change":"2024-09-17T08:16:16.648773+0000","last_active":"2024-09-17T08:18:32.932812+0000","last_peered":"2024-09-17T08:18:32.932812+0000","last_clean":"2024-09-17T08:18:32.932812+0000","last_became_active":"2024-09-17T08:16:16.648491+0000","last_became_peered":"2024-09-17T08:16:16.648491+0000","last_unstale":"2024-09-17T08:18:32.932812+0000","last_undegraded":"2024-09-17T08:18:32.932812+0000","last_fullsized":"2024-09-17T08:18:32.932812+0000","mapping_epoch":165,"log_start":"188'5900","ondisk_log_start":"188'5900","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5066,"log_dups_size":0,"ondisk_log_size":5066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398589952,"num_objects":6082,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"152'30","reported_seq":132,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.929178+0000","last_change":"2024-09-17T08:17:41.611847+0000","last_active":"2024-09-17T08:18:27.929178+0000","last_peered":"2024-09-17T08:18:27.929178+0000","last_clean":"2024-09-17T08:18:27.929178+0000","last_became_active":"2024-09-17T08:15:29.385461+0000","last_became_peered":"2024-09-17T08:15:29.385461+0000","last_unstale":"2024-09-17T08:18:27.929178+0000","last_undegraded":"2024-09-17T08:18:27.929178+0000","last_fullsized":"2024-09-17T08:18:27.929178+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'30","last_scrub_stamp":"2024-09-17T08:17:41.611678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:41.611678+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:46.598787+0000","scrub_duration":0.006094777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545975+0000","last_change":"2024-09-17T08:18:20.731803+0000","last_active":"2024-09-17T08:18:27.545975+0000","last_peered":"2024-09-17T08:18:27.545975+0000","last_clean":"2024-09-17T08:18:27.545975+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:18:27.545975+0000","last_undegraded":"2024-09-17T08:18:27.545975+0000","last_fullsized":"2024-09-17T08:18:27.545975+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:20.731620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:18:20.731620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:28.442790+0000","scrub_duration":0.014696991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":209,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554387+0000","last_change":"2024-09-17T08:17:48.455405+0000","last_active":"2024-09-17T08:18:27.554387+0000","last_peered":"2024-09-17T08:18:27.554387+0000","last_clean":"2024-09-17T08:18:27.554387+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:18:27.554387+0000","last_undegraded":"2024-09-17T08:18:27.554387+0000","last_fullsized":"2024-09-17T08:18:27.554387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.455290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:06.494744+0000","scrub_duration":0.0015673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":188,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555103+0000","last_change":"2024-09-17T08:18:25.340677+0000","last_active":"2024-09-17T08:18:27.555103+0000","last_peered":"2024-09-17T08:18:27.555103+0000","last_clean":"2024-09-17T08:18:27.555103+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:18:27.555103+0000","last_undegraded":"2024-09-17T08:18:27.555103+0000","last_fullsized":"2024-09-17T08:18:27.555103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:25.340536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:25.340536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:39.895667+0000","scrub_duration":0.0092921429999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"202'10787","reported_seq":10972,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.945063+0000","last_change":"2024-09-17T08:18:28.552653+0000","last_active":"2024-09-17T08:18:32.945063+0000","last_peered":"2024-09-17T08:18:32.945063+0000","last_clean":"2024-09-17T08:18:32.945063+0000","last_became_active":"2024-09-17T08:16:16.402701+0000","last_became_peered":"2024-09-17T08:16:16.402701+0000","last_unstale":"2024-09-17T08:18:32.945063+0000","last_undegraded":"2024-09-17T08:18:32.945063+0000","last_fullsized":"2024-09-17T08:18:32.945063+0000","mapping_epoch":165,"log_start":"187'5700","ondisk_log_start":"187'5700","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'10469","last_scrub_stamp":"2024-09-17T08:18:28.552437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:28.552437+0000","objects_scrubbed":6413,"log_size":5087,"log_dups_size":0,"ondisk_log_size":5087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:41.096952+0000","scrub_duration":2.2029995480000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395247616,"num_objects":6031,"num_object_clones":0,"num_object_copies":18093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10787,"num_write_kb":538176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"72'13","reported_seq":85,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547755+0000","last_change":"2024-09-17T08:16:33.206426+0000","last_active":"2024-09-17T08:18:27.547755+0000","last_peered":"2024-09-17T08:18:27.547755+0000","last_clean":"2024-09-17T08:18:27.547755+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:18:27.547755+0000","last_undegraded":"2024-09-17T08:18:27.547755+0000","last_fullsized":"2024-09-17T08:18:27.547755+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'13","last_scrub_stamp":"2024-09-17T08:16:33.206032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.206032+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0092541819999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":271,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.584755+0000","last_change":"2024-09-17T08:18:13.858555+0000","last_active":"2024-09-17T08:18:31.584755+0000","last_peered":"2024-09-17T08:18:31.584755+0000","last_clean":"2024-09-17T08:18:31.584755+0000","last_became_active":"2024-09-17T08:15:29.370817+0000","last_became_peered":"2024-09-17T08:15:29.370817+0000","last_unstale":"2024-09-17T08:18:31.584755+0000","last_undegraded":"2024-09-17T08:18:31.584755+0000","last_fullsized":"2024-09-17T08:18:31.584755+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:18:13.858423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:13.858423+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:43.221634+0000","scrub_duration":0.023005457,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":189,"num_read_kb":184,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545206+0000","last_change":"2024-09-17T08:17:48.327422+0000","last_active":"2024-09-17T08:18:27.545206+0000","last_peered":"2024-09-17T08:18:27.545206+0000","last_clean":"2024-09-17T08:18:27.545206+0000","last_became_active":"2024-09-17T08:09:54.504609+0000","last_became_peered":"2024-09-17T08:09:54.504609+0000","last_unstale":"2024-09-17T08:18:27.545206+0000","last_undegraded":"2024-09-17T08:18:27.545206+0000","last_fullsized":"2024-09-17T08:18:27.545206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.327064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.327064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.338047+0000","scrub_duration":0.0096668850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":201,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.637163+0000","last_change":"2024-09-17T08:18:32.637669+0000","last_active":"2024-09-17T08:18:32.637163+0000","last_peered":"2024-09-17T08:18:32.637163+0000","last_clean":"2024-09-17T08:18:32.637163+0000","last_became_active":"2024-09-17T08:09:58.513535+0000","last_became_peered":"2024-09-17T08:09:58.513535+0000","last_unstale":"2024-09-17T08:18:32.637163+0000","last_undegraded":"2024-09-17T08:18:32.637163+0000","last_fullsized":"2024-09-17T08:18:32.637163+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:18:32.635696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:36.311568+0000","scrub_duration":0.011012314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":190,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.600651+0000","last_change":"2024-09-17T08:16:44.868243+0000","last_active":"2024-09-17T08:18:31.600651+0000","last_peered":"2024-09-17T08:18:31.600651+0000","last_clean":"2024-09-17T08:18:31.600651+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:18:31.600651+0000","last_undegraded":"2024-09-17T08:18:31.600651+0000","last_fullsized":"2024-09-17T08:18:31.600651+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.868012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.868012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025635359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"202'11117","reported_seq":11299,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.765937+0000","last_change":"2024-09-17T08:18:32.286150+0000","last_active":"2024-09-17T08:18:32.765937+0000","last_peered":"2024-09-17T08:18:32.765937+0000","last_clean":"2024-09-17T08:18:32.765937+0000","last_became_active":"2024-09-17T08:16:16.647832+0000","last_became_peered":"2024-09-17T08:16:16.647832+0000","last_unstale":"2024-09-17T08:18:32.765937+0000","last_undegraded":"2024-09-17T08:18:32.765937+0000","last_fullsized":"2024-09-17T08:18:32.765937+0000","mapping_epoch":165,"log_start":"188'6000","ondisk_log_start":"188'6000","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'11082","last_scrub_stamp":"2024-09-17T08:18:32.285954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:32.285954+0000","objects_scrubbed":6165,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:43.568585+0000","scrub_duration":2.3142667160000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396165120,"num_objects":6045,"num_object_clones":0,"num_object_copies":18135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11117,"num_write_kb":549184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"84'18","reported_seq":262,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.557592+0000","last_change":"2024-09-17T08:17:02.332268+0000","last_active":"2024-09-17T08:18:27.557592+0000","last_peered":"2024-09-17T08:18:27.557592+0000","last_clean":"2024-09-17T08:18:27.557592+0000","last_became_active":"2024-09-17T08:15:29.379763+0000","last_became_peered":"2024-09-17T08:15:29.379763+0000","last_unstale":"2024-09-17T08:18:27.557592+0000","last_undegraded":"2024-09-17T08:18:27.557592+0000","last_fullsized":"2024-09-17T08:18:27.557592+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-17T08:17:02.331862+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:17:02.331862+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094874130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":202,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.556929+0000","last_change":"2024-09-17T08:16:50.181218+0000","last_active":"2024-09-17T08:18:27.556929+0000","last_peered":"2024-09-17T08:18:27.556929+0000","last_clean":"2024-09-17T08:18:27.556929+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:18:27.556929+0000","last_undegraded":"2024-09-17T08:18:27.556929+0000","last_fullsized":"2024-09-17T08:18:27.556929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:16:50.181067+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098447959999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":214,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596295+0000","last_change":"2024-09-17T08:18:29.909328+0000","last_active":"2024-09-17T08:18:31.596295+0000","last_peered":"2024-09-17T08:18:31.596295+0000","last_clean":"2024-09-17T08:18:31.596295+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:18:31.596295+0000","last_undegraded":"2024-09-17T08:18:31.596295+0000","last_fullsized":"2024-09-17T08:18:31.596295+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:29.909207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:18:29.909207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.459943+0000","scrub_duration":0.0040057720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547923+0000","last_change":"2024-09-17T08:18:15.767066+0000","last_active":"2024-09-17T08:18:27.547923+0000","last_peered":"2024-09-17T08:18:27.547923+0000","last_clean":"2024-09-17T08:18:27.547923+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:18:27.547923+0000","last_undegraded":"2024-09-17T08:18:27.547923+0000","last_fullsized":"2024-09-17T08:18:27.547923+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:15.766540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:15.766540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:31.285115+0000","scrub_duration":0.0099971980000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547190+0000","last_change":"2024-09-17T08:16:42.309390+0000","last_active":"2024-09-17T08:18:27.547190+0000","last_peered":"2024-09-17T08:18:27.547190+0000","last_clean":"2024-09-17T08:18:27.547190+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:18:27.547190+0000","last_undegraded":"2024-09-17T08:18:27.547190+0000","last_fullsized":"2024-09-17T08:18:27.547190+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:42.309103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:42.309103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057864980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"202'10944","reported_seq":11132,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:33.199039+0000","last_change":"2024-09-17T08:18:26.200261+0000","last_active":"2024-09-17T08:18:33.199039+0000","last_peered":"2024-09-17T08:18:33.199039+0000","last_clean":"2024-09-17T08:18:33.199039+0000","last_became_active":"2024-09-17T08:16:16.647748+0000","last_became_peered":"2024-09-17T08:16:16.647748+0000","last_unstale":"2024-09-17T08:18:33.199039+0000","last_undegraded":"2024-09-17T08:18:33.199039+0000","last_fullsized":"2024-09-17T08:18:33.199039+0000","mapping_epoch":165,"log_start":"184'5200","ondisk_log_start":"184'5200","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10474","last_scrub_stamp":"2024-09-17T08:18:26.200067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:26.200067+0000","objects_scrubbed":6467,"log_size":5744,"log_dups_size":0,"ondisk_log_size":5744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.122229+0000","scrub_duration":2.5348516500000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387973120,"num_objects":5920,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":539648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"139'21","reported_seq":176,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548971+0000","last_change":"2024-09-17T08:17:37.634899+0000","last_active":"2024-09-17T08:18:27.548971+0000","last_peered":"2024-09-17T08:18:27.548971+0000","last_clean":"2024-09-17T08:18:27.548971+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:18:27.548971+0000","last_undegraded":"2024-09-17T08:18:27.548971+0000","last_fullsized":"2024-09-17T08:18:27.548971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'21","last_scrub_stamp":"2024-09-17T08:17:37.634598+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.634598+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.942132+0000","scrub_duration":0.0074777940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":191,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550323+0000","last_change":"2024-09-17T08:17:36.901495+0000","last_active":"2024-09-17T08:18:27.550323+0000","last_peered":"2024-09-17T08:18:27.550323+0000","last_clean":"2024-09-17T08:18:27.550323+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:18:27.550323+0000","last_undegraded":"2024-09-17T08:18:27.550323+0000","last_fullsized":"2024-09-17T08:18:27.550323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:17:36.901360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.901360+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:41.027333+0000","scrub_duration":0.002432104,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545124+0000","last_change":"2024-09-17T08:18:19.939002+0000","last_active":"2024-09-17T08:18:27.545124+0000","last_peered":"2024-09-17T08:18:27.545124+0000","last_clean":"2024-09-17T08:18:27.545124+0000","last_became_active":"2024-09-17T08:14:58.717890+0000","last_became_peered":"2024-09-17T08:14:58.717890+0000","last_unstale":"2024-09-17T08:18:27.545124+0000","last_undegraded":"2024-09-17T08:18:27.545124+0000","last_fullsized":"2024-09-17T08:18:27.545124+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:19.938865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:18:19.938865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.705782+0000","scrub_duration":0.0064574200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":457,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596755+0000","last_change":"2024-09-17T08:17:47.212851+0000","last_active":"2024-09-17T08:18:31.596755+0000","last_peered":"2024-09-17T08:18:31.596755+0000","last_clean":"2024-09-17T08:18:31.596755+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:18:31.596755+0000","last_undegraded":"2024-09-17T08:18:31.596755+0000","last_fullsized":"2024-09-17T08:18:31.596755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.209749+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:11.287491+0000","scrub_duration":0.014148275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":192,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596589+0000","last_change":"2024-09-17T08:17:48.223952+0000","last_active":"2024-09-17T08:18:31.596589+0000","last_peered":"2024-09-17T08:18:31.596589+0000","last_clean":"2024-09-17T08:18:31.596589+0000","last_became_active":"2024-09-17T08:09:58.517477+0000","last_became_peered":"2024-09-17T08:09:58.517477+0000","last_unstale":"2024-09-17T08:18:31.596589+0000","last_undegraded":"2024-09-17T08:18:31.596589+0000","last_fullsized":"2024-09-17T08:18:31.596589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.223800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.223800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:17.257739+0000","scrub_duration":0.003735787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":82,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.559576+0000","last_change":"2024-09-17T08:16:33.884267+0000","last_active":"2024-09-17T08:18:27.559576+0000","last_peered":"2024-09-17T08:18:27.559576+0000","last_clean":"2024-09-17T08:18:27.559576+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:18:27.559576+0000","last_undegraded":"2024-09-17T08:18:27.559576+0000","last_fullsized":"2024-09-17T08:18:27.559576+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:33.884139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:33.884139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"202'10836","reported_seq":11030,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:33.198591+0000","last_change":"2024-09-17T08:18:21.273153+0000","last_active":"2024-09-17T08:18:33.198591+0000","last_peered":"2024-09-17T08:18:33.198591+0000","last_clean":"2024-09-17T08:18:33.198591+0000","last_became_active":"2024-09-17T08:16:16.646368+0000","last_became_peered":"2024-09-17T08:16:16.646368+0000","last_unstale":"2024-09-17T08:18:33.198591+0000","last_undegraded":"2024-09-17T08:18:33.198591+0000","last_fullsized":"2024-09-17T08:18:33.198591+0000","mapping_epoch":165,"log_start":"184'5100","ondisk_log_start":"184'5100","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'10070","last_scrub_stamp":"2024-09-17T08:18:21.272913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:21.272913+0000","objects_scrubbed":6830,"log_size":5736,"log_dups_size":0,"ondisk_log_size":5736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:21.750217+0000","scrub_duration":2.6393852550000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390463488,"num_objects":5958,"num_object_clones":0,"num_object_copies":17874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10836,"num_write_kb":537408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"72'19","reported_seq":104,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546492+0000","last_change":"2024-09-17T08:17:49.689057+0000","last_active":"2024-09-17T08:18:27.546492+0000","last_peered":"2024-09-17T08:18:27.546492+0000","last_clean":"2024-09-17T08:18:27.546492+0000","last_became_active":"2024-09-17T08:15:29.360821+0000","last_became_peered":"2024-09-17T08:15:29.360821+0000","last_unstale":"2024-09-17T08:18:27.546492+0000","last_undegraded":"2024-09-17T08:18:27.546492+0000","last_fullsized":"2024-09-17T08:18:27.546492+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'19","last_scrub_stamp":"2024-09-17T08:17:49.688552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:49.688552+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:10.450394+0000","scrub_duration":0.0066490250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":194,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547048+0000","last_change":"2024-09-17T08:17:40.401393+0000","last_active":"2024-09-17T08:18:27.547048+0000","last_peered":"2024-09-17T08:18:27.547048+0000","last_clean":"2024-09-17T08:18:27.547048+0000","last_became_active":"2024-09-17T08:09:54.501572+0000","last_became_peered":"2024-09-17T08:09:54.501572+0000","last_unstale":"2024-09-17T08:18:27.547048+0000","last_undegraded":"2024-09-17T08:18:27.547048+0000","last_fullsized":"2024-09-17T08:18:27.547048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:40.401262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:40.401262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:53.178795+0000","scrub_duration":0.0066464740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":201,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545055+0000","last_change":"2024-09-17T08:17:36.372735+0000","last_active":"2024-09-17T08:18:27.545055+0000","last_peered":"2024-09-17T08:18:27.545055+0000","last_clean":"2024-09-17T08:18:27.545055+0000","last_became_active":"2024-09-17T08:09:58.510367+0000","last_became_peered":"2024-09-17T08:09:58.510367+0000","last_unstale":"2024-09-17T08:18:27.545055+0000","last_undegraded":"2024-09-17T08:18:27.545055+0000","last_fullsized":"2024-09-17T08:18:27.545055+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.372603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.372603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.970558+0000","scrub_duration":0.010983404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":189,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596336+0000","last_change":"2024-09-17T08:17:34.135923+0000","last_active":"2024-09-17T08:18:31.596336+0000","last_peered":"2024-09-17T08:18:31.596336+0000","last_clean":"2024-09-17T08:18:31.596336+0000","last_became_active":"2024-09-17T08:10:00.525146+0000","last_became_peered":"2024-09-17T08:10:00.525146+0000","last_unstale":"2024-09-17T08:18:31.596336+0000","last_undegraded":"2024-09-17T08:18:31.596336+0000","last_fullsized":"2024-09-17T08:18:31.596336+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:34.135505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:34.135505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.113488+0000","scrub_duration":0.01331624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"202'10961","reported_seq":11151,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.765289+0000","last_change":"2024-09-17T08:18:26.522431+0000","last_active":"2024-09-17T08:18:32.765289+0000","last_peered":"2024-09-17T08:18:32.765289+0000","last_clean":"2024-09-17T08:18:32.765289+0000","last_became_active":"2024-09-17T08:16:16.650307+0000","last_became_peered":"2024-09-17T08:16:16.650307+0000","last_unstale":"2024-09-17T08:18:32.765289+0000","last_undegraded":"2024-09-17T08:18:32.765289+0000","last_fullsized":"2024-09-17T08:18:32.765289+0000","mapping_epoch":165,"log_start":"187'5800","ondisk_log_start":"187'5800","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'10550","last_scrub_stamp":"2024-09-17T08:18:26.522270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:18:26.522270+0000","objects_scrubbed":6574,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:56.092354+0000","scrub_duration":2.7036431570000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398262272,"num_objects":6077,"num_object_clones":0,"num_object_copies":18231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10961,"num_write_kb":545216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"72'9","reported_seq":246,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554306+0000","last_change":"2024-09-17T08:17:47.420671+0000","last_active":"2024-09-17T08:18:27.554306+0000","last_peered":"2024-09-17T08:18:27.554306+0000","last_clean":"2024-09-17T08:18:27.554306+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:18:27.554306+0000","last_undegraded":"2024-09-17T08:18:27.554306+0000","last_fullsized":"2024-09-17T08:18:27.554306+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:47.420536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.420536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.300439+0000","scrub_duration":0.010761165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546506+0000","last_change":"2024-09-17T08:17:45.666143+0000","last_active":"2024-09-17T08:18:27.546506+0000","last_peered":"2024-09-17T08:18:27.546506+0000","last_clean":"2024-09-17T08:18:27.546506+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:18:27.546506+0000","last_undegraded":"2024-09-17T08:18:27.546506+0000","last_fullsized":"2024-09-17T08:18:27.546506+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:45.666044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:17:45.666044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:56.542414+0000","scrub_duration":0.0048391390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":190,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547724+0000","last_change":"2024-09-17T08:17:31.297792+0000","last_active":"2024-09-17T08:18:27.547724+0000","last_peered":"2024-09-17T08:18:27.547724+0000","last_clean":"2024-09-17T08:18:27.547724+0000","last_became_active":"2024-09-17T08:09:58.511627+0000","last_became_peered":"2024-09-17T08:09:58.511627+0000","last_unstale":"2024-09-17T08:18:27.547724+0000","last_undegraded":"2024-09-17T08:18:27.547724+0000","last_fullsized":"2024-09-17T08:18:27.547724+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.297237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.297237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.099318+0000","scrub_duration":0.0044744609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":188,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.556728+0000","last_change":"2024-09-17T08:16:54.185896+0000","last_active":"2024-09-17T08:18:27.556728+0000","last_peered":"2024-09-17T08:18:27.556728+0000","last_clean":"2024-09-17T08:18:27.556728+0000","last_became_active":"2024-09-17T08:10:00.520956+0000","last_became_peered":"2024-09-17T08:10:00.520956+0000","last_unstale":"2024-09-17T08:18:27.556728+0000","last_undegraded":"2024-09-17T08:18:27.556728+0000","last_fullsized":"2024-09-17T08:18:27.556728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:54.185311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:54.185311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041021820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"202'11003","reported_seq":11050,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.362563+0000","last_change":"2024-09-17T08:16:16.649082+0000","last_active":"2024-09-17T08:18:32.362563+0000","last_peered":"2024-09-17T08:18:32.362563+0000","last_clean":"2024-09-17T08:18:32.362563+0000","last_became_active":"2024-09-17T08:16:16.648709+0000","last_became_peered":"2024-09-17T08:16:16.648709+0000","last_unstale":"2024-09-17T08:18:32.362563+0000","last_undegraded":"2024-09-17T08:18:32.362563+0000","last_fullsized":"2024-09-17T08:18:32.362563+0000","mapping_epoch":165,"log_start":"187'5700","ondisk_log_start":"187'5700","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397606912,"num_objects":6067,"num_object_clones":0,"num_object_copies":18201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11003,"num_write_kb":546240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"140'22","reported_seq":220,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596173+0000","last_change":"2024-09-17T08:16:57.000722+0000","last_active":"2024-09-17T08:18:31.596173+0000","last_peered":"2024-09-17T08:18:31.596173+0000","last_clean":"2024-09-17T08:18:31.596173+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:18:31.596173+0000","last_undegraded":"2024-09-17T08:18:31.596173+0000","last_fullsized":"2024-09-17T08:18:31.596173+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'22","last_scrub_stamp":"2024-09-17T08:16:57.000450+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:14:50.155143+0000","last_clean_scrub_stamp":"2024-09-17T08:16:57.000450+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060780779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":205,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.556442+0000","last_change":"2024-09-17T08:18:23.831226+0000","last_active":"2024-09-17T08:18:27.556442+0000","last_peered":"2024-09-17T08:18:27.556442+0000","last_clean":"2024-09-17T08:18:27.556442+0000","last_became_active":"2024-09-17T08:09:54.498239+0000","last_became_peered":"2024-09-17T08:09:54.498239+0000","last_unstale":"2024-09-17T08:18:27.556442+0000","last_undegraded":"2024-09-17T08:18:27.556442+0000","last_fullsized":"2024-09-17T08:18:27.556442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:23.831078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:23.831078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.006876+0000","scrub_duration":0.026694872000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"72'8","reported_seq":303,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.929425+0000","last_change":"2024-09-17T08:18:26.835132+0000","last_active":"2024-09-17T08:18:32.929425+0000","last_peered":"2024-09-17T08:18:32.929425+0000","last_clean":"2024-09-17T08:18:32.929425+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:18:32.929425+0000","last_undegraded":"2024-09-17T08:18:32.929425+0000","last_fullsized":"2024-09-17T08:18:32.929425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'8","last_scrub_stamp":"2024-09-17T08:18:26.834991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:26.834991+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:49.265674+0000","scrub_duration":0.0090097040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":93,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555705+0000","last_change":"2024-09-17T08:16:58.243174+0000","last_active":"2024-09-17T08:18:27.555705+0000","last_peered":"2024-09-17T08:18:27.555705+0000","last_clean":"2024-09-17T08:18:27.555705+0000","last_became_active":"2024-09-17T08:14:58.717856+0000","last_became_peered":"2024-09-17T08:14:58.717856+0000","last_unstale":"2024-09-17T08:18:27.555705+0000","last_undegraded":"2024-09-17T08:18:27.555705+0000","last_fullsized":"2024-09-17T08:18:27.555705+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.242877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.242877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056167969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"202'10859","reported_seq":10906,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:32.364173+0000","last_change":"2024-09-17T08:16:16.647722+0000","last_active":"2024-09-17T08:18:32.364173+0000","last_peered":"2024-09-17T08:18:32.364173+0000","last_clean":"2024-09-17T08:18:32.364173+0000","last_became_active":"2024-09-17T08:16:16.647324+0000","last_became_peered":"2024-09-17T08:16:16.647324+0000","last_unstale":"2024-09-17T08:18:32.364173+0000","last_undegraded":"2024-09-17T08:18:32.364173+0000","last_fullsized":"2024-09-17T08:18:32.364173+0000","mapping_epoch":165,"log_start":"186'5500","ondisk_log_start":"186'5500","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:15.380958+0000","last_clean_scrub_stamp":"2024-09-17T08:16:15.380958+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389742592,"num_objects":5947,"num_object_clones":0,"num_object_copies":17841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10859,"num_write_kb":537792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"72'12","reported_seq":257,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547577+0000","last_change":"2024-09-17T08:17:00.600357+0000","last_active":"2024-09-17T08:18:27.547577+0000","last_peered":"2024-09-17T08:18:27.547577+0000","last_clean":"2024-09-17T08:18:27.547577+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:18:27.547577+0000","last_undegraded":"2024-09-17T08:18:27.547577+0000","last_fullsized":"2024-09-17T08:18:27.547577+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:17:00.600225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.600225+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071635179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":201,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548024+0000","last_change":"2024-09-17T08:16:39.838818+0000","last_active":"2024-09-17T08:18:27.548024+0000","last_peered":"2024-09-17T08:18:27.548024+0000","last_clean":"2024-09-17T08:18:27.548024+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:18:27.548024+0000","last_undegraded":"2024-09-17T08:18:27.548024+0000","last_fullsized":"2024-09-17T08:18:27.548024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:16:39.838667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.838667+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050598739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":219,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550648+0000","last_change":"2024-09-17T08:17:43.435620+0000","last_active":"2024-09-17T08:18:27.550648+0000","last_peered":"2024-09-17T08:18:27.550648+0000","last_clean":"2024-09-17T08:18:27.550648+0000","last_became_active":"2024-09-17T08:15:29.374092+0000","last_became_peered":"2024-09-17T08:15:29.374092+0000","last_unstale":"2024-09-17T08:18:27.550648+0000","last_undegraded":"2024-09-17T08:18:27.550648+0000","last_fullsized":"2024-09-17T08:18:27.550648+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:17:43.435490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.510920+0000","scrub_duration":0.0037096490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":25,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.544912+0000","last_change":"2024-09-17T08:18:17.905206+0000","last_active":"2024-09-17T08:18:27.544912+0000","last_peered":"2024-09-17T08:18:27.544912+0000","last_clean":"2024-09-17T08:18:27.544912+0000","last_became_active":"2024-09-17T08:18:16.477160+0000","last_became_peered":"2024-09-17T08:18:16.477160+0000","last_unstale":"2024-09-17T08:18:27.544912+0000","last_undegraded":"2024-09-17T08:18:27.544912+0000","last_fullsized":"2024-09-17T08:18:27.544912+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:17.905078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:17.905078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:37.202342+0000","scrub_duration":0.002356089,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"85'13","reported_seq":212,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.557516+0000","last_change":"2024-09-17T08:18:25.674227+0000","last_active":"2024-09-17T08:18:27.557516+0000","last_peered":"2024-09-17T08:18:27.557516+0000","last_clean":"2024-09-17T08:18:27.557516+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:18:27.557516+0000","last_undegraded":"2024-09-17T08:18:27.557516+0000","last_fullsized":"2024-09-17T08:18:27.557516+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'13","last_scrub_stamp":"2024-09-17T08:18:25.674098+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:18:25.674098+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.172981+0000","scrub_duration":0.01064264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":223,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550767+0000","last_change":"2024-09-17T08:18:22.847651+0000","last_active":"2024-09-17T08:18:27.550767+0000","last_peered":"2024-09-17T08:18:27.550767+0000","last_clean":"2024-09-17T08:18:27.550767+0000","last_became_active":"2024-09-17T08:15:29.373114+0000","last_became_peered":"2024-09-17T08:15:29.373114+0000","last_unstale":"2024-09-17T08:18:27.550767+0000","last_undegraded":"2024-09-17T08:18:27.550767+0000","last_fullsized":"2024-09-17T08:18:27.550767+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:22.847486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:22.847486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:37.056330+0000","scrub_duration":0.0040275470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":197,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.559958+0000","last_change":"2024-09-17T08:18:17.196320+0000","last_active":"2024-09-17T08:18:27.559958+0000","last_peered":"2024-09-17T08:18:27.559958+0000","last_clean":"2024-09-17T08:18:27.559958+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:18:27.559958+0000","last_undegraded":"2024-09-17T08:18:27.559958+0000","last_fullsized":"2024-09-17T08:18:27.559958+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:17.196058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:18:17.196058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:24.847097+0000","scrub_duration":0.012365815,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":199,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546371+0000","last_change":"2024-09-17T08:18:15.013427+0000","last_active":"2024-09-17T08:18:27.546371+0000","last_peered":"2024-09-17T08:18:27.546371+0000","last_clean":"2024-09-17T08:18:27.546371+0000","last_became_active":"2024-09-17T08:10:00.522029+0000","last_became_peered":"2024-09-17T08:10:00.522029+0000","last_unstale":"2024-09-17T08:18:27.546371+0000","last_undegraded":"2024-09-17T08:18:27.546371+0000","last_fullsized":"2024-09-17T08:18:27.546371+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:15.013236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:15.013236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:31.413848+0000","scrub_duration":0.0054208879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":152,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547726+0000","last_change":"2024-09-17T08:16:29.839526+0000","last_active":"2024-09-17T08:18:27.547726+0000","last_peered":"2024-09-17T08:18:27.547726+0000","last_clean":"2024-09-17T08:18:27.547726+0000","last_became_active":"2024-09-17T08:15:29.358382+0000","last_became_peered":"2024-09-17T08:15:29.358382+0000","last_unstale":"2024-09-17T08:18:27.547726+0000","last_undegraded":"2024-09-17T08:18:27.547726+0000","last_fullsized":"2024-09-17T08:18:27.547726+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:29.839416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.839416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002703015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"72'6","reported_seq":224,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.560079+0000","last_change":"2024-09-17T08:16:30.802344+0000","last_active":"2024-09-17T08:18:27.560079+0000","last_peered":"2024-09-17T08:18:27.560079+0000","last_clean":"2024-09-17T08:18:27.560079+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:18:27.560079+0000","last_undegraded":"2024-09-17T08:18:27.560079+0000","last_fullsized":"2024-09-17T08:18:27.560079+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'6","last_scrub_stamp":"2024-09-17T08:16:30.800480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:30.800480+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":100,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555033+0000","last_change":"2024-09-17T08:17:50.171756+0000","last_active":"2024-09-17T08:18:27.555033+0000","last_peered":"2024-09-17T08:18:27.555033+0000","last_clean":"2024-09-17T08:18:27.555033+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:18:27.555033+0000","last_undegraded":"2024-09-17T08:18:27.555033+0000","last_fullsized":"2024-09-17T08:18:27.555033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:50.171624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:50.171624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:53.779301+0000","scrub_duration":0.01783852,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":200,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.600566+0000","last_change":"2024-09-17T08:18:30.899319+0000","last_active":"2024-09-17T08:18:31.600566+0000","last_peered":"2024-09-17T08:18:31.600566+0000","last_clean":"2024-09-17T08:18:31.600566+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:18:31.600566+0000","last_undegraded":"2024-09-17T08:18:31.600566+0000","last_fullsized":"2024-09-17T08:18:31.600566+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:30.899234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:30.899234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:50.657347+0000","scrub_duration":0.00108336,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"72'10","reported_seq":238,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.559665+0000","last_change":"2024-09-17T08:16:45.629362+0000","last_active":"2024-09-17T08:18:27.559665+0000","last_peered":"2024-09-17T08:18:27.559665+0000","last_clean":"2024-09-17T08:18:27.559665+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:18:27.559665+0000","last_undegraded":"2024-09-17T08:18:27.559665+0000","last_fullsized":"2024-09-17T08:18:27.559665+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:45.626489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:45.626489+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060751379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":125,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550202+0000","last_change":"2024-09-17T08:17:47.134565+0000","last_active":"2024-09-17T08:18:27.550202+0000","last_peered":"2024-09-17T08:18:27.550202+0000","last_clean":"2024-09-17T08:18:27.550202+0000","last_became_active":"2024-09-17T08:14:58.720104+0000","last_became_peered":"2024-09-17T08:14:58.720104+0000","last_unstale":"2024-09-17T08:18:27.550202+0000","last_undegraded":"2024-09-17T08:18:27.550202+0000","last_fullsized":"2024-09-17T08:18:27.550202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:47.134436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:47.134436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.627771+0000","scrub_duration":0.0068223679999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":126,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:28.926832+0000","last_change":"2024-09-17T08:18:28.926832+0000","last_active":"2024-09-17T08:18:28.926832+0000","last_peered":"2024-09-17T08:18:28.926832+0000","last_clean":"2024-09-17T08:18:28.926832+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:18:28.926832+0000","last_undegraded":"2024-09-17T08:18:28.926832+0000","last_fullsized":"2024-09-17T08:18:28.926832+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:18:28.926645+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:58.521496+0000","scrub_duration":0.0048477939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":188,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.557148+0000","last_change":"2024-09-17T08:17:33.406086+0000","last_active":"2024-09-17T08:18:27.557148+0000","last_peered":"2024-09-17T08:18:27.557148+0000","last_clean":"2024-09-17T08:18:27.557148+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:18:27.557148+0000","last_undegraded":"2024-09-17T08:18:27.557148+0000","last_fullsized":"2024-09-17T08:18:27.557148+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:33.405939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.405939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.635332+0000","scrub_duration":0.0072126600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"177'13","reported_seq":127,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.600770+0000","last_change":"2024-09-17T08:17:00.945597+0000","last_active":"2024-09-17T08:18:31.600770+0000","last_peered":"2024-09-17T08:18:31.600770+0000","last_clean":"2024-09-17T08:18:31.600770+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:18:31.600770+0000","last_undegraded":"2024-09-17T08:18:31.600770+0000","last_fullsized":"2024-09-17T08:18:31.600770+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"177'13","last_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:17:00.945463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016379540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":194,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.554403+0000","last_change":"2024-09-17T08:16:46.264262+0000","last_active":"2024-09-17T08:18:27.554403+0000","last_peered":"2024-09-17T08:18:27.554403+0000","last_clean":"2024-09-17T08:18:27.554403+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:18:27.554403+0000","last_undegraded":"2024-09-17T08:18:27.554403+0000","last_fullsized":"2024-09-17T08:18:27.554403+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:46.264138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:46.264138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046054340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"88'9","reported_seq":292,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.587613+0000","last_change":"2024-09-17T08:17:35.880572+0000","last_active":"2024-09-17T08:18:31.587613+0000","last_peered":"2024-09-17T08:18:31.587613+0000","last_clean":"2024-09-17T08:18:31.587613+0000","last_became_active":"2024-09-17T08:09:58.513860+0000","last_became_peered":"2024-09-17T08:09:58.513860+0000","last_unstale":"2024-09-17T08:18:31.587613+0000","last_undegraded":"2024-09-17T08:18:31.587613+0000","last_fullsized":"2024-09-17T08:18:31.587613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'9","last_scrub_stamp":"2024-09-17T08:17:35.880439+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:17:35.880439+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.829845+0000","scrub_duration":0.0069173250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":177,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547721+0000","last_change":"2024-09-17T08:16:44.899582+0000","last_active":"2024-09-17T08:18:27.547721+0000","last_peered":"2024-09-17T08:18:27.547721+0000","last_clean":"2024-09-17T08:18:27.547721+0000","last_became_active":"2024-09-17T08:10:00.525745+0000","last_became_peered":"2024-09-17T08:10:00.525745+0000","last_unstale":"2024-09-17T08:18:27.547721+0000","last_undegraded":"2024-09-17T08:18:27.547721+0000","last_fullsized":"2024-09-17T08:18:27.547721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:44.899473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:44.899473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048647389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"72'9","reported_seq":83,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.545819+0000","last_change":"2024-09-17T08:17:31.426355+0000","last_active":"2024-09-17T08:18:27.545819+0000","last_peered":"2024-09-17T08:18:27.545819+0000","last_clean":"2024-09-17T08:18:27.545819+0000","last_became_active":"2024-09-17T08:15:29.364047+0000","last_became_peered":"2024-09-17T08:15:29.364047+0000","last_unstale":"2024-09-17T08:18:27.545819+0000","last_undegraded":"2024-09-17T08:18:27.545819+0000","last_fullsized":"2024-09-17T08:18:27.545819+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'9","last_scrub_stamp":"2024-09-17T08:17:31.426185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.426185+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.778192+0000","scrub_duration":0.028345339000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":145,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.559740+0000","last_change":"2024-09-17T08:18:22.302629+0000","last_active":"2024-09-17T08:18:27.559740+0000","last_peered":"2024-09-17T08:18:27.559740+0000","last_clean":"2024-09-17T08:18:27.559740+0000","last_became_active":"2024-09-17T08:12:46.184269+0000","last_became_peered":"2024-09-17T08:12:46.184269+0000","last_unstale":"2024-09-17T08:18:27.559740+0000","last_undegraded":"2024-09-17T08:18:27.559740+0000","last_fullsized":"2024-09-17T08:18:27.559740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:22.298430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:22.298430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:45.222445+0000","scrub_duration":0.0051706699999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":197,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550795+0000","last_change":"2024-09-17T08:17:32.933012+0000","last_active":"2024-09-17T08:18:27.550795+0000","last_peered":"2024-09-17T08:18:27.550795+0000","last_clean":"2024-09-17T08:18:27.550795+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:18:27.550795+0000","last_undegraded":"2024-09-17T08:18:27.550795+0000","last_fullsized":"2024-09-17T08:18:27.550795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.932889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.932889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:35.988363+0000","scrub_duration":0.017331688000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":206,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.553824+0000","last_change":"2024-09-17T08:17:37.431486+0000","last_active":"2024-09-17T08:18:27.553824+0000","last_peered":"2024-09-17T08:18:27.553824+0000","last_clean":"2024-09-17T08:18:27.553824+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:18:27.553824+0000","last_undegraded":"2024-09-17T08:18:27.553824+0000","last_fullsized":"2024-09-17T08:18:27.553824+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.431366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.431366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.282544+0000","scrub_duration":0.0073929879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"72'11","reported_seq":253,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548329+0000","last_change":"2024-09-17T08:16:59.607476+0000","last_active":"2024-09-17T08:18:27.548329+0000","last_peered":"2024-09-17T08:18:27.548329+0000","last_clean":"2024-09-17T08:18:27.548329+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:18:27.548329+0000","last_undegraded":"2024-09-17T08:18:27.548329+0000","last_fullsized":"2024-09-17T08:18:27.548329+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:16:59.607325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.607325+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083300879999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546844+0000","last_change":"2024-09-17T08:17:01.469617+0000","last_active":"2024-09-17T08:18:27.546844+0000","last_peered":"2024-09-17T08:18:27.546844+0000","last_clean":"2024-09-17T08:18:27.546844+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:18:27.546844+0000","last_undegraded":"2024-09-17T08:18:27.546844+0000","last_fullsized":"2024-09-17T08:18:27.546844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:01.469473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:01.469473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032550880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546102+0000","last_change":"2024-09-17T08:17:32.455271+0000","last_active":"2024-09-17T08:18:27.546102+0000","last_peered":"2024-09-17T08:18:27.546102+0000","last_clean":"2024-09-17T08:18:27.546102+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:18:27.546102+0000","last_undegraded":"2024-09-17T08:18:27.546102+0000","last_fullsized":"2024-09-17T08:18:27.546102+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:32.455128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:32.455128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:00.037261+0000","scrub_duration":0.0091779470000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":177,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.549821+0000","last_change":"2024-09-17T08:16:39.766499+0000","last_active":"2024-09-17T08:18:27.549821+0000","last_peered":"2024-09-17T08:18:27.549821+0000","last_clean":"2024-09-17T08:18:27.549821+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:18:27.549821+0000","last_undegraded":"2024-09-17T08:18:27.549821+0000","last_fullsized":"2024-09-17T08:18:27.549821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:39.766371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:16:39.766371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003380568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"72'4","reported_seq":232,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.551245+0000","last_change":"2024-09-17T08:17:42.964511+0000","last_active":"2024-09-17T08:18:27.551245+0000","last_peered":"2024-09-17T08:18:27.551245+0000","last_clean":"2024-09-17T08:18:27.551245+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:18:27.551245+0000","last_undegraded":"2024-09-17T08:18:27.551245+0000","last_fullsized":"2024-09-17T08:18:27.551245+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'4","last_scrub_stamp":"2024-09-17T08:17:42.964394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:42.964394+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:54.742246+0000","scrub_duration":0.0020163619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":94,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.556372+0000","last_change":"2024-09-17T08:17:37.395358+0000","last_active":"2024-09-17T08:18:27.556372+0000","last_peered":"2024-09-17T08:18:27.556372+0000","last_clean":"2024-09-17T08:18:27.556372+0000","last_became_active":"2024-09-17T08:14:58.715288+0000","last_became_peered":"2024-09-17T08:14:58.715288+0000","last_unstale":"2024-09-17T08:18:27.556372+0000","last_undegraded":"2024-09-17T08:18:27.556372+0000","last_fullsized":"2024-09-17T08:18:27.556372+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:37.394936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:37.394936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:58.208429+0000","scrub_duration":0.0060254940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":146,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547426+0000","last_change":"2024-09-17T08:17:31.626868+0000","last_active":"2024-09-17T08:18:27.547426+0000","last_peered":"2024-09-17T08:18:27.547426+0000","last_clean":"2024-09-17T08:18:27.547426+0000","last_became_active":"2024-09-17T08:12:46.187857+0000","last_became_peered":"2024-09-17T08:12:46.187857+0000","last_unstale":"2024-09-17T08:18:27.547426+0000","last_undegraded":"2024-09-17T08:18:27.547426+0000","last_fullsized":"2024-09-17T08:18:27.547426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:31.626749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:52.132279+0000","last_clean_scrub_stamp":"2024-09-17T08:17:31.626749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:59.963940+0000","scrub_duration":0.002332518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":189,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.595822+0000","last_change":"2024-09-17T08:16:52.066187+0000","last_active":"2024-09-17T08:18:31.595822+0000","last_peered":"2024-09-17T08:18:31.595822+0000","last_clean":"2024-09-17T08:18:31.595822+0000","last_became_active":"2024-09-17T08:10:00.528614+0000","last_became_peered":"2024-09-17T08:10:00.528614+0000","last_unstale":"2024-09-17T08:18:31.595822+0000","last_undegraded":"2024-09-17T08:18:31.595822+0000","last_fullsized":"2024-09-17T08:18:31.595822+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:52.066058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.066058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023799279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"72'11","reported_seq":200,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550022+0000","last_change":"2024-09-17T08:17:33.932414+0000","last_active":"2024-09-17T08:18:27.550022+0000","last_peered":"2024-09-17T08:18:27.550022+0000","last_clean":"2024-09-17T08:18:27.550022+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:18:27.550022+0000","last_undegraded":"2024-09-17T08:18:27.550022+0000","last_fullsized":"2024-09-17T08:18:27.550022+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:33.932279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:33.932279+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:03.370870+0000","scrub_duration":0.0031072140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":131,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.549057+0000","last_change":"2024-09-17T08:18:18.919753+0000","last_active":"2024-09-17T08:18:27.549057+0000","last_peered":"2024-09-17T08:18:27.549057+0000","last_clean":"2024-09-17T08:18:27.549057+0000","last_became_active":"2024-09-17T08:14:58.716813+0000","last_became_peered":"2024-09-17T08:14:58.716813+0000","last_unstale":"2024-09-17T08:18:27.549057+0000","last_undegraded":"2024-09-17T08:18:27.549057+0000","last_fullsized":"2024-09-17T08:18:27.549057+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:18:18.919161+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:18:18.919161+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:26.972260+0000","scrub_duration":0.010030713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":191,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.595687+0000","last_change":"2024-09-17T08:17:39.165283+0000","last_active":"2024-09-17T08:18:31.595687+0000","last_peered":"2024-09-17T08:18:31.595687+0000","last_clean":"2024-09-17T08:18:31.595687+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:18:31.595687+0000","last_undegraded":"2024-09-17T08:18:31.595687+0000","last_fullsized":"2024-09-17T08:18:31.595687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:39.164442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:39.164442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:07.439327+0000","scrub_duration":0.0024479319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":177,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550730+0000","last_change":"2024-09-17T08:18:16.193577+0000","last_active":"2024-09-17T08:18:27.550730+0000","last_peered":"2024-09-17T08:18:27.550730+0000","last_clean":"2024-09-17T08:18:27.550730+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:18:27.550730+0000","last_undegraded":"2024-09-17T08:18:27.550730+0000","last_fullsized":"2024-09-17T08:18:27.550730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:16.193309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:16.193309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:19.369301+0000","scrub_duration":0.0087476380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":191,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.596521+0000","last_change":"2024-09-17T08:18:18.926666+0000","last_active":"2024-09-17T08:18:31.596521+0000","last_peered":"2024-09-17T08:18:31.596521+0000","last_clean":"2024-09-17T08:18:31.596521+0000","last_became_active":"2024-09-17T08:10:00.526716+0000","last_became_peered":"2024-09-17T08:10:00.526716+0000","last_unstale":"2024-09-17T08:18:31.596521+0000","last_undegraded":"2024-09-17T08:18:31.596521+0000","last_fullsized":"2024-09-17T08:18:31.596521+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:18:18.926573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:18.926573+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:28.710728+0000","scrub_duration":0.022659008000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":94,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547323+0000","last_change":"2024-09-17T08:17:46.783050+0000","last_active":"2024-09-17T08:18:27.547323+0000","last_peered":"2024-09-17T08:18:27.547323+0000","last_clean":"2024-09-17T08:18:27.547323+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:18:27.547323+0000","last_undegraded":"2024-09-17T08:18:27.547323+0000","last_fullsized":"2024-09-17T08:18:27.547323+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:46.782915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:17:46.782915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:50.927390+0000","scrub_duration":0.0019436189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"72'10","reported_seq":208,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.557441+0000","last_change":"2024-09-17T08:16:47.247472+0000","last_active":"2024-09-17T08:18:27.557441+0000","last_peered":"2024-09-17T08:18:27.557441+0000","last_clean":"2024-09-17T08:18:27.557441+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:18:27.557441+0000","last_undegraded":"2024-09-17T08:18:27.557441+0000","last_fullsized":"2024-09-17T08:18:27.557441+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'10","last_scrub_stamp":"2024-09-17T08:16:47.247339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:47.247339+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072034789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":226,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.929327+0000","last_change":"2024-09-17T08:16:52.823084+0000","last_active":"2024-09-17T08:18:27.929327+0000","last_peered":"2024-09-17T08:18:27.929327+0000","last_clean":"2024-09-17T08:18:27.929327+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:18:27.929327+0000","last_undegraded":"2024-09-17T08:18:27.929327+0000","last_fullsized":"2024-09-17T08:18:27.929327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:16:52.822979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:16:52.822979+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016070290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":84,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546578+0000","last_change":"2024-09-17T08:17:48.711279+0000","last_active":"2024-09-17T08:18:27.546578+0000","last_peered":"2024-09-17T08:18:27.546578+0000","last_clean":"2024-09-17T08:18:27.546578+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:18:27.546578+0000","last_undegraded":"2024-09-17T08:18:27.546578+0000","last_fullsized":"2024-09-17T08:18:27.546578+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:48.711144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:17:48.711144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.543032+0000","scrub_duration":0.0072007140000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":183,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.550318+0000","last_change":"2024-09-17T08:16:58.658509+0000","last_active":"2024-09-17T08:18:27.550318+0000","last_peered":"2024-09-17T08:18:27.550318+0000","last_clean":"2024-09-17T08:18:27.550318+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:18:27.550318+0000","last_undegraded":"2024-09-17T08:18:27.550318+0000","last_fullsized":"2024-09-17T08:18:27.550318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:58.658382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:16:58.658382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056271569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"72'11","reported_seq":85,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.547676+0000","last_change":"2024-09-17T08:17:38.572088+0000","last_active":"2024-09-17T08:18:27.547676+0000","last_peered":"2024-09-17T08:18:27.547676+0000","last_clean":"2024-09-17T08:18:27.547676+0000","last_became_active":"2024-09-17T08:15:29.361714+0000","last_became_peered":"2024-09-17T08:15:29.361714+0000","last_unstale":"2024-09-17T08:18:27.547676+0000","last_undegraded":"2024-09-17T08:18:27.547676+0000","last_fullsized":"2024-09-17T08:18:27.547676+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'11","last_scrub_stamp":"2024-09-17T08:17:38.571885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:17:38.571885+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:56.028615+0000","scrub_duration":0.0076271409999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":202,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:30.667581+0000","last_change":"2024-09-17T08:18:30.667581+0000","last_active":"2024-09-17T08:18:30.667581+0000","last_peered":"2024-09-17T08:18:30.667581+0000","last_clean":"2024-09-17T08:18:30.667581+0000","last_became_active":"2024-09-17T08:09:58.513724+0000","last_became_peered":"2024-09-17T08:09:58.513724+0000","last_unstale":"2024-09-17T08:18:30.667581+0000","last_undegraded":"2024-09-17T08:18:30.667581+0000","last_fullsized":"2024-09-17T08:18:30.667581+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:30.667384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:18:30.667384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.591509+0000","scrub_duration":0.007770351,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":178,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548527+0000","last_change":"2024-09-17T08:16:37.835578+0000","last_active":"2024-09-17T08:18:27.548527+0000","last_peered":"2024-09-17T08:18:27.548527+0000","last_clean":"2024-09-17T08:18:27.548527+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:18:27.548527+0000","last_undegraded":"2024-09-17T08:18:27.548527+0000","last_fullsized":"2024-09-17T08:18:27.548527+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:37.835368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:16:37.835368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001605574,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":145,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548248+0000","last_change":"2024-09-17T08:17:36.583106+0000","last_active":"2024-09-17T08:18:27.548248+0000","last_peered":"2024-09-17T08:18:27.548248+0000","last_clean":"2024-09-17T08:18:27.548248+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:18:27.548248+0000","last_undegraded":"2024-09-17T08:18:27.548248+0000","last_fullsized":"2024-09-17T08:18:27.548248+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:36.582852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:46.386759+0000","last_clean_scrub_stamp":"2024-09-17T08:17:36.582852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:02.756203+0000","scrub_duration":0.001816591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"87'22","reported_seq":299,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546734+0000","last_change":"2024-09-17T08:16:59.092346+0000","last_active":"2024-09-17T08:18:27.546734+0000","last_peered":"2024-09-17T08:18:27.546734+0000","last_clean":"2024-09-17T08:18:27.546734+0000","last_became_active":"2024-09-17T08:15:29.388191+0000","last_became_peered":"2024-09-17T08:15:29.388191+0000","last_unstale":"2024-09-17T08:18:27.546734+0000","last_undegraded":"2024-09-17T08:18:27.546734+0000","last_fullsized":"2024-09-17T08:18:27.546734+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'22","last_scrub_stamp":"2024-09-17T08:16:59.092016+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:16:59.092016+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088076999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":73,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.546704+0000","last_change":"2024-09-17T08:16:51.343826+0000","last_active":"2024-09-17T08:18:27.546704+0000","last_peered":"2024-09-17T08:18:27.546704+0000","last_clean":"2024-09-17T08:18:27.546704+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:18:27.546704+0000","last_undegraded":"2024-09-17T08:18:27.546704+0000","last_fullsized":"2024-09-17T08:18:27.546704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:16:51.343712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:16:51.343712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010532788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":178,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.559893+0000","last_change":"2024-09-17T08:18:15.193753+0000","last_active":"2024-09-17T08:18:27.559893+0000","last_peered":"2024-09-17T08:18:27.559893+0000","last_clean":"2024-09-17T08:18:27.559893+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:18:27.559893+0000","last_undegraded":"2024-09-17T08:18:27.559893+0000","last_fullsized":"2024-09-17T08:18:27.559893+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:15.193604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:18:15.193604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:35.372610+0000","scrub_duration":0.0027468470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":194,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:31.226848+0000","last_change":"2024-09-17T08:18:31.226848+0000","last_active":"2024-09-17T08:18:31.226848+0000","last_peered":"2024-09-17T08:18:31.226848+0000","last_clean":"2024-09-17T08:18:31.226848+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:18:31.226848+0000","last_undegraded":"2024-09-17T08:18:31.226848+0000","last_fullsized":"2024-09-17T08:18:31.226848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:18:31.226650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:18:31.226650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:51.678639+0000","scrub_duration":0.0088023700000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"72'12","reported_seq":246,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.555499+0000","last_change":"2024-09-17T08:16:29.751857+0000","last_active":"2024-09-17T08:18:27.555499+0000","last_peered":"2024-09-17T08:18:27.555499+0000","last_clean":"2024-09-17T08:18:27.555499+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:18:27.555499+0000","last_undegraded":"2024-09-17T08:18:27.555499+0000","last_fullsized":"2024-09-17T08:18:27.555499+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'12","last_scrub_stamp":"2024-09-17T08:16:29.751693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:16:29.751693+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054470619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":190,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-09-17T08:18:27.548347+0000","last_change":"2024-09-17T08:17:43.726427+0000","last_active":"2024-09-17T08:18:27.548347+0000","last_peered":"2024-09-17T08:18:27.548347+0000","last_clean":"2024-09-17T08:18:27.548347+0000","last_became_active":"2024-09-17T08:09:58.511675+0000","last_became_peered":"2024-09-17T08:09:58.511675+0000","last_unstale":"2024-09-17T08:18:27.548347+0000","last_undegraded":"2024-09-17T08:18:27.548347+0000","last_fullsized":"2024-09-17T08:18:27.548347+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:17:43.726340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:17:43.726340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:11.278692+0000","scrub_duration":0.0012672689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6257246232,"num_objects":95479,"num_object_clones":0,"num_object_copies":286437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174557,"num_write_kb":8641089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18672463872,"data_stored":18672451656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84681,"ondisk_log_size":84681,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":690,"num_read_kb":447,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":451,"ondisk_log_size":451,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":184,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070574,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2088956,"kb_used_data":1833076,"kb_used_omap":1,"kb_used_meta":255870,"kb_avail":91656196,"statfs":{"total":95995035648,"available":93855944704,"internally_reserved":0,"allocated":1877069824,"data_stored":1876757750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":262011339},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.4279999999999999,"15min":0.85999999999999999},"min":{"1min":0.23200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.415,"5min":27.751999999999999,"15min":27.751999999999999},"last":62.018000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.583,"15min":0.92800000000000005},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":5.1589999999999998,"5min":37.954000000000001,"15min":37.954000000000001},"last":61.896999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6990000000000001,"5min":1.012,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.255,"15min":0.23699999999999999},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":62.067},{"interface":"front","average":{"1min":1.401,"5min":0.91100000000000003,"15min":0.68400000000000005},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":5.1340000000000003,"5min":5.1340000000000003,"15min":5.1340000000000003},"last":62.194000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.117,"15min":0.74399999999999999},"min":{"1min":0.33800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.0359999999999996,"5min":5.6210000000000004,"15min":5.6210000000000004},"last":61.966999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.089,"15min":0.76900000000000002},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":5.4550000000000001,"5min":5.4550000000000001,"15min":5.4550000000000001},"last":61.843000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":1.3029999999999999,"15min":0.80700000000000005},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.3420000000000001,"5min":11.298,"15min":11.298},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.343,"5min":1.4430000000000001,"15min":0.90700000000000003},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.391,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.434}]},{"osd":4,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.57599999999999996,"15min":0.49399999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.2249999999999996,"5min":5.2249999999999996,"15min":5.2249999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.52700000000000002,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.78800000000000003,"15min":0.59099999999999997},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":5.367,"5min":16.808,"15min":16.808},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.056,"5min":0.71699999999999997,"15min":0.57799999999999996},"min":{"1min":0.32700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.7400000000000002,"5min":11.842000000000001,"15min":11.842000000000001},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":0.67200000000000004,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.0949999999999998,"5min":5.0949999999999998,"15min":5.0949999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.6659999999999999,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.50600000000000001}]}]},{"osd":6,"up_from":50,"seq":214748364915,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2083432,"kb_used_data":1831648,"kb_used_omap":1,"kb_used_meta":251774,"kb_avail":91661720,"statfs":{"total":95995035648,"available":93861601280,"internally_reserved":0,"allocated":1875607552,"data_stored":1875315194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":257817031},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":3.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.248,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":36.784999999999997,"5min":73.757999999999996,"15min":73.757999999999996},"last":2.5619999999999998},{"interface":"front","average":{"1min":2.7719999999999998,"5min":3.6709999999999998,"15min":1.5489999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":37.073,"5min":73.811000000000007,"15min":73.811000000000007},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.6829999999999998,"5min":1.5509999999999999,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":37.256999999999998,"5min":37.256999999999998,"15min":37.256999999999998},"last":3.4590000000000001},{"interface":"front","average":{"1min":2.9670000000000001,"5min":1.708,"15min":0.92300000000000004},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":37.415999999999997,"5min":37.415999999999997,"15min":37.415999999999997},"last":2.375}]},{"osd":2,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":1.5569999999999999,"15min":0.871},"min":{"1min":0.25700000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.6539999999999999,"5min":68.165000000000006,"15min":68.165000000000006},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.821,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4359999999999999,"5min":67.997,"15min":67.997},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":2.633,"5min":2.6829999999999998,"15min":1.2649999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":39.198999999999998,"5min":77.010999999999996,"15min":77.010999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":2.605,"5min":2.6259999999999999,"15min":1.236},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":36.804000000000002,"5min":76.375,"15min":76.375},"last":1.7549999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.629,"15min":0.45900000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.502},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46700000000000003,"15min":0.40999999999999998},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.63300000000000001,"15min":0.47399999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":4.8029999999999999,"5min":4.8029999999999999,"15min":4.8029999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51000000000000001,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.372}]},{"osd":7,"last update":"Tue Sep 17 08:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.70599999999999996,"15min":0.56699999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26500000000000001},"max":{"1min":4.774,"5min":4.774,"15min":4.774},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.55700000000000005,"15min":0.52700000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":0.40899999999999997}]}]},{"osd":5,"up_from":41,"seq":176093659255,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3371412,"kb_used_data":3109772,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":90373740,"statfs":{"total":95995035648,"available":92542709760,"internally_reserved":0,"allocated":3184406528,"data_stored":3184117637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":2.2999999999999998,"15min":1.105},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":4.0220000000000002,"5min":63.098999999999997,"15min":63.098999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.3049999999999999,"5min":2.012,"15min":1.0269999999999999},"min":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":6.8040000000000003,"5min":62.567,"15min":62.567},"last":3.0139999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.98099999999999998,"15min":0.70399999999999996},"min":{"1min":0.27100000000000002,"5min":0.222,"15min":0.222},"max":{"1min":4.8220000000000001,"5min":4.8220000000000001,"15min":5.056},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.288,"5min":1.1899999999999999,"15min":0.75600000000000001},"min":{"1min":0.252,"5min":0.22,"15min":0.22},"max":{"1min":4.3209999999999997,"5min":8.8119999999999994,"15min":8.8119999999999994},"last":2.5960000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.0960000000000001,"15min":0.73599999999999999},"min":{"1min":0.27500000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":4.3689999999999998,"5min":5.7629999999999999,"15min":5.7629999999999999},"last":0.441},{"interface":"front","average":{"1min":1.02,"5min":0.91800000000000004,"15min":0.66800000000000004},"min":{"1min":0.28999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.8180000000000001,"5min":3.5129999999999999,"15min":3.5129999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":2.21,"15min":1.1140000000000001},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":5.1100000000000003,"5min":75.286000000000001,"15min":75.286000000000001},"last":0.505},{"interface":"front","average":{"1min":1.161,"5min":2.1080000000000001,"15min":1.0920000000000001},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29499999999999998},"max":{"1min":3.8410000000000002,"5min":77.206999999999994,"15min":77.206999999999994},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:18:11 2024","interfaces":[{"interface":"back","average":{"1min":3.1970000000000001,"5min":1.0589999999999999,"15min":0.59699999999999998},"min":{"1min":0.27500000000000002,"5min":0.255,"15min":0.19},"max":{"1min":42.421999999999997,"5min":42.421999999999997,"15min":42.421999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":2.9449999999999998,"5min":1.0449999999999999,"15min":0.61399999999999999},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":42.191000000000003,"5min":42.191000000000003,"15min":42.191000000000003},"last":0.46700000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.2709999999999999,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.623,"15min":0.48699999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.254},"max":{"1min":5.1779999999999999,"5min":5.1779999999999999,"15min":5.1779999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:17:58 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":0.73699999999999999,"15min":0.56399999999999995},"min":{"1min":0.32300000000000001,"5min":0.27200000000000002,"15min":0.25700000000000001},"max":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.119,"5min":0.68700000000000006,"15min":0.53700000000000003},"min":{"1min":0.307,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.39800000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953596,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2119404,"kb_used_data":1948000,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":91625748,"statfs":{"total":95995035648,"available":93824765952,"internally_reserved":0,"allocated":1994752000,"data_stored":1994451455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":175503814},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.371,"15min":0.80800000000000005},"min":{"1min":0.245,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.3819999999999997,"5min":23.786000000000001,"15min":23.786000000000001},"last":2.9369999999999998},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.405,"15min":0.81699999999999995},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":4.0030000000000001,"5min":23.245000000000001,"15min":23.245000000000001},"last":0.219}]},{"osd":1,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":0.74099999999999999},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":4.4790000000000001,"5min":23.135000000000002,"15min":23.135000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.8140000000000001,"5min":1.347,"15min":0.80100000000000005},"min":{"1min":0.30599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":7.6180000000000003,"5min":23.434999999999999,"15min":23.434999999999999},"last":6.173}]},{"osd":2,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":0.95399999999999996,"15min":0.66900000000000004},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.4119999999999999,"5min":4.4119999999999999,"15min":4.4119999999999999},"last":6.2060000000000004},{"interface":"front","average":{"1min":1.492,"5min":1.0129999999999999,"15min":0.67300000000000004},"min":{"1min":0.38,"5min":0.27100000000000002,"15min":0.23799999999999999},"max":{"1min":7.0140000000000002,"5min":7.0140000000000002,"15min":7.0140000000000002},"last":2.5059999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:17:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":3.4849999999999999,"15min":1.52},"min":{"1min":0.30099999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.6469999999999998,"5min":154.137,"15min":154.137},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":3.2679999999999998,"15min":1.4490000000000001},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.27400000000000002},"max":{"1min":3.351,"5min":153.54400000000001,"15min":153.54400000000001},"last":2.1680000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.54400000000000004,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.5099999999999998,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":6.2329999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.61499999999999999,"15min":0.46899999999999997},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.25900000000000001},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.9079999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.623,"15min":0.505},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":3.097}]},{"osd":7,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":0.68700000000000006,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.1260000000000003,"5min":4.1260000000000003,"15min":4.1260000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.60899999999999999,"15min":0.51200000000000001},"min":{"1min":0.38700000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":2.1349999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117120,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2617460,"kb_used_data":2315436,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":91127692,"statfs":{"total":95995035648,"available":93314756608,"internally_reserved":0,"allocated":2371006464,"data_stored":2370693108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":309066184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:18:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9990000000000001,"5min":2.0499999999999998,"15min":1.0109999999999999},"min":{"1min":0.41499999999999998,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":5.0220000000000002,"5min":8.6829999999999998,"15min":8.6829999999999998},"last":1.375},{"interface":"front","average":{"1min":2.4039999999999999,"5min":2.0190000000000001,"15min":0.98599999999999999},"min":{"1min":0.22800000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":1.7809999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:18:27 2024","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.8879999999999999,"15min":0.97199999999999998},"min":{"1min":0.27300000000000002,"5min":0.186,"15min":0.186},"max":{"1min":9.5,"5min":9.5,"15min":9.5},"last":0.311},{"interface":"front","average":{"1min":1.248,"5min":1.867,"15min":0.94899999999999995},"min":{"1min":0.27100000000000002,"5min":0.193,"15min":0.188},"max":{"1min":4.2949999999999999,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:18:27 2024","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.863,"15min":0.97099999999999997},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":3.3410000000000002,"5min":10.786,"15min":10.786},"last":1},{"interface":"front","average":{"1min":2.157,"5min":2.2770000000000001,"15min":1.1359999999999999},"min":{"1min":0.33700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":6.2530000000000001,"5min":8.6440000000000001,"15min":8.6440000000000001},"last":0.432}]},{"osd":4,"last update":"Tue Sep 17 08:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.4610000000000001,"15min":0.84999999999999998},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":3.8159999999999998,"5min":7.742,"15min":7.742},"last":4.8929999999999998},{"interface":"front","average":{"1min":1.097,"5min":1.2969999999999999,"15min":0.81799999999999995},"min":{"1min":0.308,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.4340000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":1.286}]},{"osd":5,"last update":"Tue Sep 17 08:18:03 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.5289999999999999,"15min":0.90900000000000003},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.3849999999999998,"5min":8.2240000000000002,"15min":8.2240000000000002},"last":4.9349999999999996},{"interface":"front","average":{"1min":2.1019999999999999,"5min":1.607,"15min":0.98999999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":9.4260000000000002,"5min":9.4260000000000002,"15min":9.4260000000000002},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.393,"15min":0.85299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4870000000000001,"5min":10.945,"15min":10.945},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.796,"5min":1.6339999999999999,"15min":0.92900000000000005},"min":{"1min":0.41699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":5.1360000000000001,"5min":5.798,"15min":5.798},"last":0.92600000000000005}]},{"osd":7,"last update":"Tue Sep 17 08:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4470000000000001,"15min":0.90600000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":4.7069999999999999,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":3.6280000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.405,"15min":0.88500000000000001},"min":{"1min":0.34100000000000003,"5min":0.316,"15min":0.28999999999999998},"max":{"1min":4.3440000000000003,"5min":7.7830000000000004,"15min":7.7830000000000004},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411459,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2578408,"kb_used_data":2312668,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":91166744,"statfs":{"total":95995035648,"available":93354745856,"internally_reserved":0,"allocated":2368172032,"data_stored":2367875376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":272103878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:18:01 2024","interfaces":[{"interface":"back","average":{"1min":6.8570000000000002,"5min":3.2330000000000001,"15min":1.3380000000000001},"min":{"1min":0.22800000000000001,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":58.530000000000001,"5min":58.530000000000001,"15min":58.530000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":7.2480000000000002,"5min":3.496,"15min":1.4390000000000001},"min":{"1min":0.253,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":57.729999999999997,"5min":57.729999999999997,"15min":57.729999999999997},"last":0.28799999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:18:01 2024","interfaces":[{"interface":"back","average":{"1min":7.2530000000000001,"5min":2.887,"15min":1.2290000000000001},"min":{"1min":0.20200000000000001,"5min":0.157,"15min":0.157},"max":{"1min":57.581000000000003,"5min":57.581000000000003,"15min":57.581000000000003},"last":1.5649999999999999},{"interface":"front","average":{"1min":7.2530000000000001,"5min":2.7490000000000001,"15min":1.1859999999999999},"min":{"1min":0.32300000000000001,"5min":0.184,"15min":0.184},"max":{"1min":57.494,"5min":57.494,"15min":57.494},"last":1.897}]},{"osd":3,"last update":"Tue Sep 17 08:18:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.9430000000000001,"15min":1.0700000000000001},"min":{"1min":0.307,"5min":0.23499999999999999,"15min":0.20300000000000001},"max":{"1min":3.1179999999999999,"5min":36.792999999999999,"15min":36.792999999999999},"last":2.7349999999999999},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.9790000000000001,"15min":1.4099999999999999},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.219},"max":{"1min":6.8049999999999997,"5min":56.533000000000001,"15min":56.533000000000001},"last":3.3759999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.3899999999999999,"15min":0.83699999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.5939999999999999,"5min":13.58,"15min":13.58},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.427,"15min":0.85199999999999998},"min":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.3820000000000001,"5min":13.818,"15min":13.818},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.196,"15min":0.78000000000000003},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":5.1639999999999997,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.218,"15min":0.78700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.9630000000000001,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.753}]},{"osd":6,"last update":"Tue Sep 17 08:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":1.0960000000000001,"15min":0.84299999999999997},"min":{"1min":0.38200000000000001,"5min":0.32500000000000001,"15min":0.28199999999999997},"max":{"1min":1.7490000000000001,"5min":4.3200000000000003,"15min":5.7030000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3540000000000001,"15min":0.91600000000000004},"min":{"1min":0.38,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:18:01 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.234,"15min":0.80700000000000005},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":5.194,"5min":5.194,"15min":5.194},"last":1.236},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2849999999999999,"15min":0.80700000000000005},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.4409999999999998,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":0.72899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2998144,"kb_used_data":2706100,"kb_used_omap":1,"kb_used_meta":291966,"kb_avail":90747008,"statfs":{"total":95995035648,"available":92924936192,"internally_reserved":0,"allocated":2771046400,"data_stored":2770725442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":298973638},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":2.1219999999999999,"15min":0.995},"min":{"1min":0.193,"5min":0.16,"15min":0.16},"max":{"1min":4.9470000000000001,"5min":31.585999999999999,"15min":31.585999999999999},"last":3.722},{"interface":"front","average":{"1min":2.1749999999999998,"5min":2.4260000000000002,"15min":1.1000000000000001},"min":{"1min":0.32900000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":6.2119999999999997,"5min":35.807000000000002,"15min":35.807000000000002},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:18:04 2024","interfaces":[{"interface":"back","average":{"1min":1.9670000000000001,"5min":1.712,"15min":0.84499999999999997},"min":{"1min":0.21299999999999999,"5min":0.20100000000000001,"15min":0.19700000000000001},"max":{"1min":5.9050000000000002,"5min":7.2649999999999997,"15min":7.2649999999999997},"last":3.4340000000000002},{"interface":"front","average":{"1min":2.512,"5min":2.0299999999999998,"15min":0.95999999999999996},"min":{"1min":0.311,"5min":0.20899999999999999,"15min":0.20699999999999999},"max":{"1min":7.3049999999999997,"5min":10.638,"15min":10.638},"last":0.24299999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:18:29 2024","interfaces":[{"interface":"back","average":{"1min":1.97,"5min":1.7789999999999999,"15min":0.93100000000000005},"min":{"1min":0.44800000000000001,"5min":0.23300000000000001,"15min":0.20300000000000001},"max":{"1min":8.0559999999999992,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":3.4079999999999999},{"interface":"front","average":{"1min":3.3860000000000001,"5min":1.992,"15min":1.014},"min":{"1min":0.32300000000000001,"5min":0.217,"15min":0.189},"max":{"1min":11.18,"5min":11.18,"15min":11.18},"last":0.438}]},{"osd":4,"last update":"Tue Sep 17 08:17:54 2024","interfaces":[{"interface":"back","average":{"1min":2.218,"5min":1.518,"15min":0.86699999999999999},"min":{"1min":0.42299999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.8959999999999999,"5min":7.4859999999999998,"15min":7.4859999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":1.292,"5min":1.1910000000000001,"15min":0.78900000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.867,"5min":10.208,"15min":10.208},"last":3.4590000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:18:10 2024","interfaces":[{"interface":"back","average":{"1min":1.948,"5min":1.3560000000000001,"15min":0.83999999999999997},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":7.9139999999999997,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":1.0629999999999999},{"interface":"front","average":{"1min":2.9649999999999999,"5min":1.8140000000000001,"15min":0.96999999999999997},"min":{"1min":0.39500000000000002,"5min":0.29299999999999998,"15min":0.28299999999999997},"max":{"1min":7.4249999999999998,"5min":7.5099999999999998,"15min":7.5099999999999998},"last":0.312}]},{"osd":6,"last update":"Tue Sep 17 08:18:29 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":1.3120000000000001,"15min":0.86299999999999999},"min":{"1min":0.38700000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.0979999999999999,"5min":10.253,"15min":10.253},"last":3.488},{"interface":"front","average":{"1min":2.1920000000000002,"5min":1.4790000000000001,"15min":0.93300000000000005},"min":{"1min":0.309,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.9459999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":3.3450000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:18:00 2024","interfaces":[{"interface":"back","average":{"1min":2.347,"5min":1.659,"15min":0.93500000000000005},"min":{"1min":0.45600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":7.3289999999999997,"5min":10.066000000000001,"15min":10.066000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.4119999999999999,"15min":0.88600000000000001},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":6.2489999999999997,"5min":10.279,"15min":10.279},"last":3.5209999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2599596,"kb_used_data":2297760,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":91145556,"statfs":{"total":95995035648,"available":93333049344,"internally_reserved":0,"allocated":2352906240,"data_stored":2352606085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":309066182},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:17:41 2024","interfaces":[{"interface":"back","average":{"1min":5.6630000000000003,"5min":2.3660000000000001,"15min":1.024},"min":{"1min":0.30499999999999999,"5min":0.21299999999999999,"15min":0.19900000000000001},"max":{"1min":61.649000000000001,"5min":61.649000000000001,"15min":61.649000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":5.452,"5min":2.6859999999999999,"15min":1.1399999999999999},"min":{"1min":0.183,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":61.281999999999996,"5min":61.281999999999996,"15min":61.281999999999996},"last":1.252}]},{"osd":2,"last update":"Tue Sep 17 08:18:00 2024","interfaces":[{"interface":"back","average":{"1min":2.96,"5min":2.0070000000000001,"15min":0.98199999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":37.622,"5min":37.622,"15min":37.622},"last":1.5449999999999999},{"interface":"front","average":{"1min":3.25,"5min":2.0859999999999999,"15min":1.0149999999999999},"min":{"1min":0.217,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":37.677999999999997,"5min":37.677999999999997,"15min":37.677999999999997},"last":1.3520000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:18:31 2024","interfaces":[{"interface":"back","average":{"1min":3.923,"5min":3.0920000000000001,"15min":1.4850000000000001},"min":{"1min":0.33900000000000002,"5min":0.22,"15min":0.20999999999999999},"max":{"1min":39.942999999999998,"5min":61.926000000000002,"15min":61.926000000000002},"last":1.6419999999999999},{"interface":"front","average":{"1min":3.1200000000000001,"5min":2.9849999999999999,"15min":1.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":37.901000000000003,"5min":61.616,"15min":61.616},"last":1.3819999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:17:44 2024","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":1.728,"15min":0.95499999999999996},"min":{"1min":0.38200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":37.963999999999999,"5min":37.963999999999999,"15min":37.963999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":2.944,"5min":1.629,"15min":0.96699999999999997},"min":{"1min":0.29699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":37.731999999999999,"5min":37.731999999999999,"15min":37.731999999999999},"last":4.9649999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:18:17 2024","interfaces":[{"interface":"back","average":{"1min":2.9910000000000001,"5min":1.593,"15min":0.92400000000000004},"min":{"1min":0.28999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":37.854999999999997,"5min":37.854999999999997,"15min":37.854999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":3.1419999999999999,"5min":1.698,"15min":1.01},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":37.912999999999997,"5min":37.912999999999997,"15min":37.912999999999997},"last":0.45500000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.228,"15min":0.79900000000000004},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.4800000000000004,"5min":7.8899999999999997,"15min":7.8899999999999997},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.4159999999999999,"15min":0.86699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.4529999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.47799999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.9990000000000001,"5min":1.9890000000000001,"15min":1.109},"min":{"1min":0.42899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":37.704999999999998,"5min":37.704999999999998,"15min":37.704999999999998},"last":4.8849999999999998},{"interface":"front","average":{"1min":3.0880000000000001,"5min":1.9359999999999999,"15min":1.0640000000000001},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":37.878,"5min":37.878,"15min":37.878},"last":1.462}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2336030720,"data_stored":2336030720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2748317696,"data_stored":2748317696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2364674048,"data_stored":2364669976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2351104000,"data_stored":2351104000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1977614336,"data_stored":1977614336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3153268736,"data_stored":3153264664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1870200832,"data_stored":1870200832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1871253504,"data_stored":1871249432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:18:41.422 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-17T08:18:41.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:41.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.550 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:41.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:41.623 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[29565]: 8.5 scrub starts 2024-09-17T08:18:41.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:18:41.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:18:41.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[29565]: pgmap v376: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:41.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:41.685 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:41.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:41.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:41.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:41.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:41.884 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[40649]: 8.5 scrub starts 2024-09-17T08:18:41.884 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:18:41.884 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:18:41.884 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:41 smithi079 ceph-mon[40649]: pgmap v376: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:41.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:41.964 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:41.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:41.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:41.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:41 smithi173 ceph-mon[33187]: 8.5 scrub starts 2024-09-17T08:18:41.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:41 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:18:41.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:41 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:18:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:41 smithi173 ceph-mon[33187]: pgmap v376: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:18:42.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:42.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.117 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:42.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:42.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:42.241 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:42.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:42.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:42.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:42.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:42.476 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:42.507 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:42.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:42.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:42.636 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:42.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:42.644 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[40649]: 8.5 scrub ok 2024-09-17T08:18:42.644 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/352335306' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.644 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.644 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[29565]: 8.5 scrub ok 2024-09-17T08:18:42.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/352335306' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.645 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:42 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.646 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:18:42.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:42.771 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:42.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:42.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:42.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:42.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:42.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:42.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:42 smithi173 ceph-mon[33187]: 8.5 scrub ok 2024-09-17T08:18:42.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/352335306' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:42 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:18:42.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:43.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:43.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:43.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:43.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:43.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:43.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:43.294 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:43.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:43.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:18:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:18:43.389 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:18:43.389 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:18:43.420 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:18:43.420 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:18:43.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:43.452 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:18:43.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:18:43.467 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:43.483 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:18:43.483 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:18:43.515 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:18:43.516 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:18:43.552 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:18:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:18:43.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:43.589 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:18:43.589 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:18:43.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:43.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:43.756 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:43.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:18:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:18:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:18:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2560988094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[40649]: pgmap v378: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2560988094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:43 smithi079 ceph-mon[29565]: pgmap v378: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:18:43.900 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:43.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:43.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2560988094' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:43 smithi173 ceph-mon[33187]: pgmap v378: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:18:44.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:44.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.035 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:44.136 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:44.161 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:44.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:44.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:44.289 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:18:44.289 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:18:44.304 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:44.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:44.433 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:44.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:44.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:44.575 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:44.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:44.715 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:44.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:44.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:44.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:44.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:44.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:45.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:45.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:45.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:18:45.224 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:45.246 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:45.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.247 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:45.348 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:45.378 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:45.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:45.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:45.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:45.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:45.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:45.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.647 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:45.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:45.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:18:45.779 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:45.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:18:45.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:45.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:45.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:45.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:45.952 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:46.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:46.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:18:46.047 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:46.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:46.149 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:46.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:46.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:46.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:46.300 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:46.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:46.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:46.423 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:18:46.423 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:18:46.424 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T08:18:46.424 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T08:18:46.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2 2024-09-17T08:18:46.445 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:46.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:46.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:46.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.577 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:46.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:46.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:46.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.714 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:46.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:46.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:46 smithi173 ceph-mon[33187]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 939 op/s 2024-09-17T08:18:46.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:46 smithi079 ceph-mon[40649]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 939 op/s 2024-09-17T08:18:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:46 smithi079 ceph-mon[29565]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 939 op/s 2024-09-17T08:18:46.870 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:46 smithi079 systemd[1]: Stopping Ceph osd.2 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:18:46.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:46.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:46.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:46.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:47.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:47.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:47.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:47.136 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:47.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:47.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:47.264 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:47.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:47.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:47.392 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:47.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:47.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:47.522 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:47.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:47.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:47.665 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:47.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:47.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:47.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:47.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:47.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:18:47.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:47.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:47.938 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:47.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:48.020 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:48.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:48.021 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[40649]: pgmap v380: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 942 op/s 2024-09-17T08:18:48.021 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:48.021 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:48.021 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:47 smithi079 ceph-mon[29565]: pgmap v380: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 942 op/s 2024-09-17T08:18:48.039 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:48.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:48.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:48.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:48.191 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:48.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:48.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:18:48.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:18:48.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:47 smithi173 ceph-mon[33187]: pgmap v380: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 942 op/s 2024-09-17T08:18:48.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:48.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:48.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:48.342 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[57373]: 2024-09-17T08:18:48.128+0000 7f66e12d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:18:48.342 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[57373]: 2024-09-17T08:18:48.128+0000 7f66e12d6640 -1 osd.2 204 *** Got signal Terminated *** 2024-09-17T08:18:48.342 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[57373]: 2024-09-17T08:18:48.128+0000 7f66e12d6640 -1 osd.2 204 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:18:48.398 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:18:48.398 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:18:48.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:48.459 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:48.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:48.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:48.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:48.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:48.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:48.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:48.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:48.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:48.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.866 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:48.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:48.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:48.992 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:48.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:48.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:49.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:49.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:49.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:49.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:49.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:48 smithi173 ceph-mon[33187]: osd.2 marked itself down and dead 2024-09-17T08:18:49.237 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:48 smithi079 ceph-mon[40649]: osd.2 marked itself down and dead 2024-09-17T08:18:49.238 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:48 smithi079 ceph-mon[29565]: osd.2 marked itself down and dead 2024-09-17T08:18:49.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:49.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:49.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:49.362 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:49.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:49.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:49.485 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:49.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:49.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:18:49.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:49.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:49.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:49.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:49.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.734 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:49.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:49.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:49.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:49.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:49.869 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:49 smithi079 podman[131055]: 2024-09-17 08:18:49.544679119 +0000 UTC m=+1.831763783 container died fba9f777c246d10b21fe59cd4f218edff08245423ac9bf236aad90dd0d273a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2) 2024-09-17T08:18:49.959 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:49.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:49.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:49.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:50.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:50.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:50.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:50.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:49 smithi173 ceph-mon[33187]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:18:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:49 smithi173 ceph-mon[33187]: osdmap e205: 8 total, 7 up, 8 in 2024-09-17T08:18:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:49 smithi173 ceph-mon[33187]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:18:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:50.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:50.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:50.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[40649]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:18:50.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[40649]: osdmap e205: 8 total, 7 up, 8 in 2024-09-17T08:18:50.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[40649]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:18:50.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[29565]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:18:50.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[29565]: osdmap e205: 8 total, 7 up, 8 in 2024-09-17T08:18:50.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:49 smithi079 ceph-mon[29565]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:18:50.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:50.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:50.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:50.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:50.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:50.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:50.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.660 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:50.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:50.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:50.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:50.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:50.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:50.899 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:50 smithi079 podman[131055]: 2024-09-17 08:18:50.639371928 +0000 UTC m=+2.926456598 container cleanup fba9f777c246d10b21fe59cd4f218edff08245423ac9bf236aad90dd0d273a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-17T08:18:50.899 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:50 smithi079 bash[131055]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2 2024-09-17T08:18:50.929 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:50.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:50.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:50.932 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:51.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:51.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:51.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:51.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:51.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:51.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:50 smithi173 ceph-mon[33187]: osdmap e206: 8 total, 7 up, 8 in 2024-09-17T08:18:51.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:51.341 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:50 smithi079 ceph-mon[29565]: osdmap e206: 8 total, 7 up, 8 in 2024-09-17T08:18:51.342 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:50 smithi079 ceph-mon[40649]: osdmap e206: 8 total, 7 up, 8 in 2024-09-17T08:18:51.342 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:50 smithi079 podman[131276]: 2024-09-17 08:18:50.899114157 +0000 UTC m=+1.476840456 container remove fba9f777c246d10b21fe59cd4f218edff08245423ac9bf236aad90dd0d273a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-17T08:18:51.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.349 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:51.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:51.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:51.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:51.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:51.596 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:51.628 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:51.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:51.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:51.765 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:51.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:51.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:51.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:51.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:51.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:51.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:52.035 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:52.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:52.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:52.171 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:52.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.172 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:52.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:52.301 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:52 smithi079 podman[131458]: 2024-09-17 08:18:52.202389166 +0000 UTC m=+1.529450067 container died 93484269951f8aa4119b5958c569a8aa86f52de597768019318215b733f71e24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=zealous_margulis, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:18:52.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.305 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:52.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:52.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:52.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:52.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:52.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:52.563 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:52 smithi079 ceph-mon[40649]: pgmap v384: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 608 op/s 2024-09-17T08:18:52.563 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:52 smithi079 ceph-mon[29565]: pgmap v384: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 608 op/s 2024-09-17T08:18:52.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:52.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:52.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:52.716 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:52.722 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:52 smithi173 ceph-mon[33187]: pgmap v384: 148 pgs: 1 active+clean+scrubbing+deep, 17 stale+active+clean, 130 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 608 op/s 2024-09-17T08:18:52.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:52.853 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:52.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:52.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:52.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:18:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:18:52.987 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:52.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:52.988 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:53.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:53.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:53.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:53.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:53.252 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:53.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:53.302 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:18:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:18:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:18:53.324 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:18:53.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:53.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:53.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:53.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:53.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:53.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4187300056' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:53 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4187300056' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:53 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:53.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:53.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4187300056' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:53 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:18:53.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:53.788 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:53.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:53.890 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:53.929 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:53.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:53.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:54.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:54.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:54.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:54.189 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:54.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:54.336 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:54.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:54.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:54.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:54.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[29565]: osdmap e207: 8 total, 7 up, 8 in 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[29565]: pgmap v386: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.65k op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[40649]: osdmap e207: 8 total, 7 up, 8 in 2024-09-17T08:18:54.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:54 smithi079 ceph-mon[40649]: pgmap v386: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.65k op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:54.438 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:54.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:54.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:54.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:54.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:18:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:54 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:18:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:18:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:54 smithi173 ceph-mon[33187]: osdmap e207: 8 total, 7 up, 8 in 2024-09-17T08:18:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:54 smithi173 ceph-mon[33187]: pgmap v386: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.65k op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:54.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:54.844 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:54.875 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:54.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:54.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:54.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:54.992 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:18:55.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:55.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:55.111 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:18:55.111 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:18:55.112 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:18:55.113 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:18:55.113 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:18:55.113 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:18:55.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:55.167 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:55.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:55.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:55.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:55.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:55.404 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:55.434 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:55.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:55.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:55.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:55.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:55 smithi079 ceph-mon[29565]: Health check failed: Degraded data redundancy: 28879/229107 objects degraded (12.605%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:55.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/333658026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:55.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:55 smithi079 ceph-mon[40649]: Health check failed: Degraded data redundancy: 28879/229107 objects degraded (12.605%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:55.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/333658026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:55.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:55.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:55 smithi173 ceph-mon[33187]: Health check failed: Degraded data redundancy: 28879/229107 objects degraded (12.605%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/333658026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:18:55.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:55.830 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:55.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:55.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:55.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:55.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:55.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:56.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:56.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:56.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:56.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:56.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:56.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:56.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:56.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:56.460 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:56.491 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:56.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:56.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:56.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:56 smithi079 ceph-mon[40649]: pgmap v387: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 822 op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:56.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:56 smithi079 ceph-mon[29565]: pgmap v387: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 822 op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:56.625 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:56.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:56 smithi173 ceph-mon[33187]: pgmap v387: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 822 op/s; 28879/229107 objects degraded (12.605%) 2024-09-17T08:18:56.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:56.759 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:56.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:56.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:56.897 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:56.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:56.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:56.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:57.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:18:57.038 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:57.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:57.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:57.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.161 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:57.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:57.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:18:57.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.498 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:57.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:18:57.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:57.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:57.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:57.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:57.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:57.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:57.761 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:18:57.761 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:18:57.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:57.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:57.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:57.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:57.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:58.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:58.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:58.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:58.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.155 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:58.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:58.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:58.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:58.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:58.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:58.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.422 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:58.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:58.517 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:58 smithi079 ceph-mon[29565]: pgmap v388: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 725 op/s; 28393/227649 objects degraded (12.472%) 2024-09-17T08:18:58.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:58 smithi079 ceph-mon[40649]: pgmap v388: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 725 op/s; 28393/227649 objects degraded (12.472%) 2024-09-17T08:18:58.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:58.558 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:58.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:58.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:58.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:58.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:58 smithi173 ceph-mon[33187]: pgmap v388: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 725 op/s; 28393/227649 objects degraded (12.472%) 2024-09-17T08:18:58.796 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:58.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:58.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:58.869 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:58 smithi079 podman[132338]: 2024-09-17 08:18:58.423393602 +0000 UTC m=+0.034534150 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:18:58.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:18:58.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:58.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:58.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:18:59.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:59.097 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:59.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:59.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:18:59.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:59.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:18:59.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:18:59.358 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:59.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:18:59.460 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:18:59.493 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:18:59 smithi079 podman[132338]: 2024-09-17 08:18:59.164477151 +0000 UTC m=+0.775617702 container create 4a80a5595d3893e812881a3341e18c3951ed62afcaf5bf48f11b395ab555b643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:18:59.495 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:59.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:18:59.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:59.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:59.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:18:59.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:18:59.747 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:59 smithi173 ceph-mon[33187]: pgmap v389: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 324 B/s rd, 0 B/s wr, 1.35k op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:18:59.747 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:18:59 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 26037/206685 objects degraded (12.597%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:59.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:18:59.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:18:59.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:18:59.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:59 smithi079 ceph-mon[40649]: pgmap v389: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 324 B/s rd, 0 B/s wr, 1.35k op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:18:59.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:18:59 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 26037/206685 objects degraded (12.597%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:59.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:59 smithi079 ceph-mon[29565]: pgmap v389: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 324 B/s rd, 0 B/s wr, 1.35k op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:18:59.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:18:59 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 26037/206685 objects degraded (12.597%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:18:59.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:18:59.900 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:18:59.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:18:59.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:00.002 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:00.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.043 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:00.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:00.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:00.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:00.309 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.310 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:00.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:00.442 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:00.544 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:00.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:00.682 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:00.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.722 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:00.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:00.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:00.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:00.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:00.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:00.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:00.998 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:00.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:01.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:01.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:01.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:01.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:01.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:01.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:01.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:01.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:01.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:01.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:01.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:01.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:01.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:01.657 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:01.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:01.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:01.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:01.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[40649]: osdmap e208: 8 total, 7 up, 8 in 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[40649]: pgmap v391: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 634 B/s rd, 0 B/s wr, 926 op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[29565]: osdmap e208: 8 total, 7 up, 8 in 2024-09-17T08:19:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:01 smithi079 ceph-mon[29565]: pgmap v391: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 634 B/s rd, 0 B/s wr, 926 op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:19:01.870 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:19:01 smithi079 podman[132338]: 2024-09-17 08:19:01.701256404 +0000 UTC m=+3.312396958 container init 4a80a5595d3893e812881a3341e18c3951ed62afcaf5bf48f11b395ab555b643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:19:01.871 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:19:01 smithi079 podman[132338]: 2024-09-17 08:19:01.710996217 +0000 UTC m=+3.322136764 container start 4a80a5595d3893e812881a3341e18c3951ed62afcaf5bf48f11b395ab555b643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=reef-19e2a2b) 2024-09-17T08:19:01.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:01.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:01.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:01.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:01 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:01 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:01 smithi173 ceph-mon[33187]: osdmap e208: 8 total, 7 up, 8 in 2024-09-17T08:19:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:01 smithi173 ceph-mon[33187]: pgmap v391: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 634 B/s rd, 0 B/s wr, 926 op/s; 26037/206685 objects degraded (12.597%) 2024-09-17T08:19:02.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:02.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.061 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:02.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:02.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:02.196 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:02.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:02.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:02.331 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:02.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:02.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:02.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.467 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:02.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:02.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:02.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:02.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:02.705 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:02.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:02.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:02.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:02.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:02.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:02.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:02.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:02.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:02.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:02.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.864 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:02.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:02.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:02.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:19:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:02.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:02.997 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:02.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:03.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:03.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:03.120 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:19:02 smithi079 podman[132338]: 2024-09-17 08:19:02.802909078 +0000 UTC m=+4.414049703 container attach 4a80a5595d3893e812881a3341e18c3951ed62afcaf5bf48f11b395ab555b643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-17T08:19:03.120 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:19:02 smithi079 podman[132338]: 2024-09-17 08:19:02.803193583 +0000 UTC m=+4.414334128 container died 4a80a5595d3893e812881a3341e18c3951ed62afcaf5bf48f11b395ab555b643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2) 2024-09-17T08:19:03.127 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:03.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:03.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:03.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.268 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:03.268 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:03.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:03.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.405 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:03.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:03.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:03.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.541 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:03.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:03.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:03.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:03.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:03.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:03.813 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:03.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:03 smithi079 ceph-mon[40649]: pgmap v392: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:03 smithi079 ceph-mon[29565]: pgmap v392: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:03.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:03.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:03.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:03.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:03 smithi173 ceph-mon[33187]: pgmap v392: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:04.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:04.090 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:04.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:04.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:04.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:04.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:04.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:04.363 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:04.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:04.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:04.503 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:04.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:04.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:04.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:04.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:04 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:04.653 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:04.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:04.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:04.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:04.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:04.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:04.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:04.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:04.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:04 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:19:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:04 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:19:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:04 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:05.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:05.057 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:05.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:05.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:05.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:05.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:05.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:05.327 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:05.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:05.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:05.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:05.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:05.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:05.602 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:05.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:05 smithi079 ceph-mon[40649]: pgmap v393: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:05 smithi079 ceph-mon[29565]: pgmap v393: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:05.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:05.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:05.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:05.732 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:05.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T08:19:05.782 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:05.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:05.867 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:05.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:05.869 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:05.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:19:05.882 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T08:19:05.882 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-17T08:19:05.882 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi079.front.sepia.ceph.com 2024-09-17T08:19:05.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.2 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-17T08:19:05.969 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:05 smithi173 ceph-mon[33187]: pgmap v393: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 23448/186096 objects degraded (12.600%) 2024-09-17T08:19:06.004 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:06.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:06.160 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:06.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:06.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:06.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:06.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:06.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:06.429 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:06.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:06.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:06.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:06.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:06.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:06.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:06.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:06.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:06.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:06.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:06.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:06 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 23448/186096 objects degraded (12.600%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:06 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 23448/186096 objects degraded (12.600%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:06.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:06.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:06.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:06.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:06.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:06 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 23448/186096 objects degraded (12.600%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:07.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:07.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:07.183 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.2/config 2024-09-17T08:19:07.214 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:07.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:07.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:07.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:07.377 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:07.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:07.440 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:19:07.440 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:19:07.440 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:19:07.441 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:19:07.441 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:19:07.478 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:07.514 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:07.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:07.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:07.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:07.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:07.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:07.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:07.826 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:07 smithi079 ceph-mon[29565]: pgmap v394: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 22966/184650 objects degraded (12.438%) 2024-09-17T08:19:07.826 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:07 smithi079 ceph-mon[40649]: pgmap v394: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 22966/184650 objects degraded (12.438%) 2024-09-17T08:19:07.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:07.911 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:07.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:07.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:07 smithi173 ceph-mon[33187]: pgmap v394: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 22966/184650 objects degraded (12.438%) 2024-09-17T08:19:08.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:08.048 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:08.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:08.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:08.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:08.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:08.325 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:08.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:08.462 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:08.564 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:08.596 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:08.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:08.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:08.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:08.852 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:08.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:08.954 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:08.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:08.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:08.990 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:09.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:09.121 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:09.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:09.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:09.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:09.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:09.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:09.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:09.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:09.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:09.522 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:09.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:09.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:09.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:09.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.663 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:09.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:09.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.798 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:09.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:09.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:09.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:09.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:09.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:10.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:10.058 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:10.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:10.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:10.193 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:10.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:10.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:10.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:19:10.329 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:10.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:10.431 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:10.463 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:10.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:10.546 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:10 smithi079 ceph-mon[29565]: pgmap v395: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:10.546 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:10 smithi079 ceph-mon[40649]: pgmap v395: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:10.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:10.593 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:10.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:10.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:10 smithi173 ceph-mon[33187]: pgmap v395: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.43k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:10.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:10.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:10.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:10.868 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:10.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:10.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:10.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:11.002 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:11.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:11.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:11.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.136 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:11.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:11.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:11.375 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:11.411 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:11.513 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:11.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:11.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:11 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 20675/163857 objects degraded (12.618%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:11 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 20675/163857 objects degraded (12.618%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:11.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:11.688 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:11.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:11 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 20675/163857 objects degraded (12.618%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:11.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:11.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.826 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:11.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:11.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:11.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:11.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:11.956 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:19:11.956 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:19:11.995 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:19:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:19:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:19:12.026 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:19:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:19:12.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:12.059 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:19:12.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:19:12.088 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:12.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:12.090 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:19:12.091 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:19:12.119 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:19:12.120 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:19:12.153 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:19:12.154 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:19:12.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:12.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:12.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:12.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:12.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:12.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:12.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:12.499 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:12.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:12.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:12.634 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:12 smithi079 ceph-mon[29565]: pgmap v396: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 940 B/s rd, 0 B/s wr, 1.31k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:12.634 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2882373089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:12.634 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:12 smithi079 ceph-mon[40649]: pgmap v396: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 940 B/s rd, 0 B/s wr, 1.31k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:12.634 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2882373089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:12.646 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:12.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:12 smithi173 ceph-mon[33187]: pgmap v396: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 940 B/s rd, 0 B/s wr, 1.31k op/s; 20675/163857 objects degraded (12.618%) 2024-09-17T08:19:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2882373089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:12.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:12.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:12.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:12.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:12.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:12.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:12.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:13.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:13.058 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:13.104 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:13.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:13.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:13.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:13.216 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:13.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:13.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:13.329 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:13.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:13.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:13.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:13.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:13.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:13.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:13.865 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:13.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:13.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:13.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:14.000 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:14.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:14.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:14.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.141 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:14.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:14.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:14.261 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:19:14.262 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:19:14.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:14.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:14.375 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:14.411 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:14.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:14.494 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:14.494 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:14 smithi079 ceph-mon[29565]: pgmap v397: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:14.494 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:14.494 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:14 smithi079 ceph-mon[40649]: pgmap v397: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:14.513 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:14.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.544 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:14.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:14.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:14.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:14.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:14 smithi173 ceph-mon[33187]: pgmap v397: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:14.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:14.820 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:14.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:14.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:14.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:14.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:15.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:15.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.092 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:15.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:15.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:15.224 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:15.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:15.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:15.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.358 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:15.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:15.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:19:15.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.496 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:15.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:15.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:15.633 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:15.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:15.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:15.774 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:15.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:15.852 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1713602006' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:15.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:15 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:15.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1713602006' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:15.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:15 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:15.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:15.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:15.907 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:15.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1713602006' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:15 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:19:16.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:16.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:16.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:16.143 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:16.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:16.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:16.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:19:16.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:16.352 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:16.454 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:16.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[40649]: pgmap v398: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[40649]: osdmap e209: 8 total, 7 up, 8 in 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 17980/142011 objects degraded (12.661%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:16.560 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[29565]: pgmap v398: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:16.561 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:16.561 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:19:16.561 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[29565]: osdmap e209: 8 total, 7 up, 8 in 2024-09-17T08:19:16.561 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:16 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 17980/142011 objects degraded (12.661%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:16.586 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:16.616 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:16.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:16.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:16.752 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:16.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:16.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:16.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:16.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:16 smithi173 ceph-mon[33187]: pgmap v398: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s; 17980/142011 objects degraded (12.661%) 2024-09-17T08:19:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:16 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:16 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:19:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:16 smithi173 ceph-mon[33187]: osdmap e209: 8 total, 7 up, 8 in 2024-09-17T08:19:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:16 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 17980/142011 objects degraded (12.661%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:16.989 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:17.015 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:17.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:17.151 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:17.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:17.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:17.282 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:17.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:17.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:17.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:17.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:17.479 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:17.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:17.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:17.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:17.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:17.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:17.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:17.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[40649]: osdmap e210: 8 total, 7 up, 8 in 2024-09-17T08:19:17.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:17.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[40649]: pgmap v401: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 967 op/s; 17527/140652 objects degraded (12.461%) 2024-09-17T08:19:17.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:17.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:17.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[29565]: osdmap e210: 8 total, 7 up, 8 in 2024-09-17T08:19:17.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:17.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:17 smithi079 ceph-mon[29565]: pgmap v401: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 967 op/s; 17527/140652 objects degraded (12.461%) 2024-09-17T08:19:17.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:17.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:17.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:17.833 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:17.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:17.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:17.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:17.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:17.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:17 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:17 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:17 smithi173 ceph-mon[33187]: osdmap e210: 8 total, 7 up, 8 in 2024-09-17T08:19:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:17 smithi173 ceph-mon[33187]: pgmap v401: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 967 op/s; 17527/140652 objects degraded (12.461%) 2024-09-17T08:19:18.068 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:18.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:18.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:18.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:18.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:18.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:18.255 INFO:teuthology.orchestra.run.smithi079.stdout:6.f 2024-09-17T08:19:18.255 INFO:teuthology.orchestra.run.smithi079.stdout:6.9 2024-09-17T08:19:18.255 INFO:teuthology.orchestra.run.smithi079.stdout:8.a 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.3 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.1c 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:8.1 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.1 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.17 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:2.3 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:4.18 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:4.12 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:2.c 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:8.e 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.12 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:2.7 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:6.e 2024-09-17T08:19:18.256 INFO:teuthology.orchestra.run.smithi079.stdout:2.1c 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:5.a 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:3.0 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:2.1 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:2.e 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:8.d 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:8.4 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:4.6 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:5.4 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:2.17 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:2.16 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:4.f 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:2.10 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:4.19 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:5.6 2024-09-17T08:19:18.257 INFO:teuthology.orchestra.run.smithi079.stdout:4.1c 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:5.d 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:6.2 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:4.4 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:6.8 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:4.d 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:5.14 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:6.b 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:5.1c 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:5.1e 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:8.f 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:2.12 2024-09-17T08:19:18.258 INFO:teuthology.orchestra.run.smithi079.stdout:5.8 2024-09-17T08:19:18.259 INFO:teuthology.orchestra.run.smithi079.stdout:5.b 2024-09-17T08:19:18.259 INFO:teuthology.orchestra.run.smithi079.stdout:6.14 2024-09-17T08:19:18.259 INFO:teuthology.orchestra.run.smithi079.stdout:6.0 2024-09-17T08:19:18.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:18.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.359 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:18.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:18.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:19:18.416 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:19:18.416 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:19:18.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:18.495 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:18.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:18.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:18.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:18.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:18.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:18.750 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:18 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:19:18.750 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:18 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:19:18.750 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:18 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:19:18.750 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:18 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:19:18.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:18.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:18.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:18 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:19:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:18 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:19:18.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:18.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:18.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:18.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:19.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:19.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.049 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:19.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:19.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:19.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:19.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:19.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:19.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.316 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:19.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:19.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:19.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.457 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:19.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:19.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:19.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:19.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:19.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:19 smithi079 ceph-mon[40649]: pgmap v402: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:19.730 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:19 smithi079 ceph-mon[29565]: pgmap v402: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:19.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.739 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:19.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:19.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:19.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:19.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:19.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:19 smithi173 ceph-mon[33187]: pgmap v402: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:19.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:20.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:20.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:20.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:20.141 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:20.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:20.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:20.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:20.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:20.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:20.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.531 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:20.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:20.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:20.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:20.765 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:20.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:20.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:20 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:19:20.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:20 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:19:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:20 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:19:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:20 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:19:20.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:20.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:20.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:20.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:20 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:19:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:20 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:19:21.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:21.058 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:21.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:21.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:21.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:21.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:21.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:21.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:21.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:21.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:21.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:21.566 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:21.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:21.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:21.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:21.739 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:21.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:21.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:21.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:19:21.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:19:21.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 15263/120675 objects degraded (12.648%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:21.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[29565]: pgmap v403: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 889 op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:21.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:19:21.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:19:21.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 15263/120675 objects degraded (12.648%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:21.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:21 smithi079 ceph-mon[40649]: pgmap v403: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 889 op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:21.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:21.874 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:21.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:21 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:19:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:21 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:19:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:21 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 15263/120675 objects degraded (12.648%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:21 smithi173 ceph-mon[33187]: pgmap v403: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 889 op/s; 15263/120675 objects degraded (12.648%) 2024-09-17T08:19:21.975 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:22.010 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:22.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:19:22.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:22.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:22.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:22.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:22.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:22.381 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:22.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:22.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:22.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.2 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 6.12 --file /var/log/ceph/exp.6.12.2 2024-09-17T08:19:22.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:22.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:22.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:22.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:22.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.2/config 2024-09-17T08:19:22.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:19:22.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:19:22.817 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:22 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:19:22.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:22.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:22.957 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:22.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:22.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:22.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:22 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:19:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:22 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:19:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:22 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:19:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:22 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:19:23.059 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:23.093 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:23.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:23.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:23.195 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:23.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:23.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:23.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:23.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:23.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:23.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:23.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:23.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:23.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:23.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:23.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:23.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:23.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.777 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:23.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:23.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:23.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:19:23.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:19:23.797 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[40649]: pgmap v404: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:23.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:23.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:19:23.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:19:23.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:23 smithi079 ceph-mon[29565]: pgmap v404: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:23.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:23.912 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:23.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:23.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:23 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:19:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:23 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:19:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:23 smithi173 ceph-mon[33187]: pgmap v404: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:24.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:24.046 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.047 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:24.081 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:19:24.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:19:24.109 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:19:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:19:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:19:24.140 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:19:24.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:19:24.148 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:24.170 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:19:24.171 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:19:24.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:24.204 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:19:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:19:24.236 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:19:24.236 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:19:24.266 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:19:24.267 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:19:24.285 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:24.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:24.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:24.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:24.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:24.578 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:24.589 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:24 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1979377226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:24.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:24 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1979377226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:24.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:24.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:24.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:24.858 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:24.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:24 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1979377226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:24.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:24.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:24.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:25.129 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:25.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:25.269 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.371 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:25.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.409 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:19:25.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:19:25.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:25.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.644 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: osdmap e211: 8 total, 7 up, 8 in 2024-09-17T08:19:25.660 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[40649]: pgmap v406: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.62k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:25.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:25.662 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: osdmap e211: 8 total, 7 up, 8 in 2024-09-17T08:19:25.662 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:25 smithi079 ceph-mon[29565]: pgmap v406: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.62k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:25.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:25.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:25.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:25.852 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:25.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:25.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:25.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:25.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: osdmap e211: 8 total, 7 up, 8 in 2024-09-17T08:19:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:25 smithi173 ceph-mon[33187]: pgmap v406: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.62k op/s; 12512/98739 objects degraded (12.672%) 2024-09-17T08:19:26.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:26.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:26.123 INFO:teuthology.orchestra.run.smithi079.stderr:Exporting 6.12 info 6.12( empty local-lis/les=70/71 n=0 ec=70/70 lis/c=70/70 les/c/f=71/71/0 sis=70) 2024-09-17T08:19:26.127 INFO:teuthology.orchestra.run.smithi079.stderr:Export successful 2024-09-17T08:19:26.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:26.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.231 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:26.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:26.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:26.360 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:26.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:26.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:26.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.499 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:26.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:26.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:26.637 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:26.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:26.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:26.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:26.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 12512/98739 objects degraded (12.672%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:19:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:26 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 12512/98739 objects degraded (12.672%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:26.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:26.916 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:26.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:26.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:26 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:19:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:26 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:19:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:26 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:19:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:26 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:19:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:26 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 12512/98739 objects degraded (12.672%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:27.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:27.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:27.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:27.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:27.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:27.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:27.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:27.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.331 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:27.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:27.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:19:27.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:27.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:27.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:27.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:27.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[29565]: 8.8 scrub starts 2024-09-17T08:19:27.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:19:27.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:19:27.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[29565]: 8.8 scrub ok 2024-09-17T08:19:27.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[29565]: pgmap v407: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 12058/97377 objects degraded (12.383%) 2024-09-17T08:19:27.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:27.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:27.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:27.750 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:27.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:27.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.2 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 6.12 2024-09-17T08:19:27.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[40649]: 8.8 scrub starts 2024-09-17T08:19:27.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:19:27.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:19:27.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[40649]: 8.8 scrub ok 2024-09-17T08:19:27.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:27 smithi079 ceph-mon[40649]: pgmap v407: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 12058/97377 objects degraded (12.383%) 2024-09-17T08:19:27.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:27.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:27.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:27.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:27 smithi173 ceph-mon[33187]: 8.8 scrub starts 2024-09-17T08:19:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:27 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:19:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:27 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:19:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:27 smithi173 ceph-mon[33187]: 8.8 scrub ok 2024-09-17T08:19:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:27 smithi173 ceph-mon[33187]: pgmap v407: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 2.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 12058/97377 objects degraded (12.383%) 2024-09-17T08:19:27.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:28.021 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:28.022 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.2/config 2024-09-17T08:19:28.122 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:28.154 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:28.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:28.292 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:28.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:28.431 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:28.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:28.569 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:28.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:28.710 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:28.813 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:28.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.846 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:28.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:28.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:19:28.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:19:28.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: osdmap e212: 8 total, 7 up, 8 in 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: osdmap e212: 8 total, 7 up, 8 in 2024-09-17T08:19:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:19:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:19:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:19:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:28 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:19:28.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: osdmap e212: 8 total, 7 up, 8 in 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:19:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:28 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:19:28.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:28.986 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:28.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:29.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:29.114 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:29.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:29.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:29.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:29.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:29.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:29.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:29.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:29.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:29.514 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:29.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.515 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:29.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:29.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:29.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:29.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:29.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:29.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:19:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:19:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[40649]: pgmap v409: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:19:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:19:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:29 smithi079 ceph-mon[29565]: pgmap v409: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:29.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:29.906 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:29.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:29.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:29 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:19:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:29 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:19:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:29 smithi173 ceph-mon[33187]: pgmap v409: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:30.007 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:30.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:30.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:30.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:30.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:30.311 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:30.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:30.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:30.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:30.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:30.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.579 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:30.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:30.712 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:30.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:30.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:30.849 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:30 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:19:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:30 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:19:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:30 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:19:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:30 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:19:30.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:30.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:30 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:19:30.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:30 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:19:30.978 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:30.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:30.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:31.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:31.116 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:31.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:31.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:31.259 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:31.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:31.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:31.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:31.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:31.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:31.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:31.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:31.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:31.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:31.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:31.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:31.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.810 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:31.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 9606/76182 objects degraded (12.609%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[40649]: pgmap v410: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 940 op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:19:31.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:19:31.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:19:31.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 9606/76182 objects degraded (12.609%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:31.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:31 smithi079 ceph-mon[29565]: pgmap v410: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 940 op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:31.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:31.948 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:31.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:31.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:19:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:19:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:19:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:19:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 9606/76182 objects degraded (12.609%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:31 smithi173 ceph-mon[33187]: pgmap v410: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 940 op/s; 9606/76182 objects degraded (12.609%) 2024-09-17T08:19:32.024 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:19:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:19:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:250785 8.f 8:f5aeb2cf:::benchmark_data_smithi079_115040_object115765:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:19:32.013602+0000", 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0098288190000000008, 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098498609999999997, 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 250785 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013602+0000", 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013602+0000", 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:19:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013601+0000", 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013608+0000", 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0079999999999996e-06 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013625+0000", 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6966e-05 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.013633+0000", 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7859999999999997e-06 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.015894+0000", 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00226121 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:19:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.016013+0000", 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118597 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.016135+0000", 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012252199999999999 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:250794 8.f 8:f50e5ea3:::benchmark_data_smithi079_115040_object115774:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:19:32.018891+0000", 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0045398979999999997, 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046083859999999999, 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 250794 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:19:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018891+0000", 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018891+0000", 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:19:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018890+0000", 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018895+0000", 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6430000000000003e-06 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018909+0000", 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3651e-05 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.018915+0000", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7410000000000003e-06 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019222+0000", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030624100000000001 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019303+0000", 2024-09-17T08:19:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1240000000000001e-05 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019369+0000", 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6459999999999997e-05 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:250796 8.f 8:f9c09249:::benchmark_data_smithi079_115040_object115776:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:19:32.019975+0000", 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0034560239999999998, 2024-09-17T08:19:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035725330000000001, 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 250796 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019975+0000", 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019975+0000", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019974+0000", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019979+0000", 2024-09-17T08:19:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3540000000000002e-06 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019992+0000", 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.292e-05 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.019998+0000", 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2580000000000001e-06 2024-09-17T08:19:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.021771+0000", 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001772994 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.022080+0000", 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030893899999999999 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-17T08:19:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:19:32.022429+0000", 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034937199999999997 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 3 2024-09-17T08:19:32.039 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:19:32.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:32.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.104 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:32.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:32.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:32.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:32.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:32.373 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:32.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: marking collection for removal 2024-09-17T08:19:32.423 INFO:teuthology.orchestra.run.smithi079.stdout:setting '_remove' omap key 2024-09-17T08:19:32.424 INFO:teuthology.orchestra.run.smithi079.stdout:finish_remove_pgs 6.12_head removing 6.12 2024-09-17T08:19:32.427 INFO:teuthology.orchestra.run.smithi079.stdout:Remove successful 2024-09-17T08:19:32.475 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:32.508 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:32.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:32.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:32.629 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:19:32.649 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:32.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:32.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:32.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:32.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:19:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3752922565' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3752922565' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:32.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:32.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:32.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:32.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:32.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:19:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:19:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3752922565' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:19:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:33.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:33.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:33.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:33.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:33.188 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:33.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:33.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:19:33.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:33.291 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:33.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:33.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:33.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:33.465 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:33.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:33.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:33.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:33.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: osdmap e213: 8 total, 7 up, 8 in 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[29565]: pgmap v412: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.87k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:33.617 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:19:33.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:19:33.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:19:33.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:19:33.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:19:33.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:33.732 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:33.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:33.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:33.863 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:33.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:19:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: osdmap e213: 8 total, 7 up, 8 in 2024-09-17T08:19:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:33 smithi079 ceph-mon[40649]: pgmap v412: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.87k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:33.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.2 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.6.12.2 2024-09-17T08:19:33.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: osdmap e213: 8 total, 7 up, 8 in 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:33 smithi173 ceph-mon[33187]: pgmap v412: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.87k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:33.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:33.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:33.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:34.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:34.135 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:34.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:34.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:34.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.269 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:34.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:34.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:34.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:34.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:34.508 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.2/config 2024-09-17T08:19:34.555 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:34.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:34.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:34.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:34.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:34.796 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:34.831 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:19:34.832 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:19:34.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:19:34.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:34 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:19:34.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.834 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:34.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:34.908 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:34.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:19:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:34 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:19:34.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:34.977 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:34.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:35.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:35.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.112 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:35.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:35.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:35.244 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:35.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:35.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:35.354 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:35.375 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:35.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:35.476 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:35.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:35.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:35.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:35.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.639 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:35.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:35.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:35.767 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:35.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:35.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 5.3 deep-scrub starts 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 5.3 deep-scrub ok 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[40649]: pgmap v413: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:19:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:19:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 5.3 deep-scrub starts 2024-09-17T08:19:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 5.3 deep-scrub ok 2024-09-17T08:19:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:19:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:19:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:35 smithi079 ceph-mon[29565]: pgmap v413: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:35.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:35.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 5.3 deep-scrub starts 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 5.3 deep-scrub ok 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:19:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:35 smithi173 ceph-mon[33187]: pgmap v413: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 6761/53811 objects degraded (12.564%) 2024-09-17T08:19:36.012 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:36.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:36.151 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:36.186 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:36.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:36.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:36.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:36.445 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:36.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:36.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:19:36.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:36.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:36.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:36.721 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:36.823 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:36.853 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:36.955 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:19:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:19:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:19:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:19:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:19:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:19:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:36 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 6761/53811 objects degraded (12.564%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:36.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:36.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:36.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:37.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:37.119 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:19:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 6761/53811 objects degraded (12.564%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:37.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:19:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:19:37.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:19:37.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:19:37.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:19:37.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:19:37.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:36 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 6761/53811 objects degraded (12.564%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:37.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:37.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.258 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:37.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:37.389 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:37.491 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:37.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:37.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:37.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:37.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:37.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.782 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:37.883 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:37.908 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:37.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:37.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 2.6 deep-scrub starts 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 2.6 deep-scrub ok 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:19:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:37 smithi173 ceph-mon[33187]: pgmap v414: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.38k op/s; 6301/52431 objects degraded (12.018%) 2024-09-17T08:19:38.009 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:38.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:38.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 2.6 deep-scrub starts 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 2.6 deep-scrub ok 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[40649]: pgmap v414: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.38k op/s; 6301/52431 objects degraded (12.018%) 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 2.6 deep-scrub starts 2024-09-17T08:19:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 2.6 deep-scrub ok 2024-09-17T08:19:38.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:19:38.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:19:38.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:37 smithi079 ceph-mon[29565]: pgmap v414: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 1.1 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.38k op/s; 6301/52431 objects degraded (12.018%) 2024-09-17T08:19:38.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:38.186 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:38.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:38.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:38.323 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:38.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.324 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:38.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:38.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:38.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:38.569 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:38.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.605 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:38.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:38.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:38.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:38.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:38.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:38.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:38.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:38 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:19:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:38 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:19:38.985 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:39.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:39.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:39.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:38 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:19:39.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:38 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:19:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:38 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:19:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:38 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:19:39.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:39.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:39.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:39.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:39.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.307 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:39.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:39.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:39.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:39.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:39.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:39.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:39.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:39.713 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:39.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:39.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:39.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.851 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:39.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:39.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:19:39.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:19:39.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[29565]: pgmap v415: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:19:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:19:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:39 smithi079 ceph-mon[40649]: pgmap v415: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:39.953 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:39 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:19:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:39 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:19:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:39 smithi173 ceph-mon[33187]: pgmap v415: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:39.984 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:39.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:39.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:40.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:40.119 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:40.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:40.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:40.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:40.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:40.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:40.394 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:40.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:40.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:40.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:40.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:40.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:40.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:40.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:40.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:40.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:40.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[40649]: osdmap e214: 8 total, 7 up, 8 in 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:19:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:19:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:40 smithi079 ceph-mon[29565]: osdmap e214: 8 total, 7 up, 8 in 2024-09-17T08:19:40.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:40.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:40.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:40.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:19:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:40 smithi173 ceph-mon[33187]: osdmap e214: 8 total, 7 up, 8 in 2024-09-17T08:19:41.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:41.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.069 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:41.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:41.170 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:41.202 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:41.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:41.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:41.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.342 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:41.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:41.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:41.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:41.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:41.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:41.616 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:41.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:41.697 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:19:41.697 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:19:41.697 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 4058/32181 objects degraded (12.610%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:41.697 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[40649]: pgmap v417: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:41.697 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:19:41.698 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:19:41.698 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 4058/32181 objects degraded (12.610%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:41.698 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:41 smithi079 ceph-mon[29565]: pgmap v417: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:41.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:41.750 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:41.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.751 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:41.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:41.882 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:41.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:41.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:41 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:19:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:41 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:19:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:41 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 4058/32181 objects degraded (12.610%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:41 smithi173 ceph-mon[33187]: pgmap v417: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 659 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s; 4058/32181 objects degraded (12.610%) 2024-09-17T08:19:41.983 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:42.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:42.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:42.162 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:42.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:42.297 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:42.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:get_pg_num_history pg_num_history pg_num_history(e204 pg_nums {1={20=1},2={64=32},3={64=3},4={66=32},5={68=32},6={70=32},7={91=16},8={165=16}} deleted_pools 163,7) 2024-09-17T08:19:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:Importing pgid 6.12 2024-09-17T08:19:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:write_pg epoch 177 info 6.12( empty local-lis/les=70/71 n=0 ec=70/70 lis/c=70/70 les/c/f=71/71/0 sis=70) 2024-09-17T08:19:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:Import successful 2024-09-17T08:19:42.391 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:42.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:42.432 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.433 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:42.534 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:42.569 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:42.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:42.700 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:42.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:42.802 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:42.839 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:42.941 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:42.977 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:42.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:42.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:43.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:43.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:43.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:43.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:43.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:43.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:43.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:43.313 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:43.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:43.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:43.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:43.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:43.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.524 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:19:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:19:43.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:43 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:19:43.625 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:43.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:43.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:19:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:43 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:19:43.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:43.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.800 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:43.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:43.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:43.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:43.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:43.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:44.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:44.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:44.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:44.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:44.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:44.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:44.308 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:44.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:44.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:44.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:44.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:44.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:44.605 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:44.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:44 smithi079 ceph-mon[40649]: pgmap v418: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:44.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:44.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:44 smithi079 ceph-mon[29565]: pgmap v418: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:44.706 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:44 smithi173 ceph-mon[33187]: pgmap v418: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:44.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:44.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:44.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:44.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:44.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:44.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:44.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:45.004 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:45.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:45.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:45.141 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:45.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:45.173 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Removed 135017 objects 2024-09-17T08:19:45.173 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Clean up completed and total clean up time :105.634 2024-09-17T08:19:45.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:45.264 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:45.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:45.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:45.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:45.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.390 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:45.469 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[40649]: 2.4 deep-scrub starts 2024-09-17T08:19:45.469 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[40649]: 2.4 deep-scrub ok 2024-09-17T08:19:45.469 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:19:45.470 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:19:45.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[29565]: 2.4 deep-scrub starts 2024-09-17T08:19:45.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[29565]: 2.4 deep-scrub ok 2024-09-17T08:19:45.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:19:45.470 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:45 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:19:45.491 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:45.512 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:45.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:45.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:45.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:45.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:45.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:45 smithi173 ceph-mon[33187]: 2.4 deep-scrub starts 2024-09-17T08:19:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:45 smithi173 ceph-mon[33187]: 2.4 deep-scrub ok 2024-09-17T08:19:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:45 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:19:45.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:45 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:19:45.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:45.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:45.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:45.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:45.883 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:45.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:45.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:45.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:46.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:46.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:46.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:46.115 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:19:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:19:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:19:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:19:46.115 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:19:46.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.129 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:46.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:46.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:46.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:19:46.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:19:46.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:19:46.331 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:19:46.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: osdmap e215: 8 total, 7 up, 8 in 2024-09-17T08:19:46.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: 2.1d deep-scrub starts 2024-09-17T08:19:46.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: 2.1d deep-scrub ok 2024-09-17T08:19:46.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: pgmap v420: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.77k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:46.332 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 1250/9981 objects degraded (12.524%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:46.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:46.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:46.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:46.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:46.499 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:46.601 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: osdmap e215: 8 total, 7 up, 8 in 2024-09-17T08:19:46.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: 2.1d deep-scrub starts 2024-09-17T08:19:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: 2.1d deep-scrub ok 2024-09-17T08:19:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: pgmap v420: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.77k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:46.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:46 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 1250/9981 objects degraded (12.524%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:46.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:46.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: osdmap e215: 8 total, 7 up, 8 in 2024-09-17T08:19:46.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: 2.1d deep-scrub starts 2024-09-17T08:19:46.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: 2.1d deep-scrub ok 2024-09-17T08:19:46.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: pgmap v420: 148 pgs: 28 active+undersized, 19 active+undersized+degraded, 101 active+clean; 196 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.77k op/s; 1250/9981 objects degraded (12.524%) 2024-09-17T08:19:46.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:46 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 1250/9981 objects degraded (12.524%), 19 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:46.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.746 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:46.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:46.869 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:46.970 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:46.991 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:46.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:47.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:47.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:47.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:47.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:47.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:47.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:47.362 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:47.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:47.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /var/log/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/exp.6.12.2 2024-09-17T08:19:47.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:47.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:47.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:47.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:47.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:47.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:19:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:19:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: 2.a deep-scrub starts 2024-09-17T08:19:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: 2.a deep-scrub ok 2024-09-17T08:19:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: 2.a deep-scrub starts 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: 2.a deep-scrub ok 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:47.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:19:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:19:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: 2.a deep-scrub starts 2024-09-17T08:19:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: 2.a deep-scrub ok 2024-09-17T08:19:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:19:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:19:47.730 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:47.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:47.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:19:47.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:47.853 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:47.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:47.955 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:47.977 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:47.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:47.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:48.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:48.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:48.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:48.204 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:48.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:48.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:48.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:48.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:48.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:48.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:48.459 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: pgmap v421: 148 pgs: 29 active+undersized, 18 active+undersized+degraded, 101 active+clean; 181 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 956 op/s; 999/9228 objects degraded (10.826%) 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 6.1b deep-scrub starts 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[40649]: 6.1b deep-scrub ok 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:19:48.460 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:19:48.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:19:48.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: pgmap v421: 148 pgs: 29 active+undersized, 18 active+undersized+degraded, 101 active+clean; 181 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 956 op/s; 999/9228 objects degraded (10.826%) 2024-09-17T08:19:48.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 6.1b deep-scrub starts 2024-09-17T08:19:48.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:48 smithi079 ceph-mon[29565]: 6.1b deep-scrub ok 2024-09-17T08:19:48.485 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:48.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:48.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:48.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:48.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: pgmap v421: 148 pgs: 29 active+undersized, 18 active+undersized+degraded, 101 active+clean; 181 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 956 op/s; 999/9228 objects degraded (10.826%) 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 6.1b deep-scrub starts 2024-09-17T08:19:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:48 smithi173 ceph-mon[33187]: 6.1b deep-scrub ok 2024-09-17T08:19:48.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:48.737 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:48.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:48.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:48.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:48.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:48.940 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:19:48.941 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:19:48.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:48.966 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:19:48.966 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:19:48.966 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:19:48.983 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:19:48.983 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:19:48.985 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:48.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:48.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:49.000 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:19:49.001 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:19:49.017 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:19:49.017 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:19:49.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:19:49.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:19:49.050 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:19:49.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:19:49.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:49.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:49.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:49.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:49.237 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:49.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:49.338 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:19:49.338 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:19:49.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:19:49.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:19:49.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:49.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:49.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.365 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:49.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:49.487 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:49.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:49.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:49.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:49.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:49.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[40649]: 8.6 scrub starts 2024-09-17T08:19:49.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[40649]: 8.6 scrub ok 2024-09-17T08:19:49.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/299863821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:49.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[29565]: 8.6 scrub starts 2024-09-17T08:19:49.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[29565]: 8.6 scrub ok 2024-09-17T08:19:49.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/299863821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:49.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:49 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:19:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:49 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:19:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:49 smithi173 ceph-mon[33187]: 8.6 scrub starts 2024-09-17T08:19:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:49 smithi173 ceph-mon[33187]: 8.6 scrub ok 2024-09-17T08:19:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/299863821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:49.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:49.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:49.805 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:19:49.805 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:19:49.807 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:49.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:49.861 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:49.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:49.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:49.983 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:49.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:49.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:50.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:50.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:50.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:50.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:50.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:50.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:50.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:50.343 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:19:50.360 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:50.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:50.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:50.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:50.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:50.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:50.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:50.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2215803877' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[29565]: pgmap v422: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2215803877' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:50 smithi079 ceph-mon[40649]: pgmap v422: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:50.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2215803877' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:50 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:19:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:50 smithi173 ceph-mon[33187]: pgmap v422: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:50.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:50.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:50.802 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-17T08:19:50.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-17T08:19:50.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:50.867 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:50.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:50.969 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:50.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:50.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:50.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:51.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:51.117 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:51.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:51.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:51.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:51.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:51.369 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: osdmap e216: 8 total, 7 up, 8 in 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:51.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: osdmap e216: 8 total, 7 up, 8 in 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:19:51.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:51 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:51.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:51.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:51.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:51.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:51.619 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:51.647 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:51.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: osdmap e216: 8 total, 7 up, 8 in 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:19:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:51 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:19:51.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:51.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:19:51.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:51.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:51.873 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:51.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:51.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:51.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:51.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:52.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:52.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.119 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:52.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:52.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:52.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.242 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:52.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:52.343 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:52.369 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:52.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:52.405 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:52.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:52.487 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-17T08:19:52.488 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:19:52.488 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:19:52.488 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.423451 2024-09-17T08:19:52.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 0.4234512392916697 2024-09-17T08:19:52.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.494 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:52.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:52.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:52.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[40649]: 8.c deep-scrub starts 2024-09-17T08:19:52.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[40649]: 8.c deep-scrub ok 2024-09-17T08:19:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[40649]: pgmap v424: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 391 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[29565]: 8.c deep-scrub starts 2024-09-17T08:19:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[29565]: 8.c deep-scrub ok 2024-09-17T08:19:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:52 smithi079 ceph-mon[29565]: pgmap v424: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 391 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:52.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:52.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:52.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:52 smithi173 ceph-mon[33187]: 8.c deep-scrub starts 2024-09-17T08:19:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:52 smithi173 ceph-mon[33187]: 8.c deep-scrub ok 2024-09-17T08:19:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:52 smithi173 ceph-mon[33187]: pgmap v424: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 391 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:52.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:52.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:52.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:52.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:52.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:52.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:52.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:52.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:52.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:53.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:53.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:53.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:53.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:19:53.184 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:19:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:19:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:19:53.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:53.224 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:19:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:19:53.249 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:53.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:53.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:53.373 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:53.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:53.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:53.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:53.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:53.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:53.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:53.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:53.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:53.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:53.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:53.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:53.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:53.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:53.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:53.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:54.011 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:54.037 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:19:54.037 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:19:54.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:54.141 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:54.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:54.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.270 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:54.371 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:54.399 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:54.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:54.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.526 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:54 smithi079 ceph-mon[29565]: pgmap v425: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 388 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:54.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:54.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:54 smithi079 ceph-mon[40649]: pgmap v425: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 388 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:54.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:54.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:54.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:19:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:54 smithi173 ceph-mon[33187]: pgmap v425: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 388 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:54.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:54.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:54.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:54.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:54.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:54.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:54.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:55.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:55.024 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:55.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:55.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:19:55.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:55.148 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:55.149 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:19:55.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:55.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:55.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:55.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.291 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:55.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:55.313 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:19:55.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:55.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:55.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_1' removed 2024-09-17T08:19:55.515 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:55.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:55.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:55.638 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:55.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:55.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3691260156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:55.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:55.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:55.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3691260156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.818 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.818 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3691260156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.818 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:19:55.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:55.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:55.912 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:55.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:56.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:56.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:56.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:56.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:56.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:56.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:56.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:56.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:56.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:56.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:56.403 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:56.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:56.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:56.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:56.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.529 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:56.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.652 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:56.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[40649]: pgmap v426: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[40649]: osdmap e217: 8 total, 7 up, 8 in 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 47 pgs undersized (PG_DEGRADED) 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[29565]: pgmap v426: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:19:56.700 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[29565]: osdmap e217: 8 total, 7 up, 8 in 2024-09-17T08:19:56.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:56 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 47 pgs undersized (PG_DEGRADED) 2024-09-17T08:19:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:56 smithi173 ceph-mon[33187]: pgmap v426: 148 pgs: 34 active+undersized, 13 active+undersized+degraded, 101 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:19:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:56 smithi173 ceph-mon[33187]: osdmap e217: 8 total, 7 up, 8 in 2024-09-17T08:19:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:56 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 47 pgs undersized (PG_DEGRADED) 2024-09-17T08:19:56.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:56.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:56.777 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:56.878 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:56.899 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:56.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:56.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:57.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:57.023 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:57.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:57.025 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:19:57.026 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:19:57.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:57.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:57.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:57.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:57.268 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:57.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:57.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:57.391 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:57.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:57.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:57.516 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:57.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:57.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:57.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:19:57.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:57.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:57.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:57.765 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:57.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:57.824 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:19:57.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:57.888 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:57.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:57.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:57.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:58.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.012 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:58.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:58.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:58.135 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:58.236 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-17T08:19:58.237 INFO:tasks.radosbench:Beginning radosbench... 2024-09-17T08:19:58.237 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-17T08:19:58.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_2 16 2024-09-17T08:19:58.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:58.260 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:58.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:58.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:19:58.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:19:58.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:58.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:58.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:58.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:58 smithi173 ceph-mon[33187]: pgmap v428: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:58.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:19:58 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1333471188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:58.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:58.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:58.777 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:19:58.777 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:19:58.778 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:19:58.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:19:58.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:19:58.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:19:58.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:19:58.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:58 smithi079 ceph-mon[40649]: pgmap v428: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:58.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:19:58 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1333471188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:58.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:58 smithi079 ceph-mon[29565]: pgmap v428: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:19:58.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:19:58 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1333471188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:19:58.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:58.877 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:58.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:58.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:58.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:19:59.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:59.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:19:59.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:59.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:59.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:59.227 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:19:59.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:59.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:19:59.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:59.444 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:59.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:19:59.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:19:59.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:19:59.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:19:59.568 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:59.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:19:59.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:19:59.694 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:59.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:19:59.781 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:19:59.782 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:19:59.782 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:19:59.782 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:19:59.782 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:19:59.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:19:59.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:19:59.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:19:59.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:19:59.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:19:59.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:19:59.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:00.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:00.065 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:00.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:00.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:00.188 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:00.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:00.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:00.312 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:00.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:00.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:20:00.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:00.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:00.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pgmap v429: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1016 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: osdmap e218: 8 total, 7 up, 8 in 2024-09-17T08:20:00.534 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: Health detail: HEALTH_WARN noscrub flag(s) set; 1 osds down; Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: [WRN] OSD_DOWN: 1 osds down 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: osd.2 (root=default,host=smithi079) is down 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.1 is stuck undersized for 69s, current state active+undersized, last acting [3,0] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.3 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.7 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.c is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.e is stuck undersized for 69s, current state active+undersized, last acting [3,7] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.10 is stuck undersized for 69s, current state active+undersized, last acting [1,0] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.12 is stuck undersized for 69s, current state active+undersized, last acting [3,5] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.16 is stuck undersized for 69s, current state active+undersized, last acting [5,6] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.17 is stuck undersized for 69s, current state active+undersized+degraded, last acting [6,5] 2024-09-17T08:20:00.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 2.1c is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 3.0 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.4 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.6 is stuck undersized for 69s, current state active+undersized+degraded, last acting [0,1] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,1] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.f is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,4] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.12 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.18 is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,6] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.19 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,0] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 4.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.4 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.536 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.6 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pgmap v429: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1016 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: osdmap e218: 8 total, 7 up, 8 in 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: Health detail: HEALTH_WARN noscrub flag(s) set; 1 osds down; Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: [WRN] OSD_DOWN: 1 osds down 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: osd.2 (root=default,host=smithi079) is down 2024-09-17T08:20:00.537 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.1 is stuck undersized for 69s, current state active+undersized, last acting [3,0] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.3 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.7 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.c is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.e is stuck undersized for 69s, current state active+undersized, last acting [3,7] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.10 is stuck undersized for 69s, current state active+undersized, last acting [1,0] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.12 is stuck undersized for 69s, current state active+undersized, last acting [3,5] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.16 is stuck undersized for 69s, current state active+undersized, last acting [5,6] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.17 is stuck undersized for 69s, current state active+undersized+degraded, last acting [6,5] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 2.1c is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 3.0 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.4 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.538 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.6 is stuck undersized for 69s, current state active+undersized+degraded, last acting [0,1] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,1] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.f is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,4] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.12 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.18 is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,6] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.19 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,0] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 4.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.4 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.6 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.8 is stuck undersized for 69s, current state active+undersized, last acting [0,1] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.a is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.b is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.14 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,7] 2024-09-17T08:20:00.539 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.1c is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 5.1e is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.0 is stuck undersized for 69s, current state active+undersized, last acting [0,3] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.1 is stuck undersized for 69s, current state active+undersized, last acting [1,6] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.2 is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.3 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.8 is stuck undersized for 69s, current state active+undersized, last acting [7,3] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.9 is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.b is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.e is stuck undersized for 69s, current state active+undersized, last acting [4,1] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.f is stuck undersized for 69s, current state active+undersized, last acting [3,4] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.12 is stuck undersized for 69s, current state active+undersized, last acting [7,4] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.14 is stuck undersized for 69s, current state active+undersized, last acting [4,7] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.17 is stuck undersized for 69s, current state active+undersized, last acting [4,5] 2024-09-17T08:20:00.540 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[40649]: pg 6.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.8 is stuck undersized for 69s, current state active+undersized, last acting [0,1] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.a is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.b is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.14 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,7] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.1c is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 5.1e is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.0 is stuck undersized for 69s, current state active+undersized, last acting [0,3] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.1 is stuck undersized for 69s, current state active+undersized, last acting [1,6] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.2 is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.3 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.8 is stuck undersized for 69s, current state active+undersized, last acting [7,3] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.9 is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.b is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.e is stuck undersized for 69s, current state active+undersized, last acting [4,1] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.f is stuck undersized for 69s, current state active+undersized, last acting [3,4] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.12 is stuck undersized for 69s, current state active+undersized, last acting [7,4] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.14 is stuck undersized for 69s, current state active+undersized, last acting [4,7] 2024-09-17T08:20:00.542 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.17 is stuck undersized for 69s, current state active+undersized, last acting [4,5] 2024-09-17T08:20:00.543 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:00 smithi079 ceph-mon[29565]: pg 6.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:00.598 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:00.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:00.612 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:00.700 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:00.721 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:00.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pgmap v429: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1016 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: osdmap e218: 8 total, 7 up, 8 in 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: Health detail: HEALTH_WARN noscrub flag(s) set; 1 osds down; Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: [WRN] OSD_DOWN: 1 osds down 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: osd.2 (root=default,host=smithi079) is down 2024-09-17T08:20:00.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.1 is stuck undersized for 69s, current state active+undersized, last acting [3,0] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.3 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.7 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.c is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.e is stuck undersized for 69s, current state active+undersized, last acting [3,7] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.10 is stuck undersized for 69s, current state active+undersized, last acting [1,0] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.12 is stuck undersized for 69s, current state active+undersized, last acting [3,5] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.16 is stuck undersized for 69s, current state active+undersized, last acting [5,6] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.17 is stuck undersized for 69s, current state active+undersized+degraded, last acting [6,5] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 2.1c is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 3.0 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.4 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.6 is stuck undersized for 69s, current state active+undersized+degraded, last acting [0,1] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,1] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.f is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,4] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.12 is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,6] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.18 is stuck undersized for 69s, current state active+undersized+degraded, last acting [4,6] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.19 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,0] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 4.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [1,3] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.4 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.6 is stuck undersized for 69s, current state active+undersized, last acting [7,5] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.8 is stuck undersized for 69s, current state active+undersized, last acting [0,1] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.a is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.b is stuck undersized for 69s, current state active+undersized, last acting [0,5] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.d is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.14 is stuck undersized for 69s, current state active+undersized+degraded, last acting [3,7] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.1c is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 5.1e is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.0 is stuck undersized for 69s, current state active+undersized, last acting [0,3] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.1 is stuck undersized for 69s, current state active+undersized, last acting [1,6] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.2 is stuck undersized for 69s, current state active+undersized, last acting [4,3] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.3 is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.8 is stuck undersized for 69s, current state active+undersized, last acting [7,3] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.9 is stuck undersized for 69s, current state active+undersized, last acting [0,7] 2024-09-17T08:20:00.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.b is stuck undersized for 69s, current state active+undersized, last acting [7,6] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.e is stuck undersized for 69s, current state active+undersized, last acting [4,1] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.f is stuck undersized for 69s, current state active+undersized, last acting [3,4] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.12 is stuck undersized for 69s, current state active+undersized, last acting [7,4] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.14 is stuck undersized for 69s, current state active+undersized, last acting [4,7] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.17 is stuck undersized for 69s, current state active+undersized, last acting [4,5] 2024-09-17T08:20:00.729 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:00 smithi173 ceph-mon[33187]: pg 6.1c is stuck undersized for 69s, current state active+undersized+degraded, last acting [7,5] 2024-09-17T08:20:00.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:00.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:00.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:00.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:00.968 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:00.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:00.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:00.986 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:01.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:01.090 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:01.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:01.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:01.213 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:01.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:01.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:01.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:01.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:01.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:01.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.462 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:01.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:01.488 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:20:01.489 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 0.423451 (8277512) 2024-09-17T08:20:01.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:20:01.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:01.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:01.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:01.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:01.708 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:01.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:01.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3874455109' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:01 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:01 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:01.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:01 smithi173 ceph-mon[33187]: pgmap v431: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3874455109' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[40649]: pgmap v431: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3874455109' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]: dispatch 2024-09-17T08:20:01.732 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:01.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 41 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:01.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:01 smithi079 ceph-mon[29565]: pgmap v431: 132 pgs: 28 active+undersized, 13 active+undersized+degraded, 91 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:01.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:01.832 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:01.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:01.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:01.955 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:01.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:01.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:02.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:02.081 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:02.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:02.204 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:02.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:02.306 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:02.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.336 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30828 7.1 7:814ad74a:::benchmark_data_smithi079_83551_object30827:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 452.30487977799999, 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14898619199999999, 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30828 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029668+0000", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029666+0000", 2024-09-17T08:20:02.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029758+0000", 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1730000000000004e-05 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029772+0000", 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3888e-05 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029779+0000", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0060000000000003e-06 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029822+0000", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2682999999999998e-05 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.029949+0000", 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127482 2024-09-17T08:20:02.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.030066+0000", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000117027 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.032345+0000", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022790940000000002 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.035973+0000", 2024-09-17T08:20:02.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003627791 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178565+0000", 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14259147699999999 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2923999999999997e-05 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178655+0000", 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7166000000000003e-05 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30837 7.1 7:84e0e382:::benchmark_data_smithi079_83551_object30836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 452.29735145400002, 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14170434200000001, 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.343 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30837 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037197+0000", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037195+0000", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037307+0000", 2024-09-17T08:20:02.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111345 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037318+0000", 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1612e-05 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037326+0000", 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4959999999999999e-06 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037356+0000", 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0233e-05 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037437+0000", 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0385999999999999e-05 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.037519+0000", 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2231000000000006e-05 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.038523+0000", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001004158 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.039311+0000", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078787300000000002 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178816+0000", 2024-09-17T08:20:02.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13950497000000001 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178860+0000", 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4440000000000001e-05 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178901+0000", 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0961999999999998e-05 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.349 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68437 7.1 7:8794a2e1:::benchmark_data_smithi079_83551_object68436:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 427.57252544099998, 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11749947199999999, 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.350 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.351 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.351 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68437 2024-09-17T08:20:02.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.351 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762022+0000", 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762153+0000", 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131729 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762168+0000", 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4851999999999999e-05 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762174+0000", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6439999999999997e-06 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762209+0000", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5030999999999997e-05 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762299+0000", 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0492e-05 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762395+0000", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5126999999999999e-05 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764253+0000", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018581089999999999 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764600+0000", 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034766899999999999 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879354+0000", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11475331499999999 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879444+0000", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0832e-05 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879522+0000", 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7851999999999999e-05 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.356 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68447 7.1 7:81e8cead:::benchmark_data_smithi079_83551_object68446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 427.56636548799997, 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.116018159, 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.357 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68447 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:20:02.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768181+0000", 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768276+0000", 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5210999999999994e-05 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768289+0000", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2994e-05 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768296+0000", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8510000000000001e-06 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768336+0000", 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9638999999999998e-05 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768469+0000", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132881 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768636+0000", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016691399999999999 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771127+0000", 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002491643 2024-09-17T08:20:02.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771774+0000", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064712199999999998 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884106+0000", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11233185 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884160+0000", 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3644999999999999e-05 2024-09-17T08:20:02.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884201+0000", 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1152000000000002e-05 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68453 7.1 7:847278dd:::benchmark_data_smithi079_83551_object68452:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 427.562616822, 2024-09-17T08:20:02.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.115527932, 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68453 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771930+0000", 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772008+0000", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7849999999999995e-05 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772019+0000", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1488000000000001e-05 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772025+0000", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1270000000000002e-06 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772067+0000", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2552000000000002e-05 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772178+0000", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110838 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772272+0000", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3548999999999995e-05 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774206+0000", 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001934727 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774634+0000", 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00042775400000000002 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887387+0000", 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.112753373 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887423+0000", 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5494999999999998e-05 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887459+0000", 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6498999999999997e-05 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183015 7.1 7:881aa8e5:::benchmark_data_smithi079_83551_object55615:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:20:02.369 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 320.58460079499997, 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110799543, 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183015 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:20:02.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749946+0000", 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749953+0000", 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749968+0000", 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4815e-05 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749975+0000", 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7070000000000003e-06 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750019+0000", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4474000000000001e-05 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750110+0000", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0523e-05 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750189+0000", 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9081999999999999e-05 2024-09-17T08:20:02.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752660+0000", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024708320000000001 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752849+0000", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018942199999999999 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860669+0000", 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10781995900000001 2024-09-17T08:20:02.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860711+0000", 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2150000000000001e-05 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860747+0000", 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5818999999999997e-05 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183025 7.1 7:819884a4:::benchmark_data_smithi079_83551_object55625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 320.57551954299998, 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10716666699999999, 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183025 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759027+0000", 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759036+0000", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9169999999999995e-06 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759053+0000", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6585000000000001e-05 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759062+0000", 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0010000000000007e-06 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759107+0000", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5754000000000003e-05 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759217+0000", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110092 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759305+0000", 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7967999999999995e-05 2024-09-17T08:20:02.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761460+0000", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002154885 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761625+0000", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000164461 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866070+0000", 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.104445353 2024-09-17T08:20:02.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866155+0000", 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5018000000000005e-05 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866195+0000", 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0274000000000001e-05 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.381 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35137 8.2 8:468fa297:::benchmark_data_smithi079_115040_object35136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 198.991301413, 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.131430191, 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.382 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35137 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343245+0000", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343349+0000", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103587 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343366+0000", 2024-09-17T08:20:02.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7232999999999999e-05 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343373+0000", 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3030000000000002e-06 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343441+0000", 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7742000000000003e-05 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343563+0000", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012195499999999999 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343696+0000", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132664 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345071+0000", 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013757260000000001 2024-09-17T08:20:02.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345717+0000", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064516599999999997 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474616+0000", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12889890800000001 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474651+0000", 2024-09-17T08:20:02.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5364000000000002e-05 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474677+0000", 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6166e-05 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35143 8.2 8:48b7532d:::benchmark_data_smithi079_115040_object35142:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:20:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 198.98825945300001, 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13410802799999999, 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35143 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346288+0000", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346342+0000", 2024-09-17T08:20:02.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3986000000000001e-05 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346353+0000", 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1019000000000001e-05 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346361+0000", 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7889999999999995e-06 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346403+0000", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2738999999999999e-05 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346519+0000", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115482 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346671+0000", 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015182600000000001 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.347889+0000", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012181220000000001 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.349481+0000", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001591631 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480320+0000", 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.130839172 2024-09-17T08:20:02.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480362+0000", 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1878000000000001e-05 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480397+0000", 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5224e-05 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.394 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:36507 8.2 8:46403afa:::benchmark_data_smithi079_115040_object36506:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 198.18601427999999, 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14247363399999999, 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 36507 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:20:02.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148533+0000", 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148580+0000", 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7367999999999998e-05 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148593+0000", 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2622e-05 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148598+0000", 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6359999999999999e-06 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148634+0000", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5598e-05 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148725+0000", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0792999999999996e-05 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148820+0000", 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5533999999999997e-05 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.149972+0000", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001151727 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.150108+0000", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013644899999999999 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290944+0000", 2024-09-17T08:20:02.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14083580900000001 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290980+0000", 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5593000000000003e-05 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.291008+0000", 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7813e-05 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:139803 8.4 8:2f12791c:::benchmark_data_smithi079_115040_object4783:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.857076378, 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.103915984, 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 139803 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477471+0000", 2024-09-17T08:20:02.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477474+0000", 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4329999999999998e-06 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477481+0000", 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0249999999999997e-06 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477485+0000", 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5539999999999999e-06 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477562+0000", 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6897999999999995e-05 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477666+0000", 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477769+0000", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102635 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.480755+0000", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002986107 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.486665+0000", 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0059098589999999999 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581308+0000", 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.094643730999999995 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581355+0000", 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6374e-05 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581388+0000", 2024-09-17T08:20:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3050999999999999e-05 2024-09-17T08:20:02.407 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:139805 8.5 8:a4d9555f:::benchmark_data_smithi079_115040_object4785:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 118.856824214, 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.102633397, 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.408 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 139805 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477723+0000", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477728+0000", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2950000000000004e-06 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477737+0000", 2024-09-17T08:20:02.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6590000000000002e-06 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477743+0000", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9479999999999996e-06 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477778+0000", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5317000000000001e-05 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477914+0000", 2024-09-17T08:20:02.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013540199999999999 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.478013+0000", 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9325000000000002e-05 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.484825+0000", 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068114040000000001 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.488487+0000", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036620569999999998 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580242+0000", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091755428 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580316+0000", 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3919999999999997e-05 2024-09-17T08:20:02.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580357+0000", 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1443999999999998e-05 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:205896 8.2 8:48183680:::benchmark_data_smithi079_115040_object70876:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.183242047, 2024-09-17T08:20:02.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12536080999999999, 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 205896 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151305+0000", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151312+0000", 2024-09-17T08:20:02.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7580000000000003e-06 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151325+0000", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3677e-05 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151332+0000", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1840000000000002e-06 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151449+0000", 2024-09-17T08:20:02.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116208 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151569+0000", 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120178 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151680+0000", 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111673 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155861+0000", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041803700000000001 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155894+0000", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3634999999999999e-05 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276563+0000", 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12066801000000001 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276626+0000", 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3219999999999994e-05 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276667+0000", 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1294999999999997e-05 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:205897 8.2 8:4ad2b373:::benchmark_data_smithi079_115040_object70877:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.420 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.183196758000001, 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12573102799999999, 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 205897 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151351+0000", 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151354+0000", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.841e-06 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151363+0000", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5570000000000001e-06 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151368+0000", 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5870000000000003e-06 2024-09-17T08:20:02.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.153985+0000", 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026173799999999999 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.154064+0000", 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9078000000000004e-05 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.154138+0000", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3314999999999999e-05 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155918+0000", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001779751 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155991+0000", 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3767e-05 2024-09-17T08:20:02.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277006+0000", 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12101434899999999 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277047+0000", 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1094000000000003e-05 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277083+0000", 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5809999999999998e-05 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212175 8.2 8:42f488fd:::benchmark_data_smithi079_115040_object77155:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.699387692999998, 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10989600099999999, 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.427 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212175 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635159+0000", 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635168+0000", 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8929999999999994e-06 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635182+0000", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3987e-05 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635190+0000", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8939999999999993e-06 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636415+0000", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012254799999999999 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636516+0000", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010071199999999999 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636616+0000", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010017399999999999 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639458+0000", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028418710000000002 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.645864+0000", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064057109999999997 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.744926+0000", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099062219000000007 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745018+0000", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1884000000000003e-05 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745057+0000", 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8574e-05 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212180 8.2 8:49a422e1:::benchmark_data_smithi079_115040_object77160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.698069785999998, 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.108731759, 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.433 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212180 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636477+0000", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636483+0000", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8939999999999998e-06 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636495+0000", 2024-09-17T08:20:02.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2156e-05 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636502+0000", 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.725e-06 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639480+0000", 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029782770000000001 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639577+0000", 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7027000000000005e-05 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639676+0000", 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8287000000000003e-05 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642500+0000", 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002824769 2024-09-17T08:20:02.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648886+0000", 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063861100000000004 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745137+0000", 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096250496000000005 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745177+0000", 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0241000000000003e-05 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745210+0000", 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3050999999999999e-05 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212181 8.2 8:47e87e5d:::benchmark_data_smithi079_115040_object77161:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:20:02.439 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.698040337999998, 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10882399, 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212181 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:20:02.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636507+0000", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636510+0000", 2024-09-17T08:20:02.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.323e-06 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636515+0000", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2089999999999996e-06 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636519+0000", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2459999999999997e-06 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642517+0000", 2024-09-17T08:20:02.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00599783 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642629+0000", 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011173800000000001 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642749+0000", 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120536 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648894+0000", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0061442880000000004 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648925+0000", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1751999999999997e-05 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745256+0000", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096330490000000005 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745291+0000", 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5256999999999998e-05 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745332+0000", 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0760999999999998e-05 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212193 8.4 8:2d0a4b0d:::benchmark_data_smithi079_115040_object77173:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:20:02.445 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.690657405000003, 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.106655758, 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212193 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:20:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643890+0000", 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643893+0000", 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7329999999999999e-06 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643899+0000", 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1759999999999998e-06 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643904+0000", 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.028e-06 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.649054+0000", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0051491879999999999 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738041+0000", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.088987219000000006 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738172+0000", 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000130827 2024-09-17T08:20:02.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750356+0000", 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012184053 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750399+0000", 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3609000000000002e-05 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750452+0000", 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2630000000000003e-05 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750547+0000", 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4753000000000006e-05 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.451 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212197 8.2 8:4ffe1ab7:::benchmark_data_smithi079_115040_object77177:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.688342646999999, 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.107564755, 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212197 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646204+0000", 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:02.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646211+0000", 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2430000000000002e-06 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646220+0000", 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8119999999999997e-06 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646227+0000", 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7640000000000005e-06 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648948+0000", 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00272098 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.649078+0000", 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012928700000000001 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738858+0000", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.089780486000000007 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745099+0000", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0062409049999999997 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745222+0000", 2024-09-17T08:20:02.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000123495 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753521+0000", 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082982349999999993 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753639+0000", 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118638 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753770+0000", 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131023 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212199 8.2 8:4d5a0918:::benchmark_data_smithi079_115040_object77179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 56.686433024999999, 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11034735599999999, 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:02.458 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212199 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:20:02.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648114+0000", 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648122+0000", 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3650000000000004e-06 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:02.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648132+0000", 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0039e-05 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648138+0000", 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.699e-06 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741641+0000", 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093503257000000006 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741745+0000", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010346 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741848+0000", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102626 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745342+0000", 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034946619999999999 2024-09-17T08:20:02.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.751505+0000", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0061626529999999997 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758277+0000", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067723999999999996 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758427+0000", 2024-09-17T08:20:02.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015008299999999999 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758463+0000", 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5253000000000003e-05 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:02.464 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:02.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:02.487 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:02.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:02.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:02.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]': finished 2024-09-17T08:20:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: osdmap e219: 8 total, 7 up, 8 in 2024-09-17T08:20:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:02.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]': finished 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: osdmap e219: 8 total, 7 up, 8 in 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:02.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.4234512392916697}]': finished 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4124452707' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: osdmap e219: 8 total, 7 up, 8 in 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:02.808 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:02.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:02.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:02.961 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:02.982 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:02.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:02.983 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:03.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:03.106 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:03.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:03.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:03.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:03.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:20:03.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:03.249 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:03.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:03.341 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:03.350 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:03.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:03.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:03.474 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:03.496 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:03.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:03.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:03.620 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:03.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:03.714 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:03 smithi173 ceph-mon[33187]: osdmap e220: 8 total, 7 up, 8 in 2024-09-17T08:20:03.714 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:03.714 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:03 smithi173 ceph-mon[33187]: pgmap v434: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:03.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:03.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.744 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:03.744 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:03.747 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:03.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:03.866 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:03.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[40649]: osdmap e220: 8 total, 7 up, 8 in 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[40649]: pgmap v434: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[29565]: osdmap e220: 8 total, 7 up, 8 in 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:03.868 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:03 smithi079 ceph-mon[29565]: pgmap v434: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:03.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:03.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:03.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:03.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:04.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:04.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:04.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:04.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:04.237 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:04.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:04.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:04.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:04.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:04.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:04.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:04.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:04.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:04.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:04.679 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:04 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-17T08:20:04.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:04.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:04.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:04.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:20:04.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:04.849 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:04 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-17T08:20:04.850 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:04 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 5 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-17T08:20:04.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.857 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:04.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:04.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:04.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:04.982 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:04.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:05.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:05.104 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:05.206 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:05.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:05.298 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:05.299 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:20:05.299 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:20:05.299 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:05.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:05.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:05.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:05.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:05.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:05.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:05.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:05.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:05.720 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:05.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:05.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:05.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/265372492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/265372492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:05.944 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:05 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:20:05.945 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:05 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:20:05.945 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:05 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:20:05.945 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:05 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:05.945 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/265372492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:05.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:05.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:20:05.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:05.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:05.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:06.063 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:06.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:06.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:06.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:06.152 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_2' created 2024-09-17T08:20:06.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:06.213 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:06.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:06.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:06.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:06.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.345 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:06.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:06.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:06.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:06.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:06.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:06.592 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:06.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:06.681 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:06 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:06.681 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:06 smithi173 ceph-mon[33187]: pgmap v435: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 71 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:06.681 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:06 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-17T08:20:06.681 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:06 smithi173 ceph-mon[33187]: osdmap e221: 8 total, 7 up, 8 in 2024-09-17T08:20:06.681 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:06 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:06.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:06.716 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:06.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:06.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:06.844 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:06.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[29565]: pgmap v435: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 71 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-17T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[29565]: osdmap e221: 8 total, 7 up, 8 in 2024-09-17T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-17T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[40649]: pgmap v435: 132 pgs: 7 peering, 27 active+undersized, 13 active+undersized+degraded, 85 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 71 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-17T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[40649]: osdmap e221: 8 total, 7 up, 8 in 2024-09-17T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:06 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 59/588 objects degraded (10.034%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:06.946 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:06.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:06.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:06.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:07.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:07.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:07.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:07.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.214 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:07.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:07.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-17T08:20:07.337 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:07.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:07.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:07.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:07.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:07.586 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:07.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:07.710 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:07.804 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:20:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:20:07.804 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:20:07.804 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:07.813 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:07.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.836 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:07.836 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:07.851 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:20:07.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:20:07.867 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:20:07.867 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:20:07.867 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:20:07.882 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:20:07.883 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:20:07.899 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:20:07.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:20:07.915 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:20:07.915 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:20:07.931 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:20:07.931 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:20:07.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:07.947 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:20:07.948 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:20:07.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:07.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:07.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:08.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:08.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.083 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:08.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:08.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:08.187 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-17T08:20:08.187 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-17T08:20:08.187 INFO:tasks.cephadm.osd.2:Restarting osd.2 (starting--it wasn't running)... 2024-09-17T08:20:08.187 DEBUG:teuthology.orchestra.run.smithi079:osd.2> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T08:20:08.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl start ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2 2024-09-17T08:20:08.205 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:08.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:08.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:08.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:08.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:08.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:08.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:08.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:08.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[40649]: osdmap e222: 8 total, 7 up, 8 in 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[40649]: pgmap v438: 148 pgs: 12 unknown, 28 active+undersized, 13 active+undersized+degraded, 95 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2470830250' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[29565]: osdmap e222: 8 total, 7 up, 8 in 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[29565]: pgmap v438: 148 pgs: 12 unknown, 28 active+undersized, 13 active+undersized+degraded, 95 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:08.434 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2470830250' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:08.435 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:08 smithi079 systemd[1]: Starting Ceph osd.2 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:20:08.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:08.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:08.458 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:08 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:08.458 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:08 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:08.458 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:08 smithi173 ceph-mon[33187]: osdmap e222: 8 total, 7 up, 8 in 2024-09-17T08:20:08.458 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:08 smithi173 ceph-mon[33187]: pgmap v438: 148 pgs: 12 unknown, 28 active+undersized, 13 active+undersized+degraded, 95 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:08.458 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2470830250' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:08.478 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:20:08.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:08.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.576 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:08.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:08.676 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:08.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:08.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:08.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:08.802 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:08.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.824 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:08.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:08.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:08.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:08.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:08.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:09.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:09.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:09.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:09.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:09.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.196 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:09.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:09.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:09.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:09.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:09.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[40649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:20:09.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-09-17T08:20:09.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[40649]: osdmap e223: 8 total, 7 up, 8 in 2024-09-17T08:20:09.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[29565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:20:09.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-09-17T08:20:09.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:09 smithi079 ceph-mon[29565]: osdmap e223: 8 total, 7 up, 8 in 2024-09-17T08:20:09.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:09.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:09.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:09.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:09 smithi173 ceph-mon[33187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:20:09.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:09 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 7 pgs peering) 2024-09-17T08:20:09.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:09 smithi173 ceph-mon[33187]: osdmap e223: 8 total, 7 up, 8 in 2024-09-17T08:20:09.482 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:20:09.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:20:09.543 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:09.564 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:09.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:09.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:09.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.688 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:09.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:09.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:09.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:09.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:09.913 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:09.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:09.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:09.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:09.937 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:10.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:10.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:10.158 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:20:10.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:10.183 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:10.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:10.285 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:10.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:10.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:10 smithi079 ceph-mon[40649]: pgmap v440: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 49 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:10.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:20:10.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:10 smithi079 ceph-mon[29565]: pgmap v440: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 49 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:10.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:20:10.409 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:10.430 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:10 smithi173 ceph-mon[33187]: pgmap v440: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 49 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:10.430 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:20:10.431 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:10.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:10.619 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:20:10.619 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:20:10.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:10.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:10.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:10.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:10.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:10.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:10.924 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:10.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:10.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:11.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:11.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:11.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:11.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:11.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:11.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:11.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:11.301 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:11.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:11.389 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:11 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:11.390 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:11 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:20:11.390 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:11 smithi173 ceph-mon[33187]: osdmap e224: 8 total, 7 up, 8 in 2024-09-17T08:20:11.403 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:11.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:20:11.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:11.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:11.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[40649]: osdmap e224: 8 total, 7 up, 8 in 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2099149436' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:20:11.510 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-mon[29565]: osdmap e224: 8 total, 7 up, 8 in 2024-09-17T08:20:11.511 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 podman[142362]: 2024-09-17 08:20:11.257500558 +0000 UTC m=+1.049125751 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:20:11.511 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 podman[142362]: 2024-09-17 08:20:11.382219986 +0000 UTC m=+1.173845155 container create 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-17T08:20:11.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:11.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:11.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:11.632 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:11.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:11.672 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:11.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:11.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:11.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:11.804 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 podman[142362]: 2024-09-17 08:20:11.618432606 +0000 UTC m=+1.410057787 container init 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:20:11.804 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 podman[142362]: 2024-09-17 08:20:11.623732828 +0000 UTC m=+1.415358004 container start 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:20:11.804 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 podman[142362]: 2024-09-17 08:20:11.804830144 +0000 UTC m=+1.596455324 container attach 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, RELEASE=reef-19e2a2b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-17T08:20:11.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:11.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:11.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:11.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:12.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:12.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:12.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:12.120 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate[142459]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-17T08:20:12.121 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:11 smithi079 bash[142362]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-17T08:20:12.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:12.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:12.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:12.292 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:12.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:12.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:12.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:12.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:20:12.472 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-mon[33187]: pgmap v442: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 55 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:12.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:20:12.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:20:12.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:20:12.507 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[40649]: pgmap v442: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 55 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:20:12.508 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:12 smithi079 ceph-mon[29565]: pgmap v442: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 55 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:12.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:12.539 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:12.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:12.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:12.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:12.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:12.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:12.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.790 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:12.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:12.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:12.914 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:12.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:12.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:13.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:13.025 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:13.039 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:13.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:13.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:13.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:13.120 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:12 smithi079 podman[142362]: 2024-09-17 08:20:12.832046243 +0000 UTC m=+2.623671415 container died 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2) 2024-09-17T08:20:13.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:13.164 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:13.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:13.257 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:20:13.257 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:20:13.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:20:13.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:13.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:13.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:13.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:13.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:13.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:13.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:13 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:20:13.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:13 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:20:13.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:13 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:20:13.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:13 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:20:13.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:13.537 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:13.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:13.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:20:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:13 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:20:13.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:13.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.665 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:13.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:13.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:13.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:13.788 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:13.883 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:20:13.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:13.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:13.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:13.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:13.920 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:13 smithi079 podman[142680]: 2024-09-17 08:20:13.663747938 +0000 UTC m=+1.652121327 container remove 72ec4365c46b0c811f15c2c59486f6226a735984a788b227a0d50260dbca69a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T08:20:14.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:14.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:14.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:14.083 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:14.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:14.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.160 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:14.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:14.283 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: pgmap v443: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 47 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:20:14.360 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: pgmap v443: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 47 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:20:14.361 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:14 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:20:14.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:14.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:14.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: pgmap v443: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 47 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:20:14.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:14 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:20:14.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:14.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:14.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:14.654 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:14.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:14.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.779 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:14.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:14.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:14.903 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:14.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:15.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:15.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:15.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:15.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:15.152 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:15.171 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-17T08:20:15.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:20:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:15 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:20:15.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:15.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.277 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:15.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:20:15.364 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:20:15.365 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:15 smithi079 podman[143065]: 2024-09-17 08:20:15.130623144 +0000 UTC m=+0.953170242 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:20:15.365 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:15 smithi079 podman[143065]: 2024-09-17 08:20:15.240272788 +0000 UTC m=+1.062819872 container create 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:20:15.365 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:20:15.366 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:15 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:20:15.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:15.399 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:15.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:15.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:15.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.523 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:15.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:15.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:15.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:15.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:15.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:15.772 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:15.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:15.831 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:20:15.831 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:20:15.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:15.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:15.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:15.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:15.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:16.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.020 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:16.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:20:16.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:16.144 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:16.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:16.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:16.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:16.270 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: osdmap e225: 8 total, 7 up, 8 in 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: pgmap v445: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:20:16.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: osdmap e225: 8 total, 7 up, 8 in 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: pgmap v445: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:20:16.344 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:20:16.345 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 podman[143065]: 2024-09-17 08:20:16.005126555 +0000 UTC m=+1.827673640 container init 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:20:16.345 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 podman[143065]: 2024-09-17 08:20:16.010177678 +0000 UTC m=+1.832724763 container start 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-17T08:20:16.345 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-osd[143263]: -- 172.21.15.79:0/3764844902 <== mon.2 v2:172.21.15.79:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564935c431e0 con 0x564935be1c00 2024-09-17T08:20:16.345 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 bash[143065]: 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d 2024-09-17T08:20:16.345 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 systemd[1]: Started Ceph osd.2 for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:20:16.361 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:20:16.361 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:20:16.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:20:16.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:20:16.362 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-17T08:20:16.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:16.376 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:20:16.376 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:20:16.392 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:16.393 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:20:16.393 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:20:16.410 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:20:16.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:20:16.426 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:20:16.426 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:20:16.434 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1396474601' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: osdmap e225: 8 total, 7 up, 8 in 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: pgmap v445: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:20:16.435 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:16 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:20:16.442 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:20:16.443 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:20:16.448 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:16.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:20:16.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:16.520 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:16.619 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:16 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:20:16.547+0000 7fd5ec7a5640 -1 Falling back to public interface 2024-09-17T08:20:16.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:16.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:16.655 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:16.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:16.767 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:16.868 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:16.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:16.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:16.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:16.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:17.014 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:17.116 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:17.138 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:17.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:17.212 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:17.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:17.263 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:17.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:17.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:17.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:20:17.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:20:17.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:20:17.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2349364662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:17.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:20:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:20:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:20:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2349364662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:17.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:17.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:17.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:17.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:17 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:20:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:17 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:20:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:17 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:20:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:17 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2349364662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:17.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:17.490 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:17.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:17.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:17.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:17.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:17.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:17.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:17.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:17.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:17.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:17.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:17.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:17.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:17.953 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:20:17.953 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:20:17.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:18.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:18.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:18.129 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:18.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:18.140 DEBUG:teuthology.orchestra.run.smithi079:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-17T08:20:18.143 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-17T08:20:18.143 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-17T08:20:18.193 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:18 smithi079 ceph-mon[29565]: pgmap v446: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:18.194 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:18 smithi079 ceph-mon[40649]: pgmap v446: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:18.194 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:hints = 1 2024-09-17T08:20:18.194 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-17T08:20:18.194 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object prefix: benchmark_data_smithi079_143589 2024-09-17T08:20:18.195 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:20:18.195 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 0 0 0 0 0 0 - 0 2024-09-17T08:20:18.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:18.261 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:18.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:18.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:18.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:18.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:18.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:18 smithi173 ceph-mon[33187]: pgmap v446: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 582 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/588 objects degraded (10.034%) 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28716 7.7 7:edba59f3:::benchmark_data_smithi079_83551_object28715:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:20:18.483 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 469.50436454200002, 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.063303783000000002, 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28716 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975926+0000", 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.975925+0000", 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976073+0000", 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014827699999999999 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976086+0000", 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2918000000000001e-05 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976096+0000", 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0256e-05 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976145+0000", 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9243999999999998e-05 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976258+0000", 2024-09-17T08:20:18.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011240500000000001 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.976379+0000", 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121612 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.977391+0000", 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001011685 2024-09-17T08:20:18.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033061+0000", 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055669996999999999 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039074+0000", 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0060129399999999996 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039129+0000", 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4897e-05 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039229+0000", 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100722 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.490 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:28722 7.7 7:e67c57be:::benchmark_data_smithi079_83551_object28721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 469.50105032099998, 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.060131948999999997, 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 28722 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979240+0000", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979239+0000", 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979357+0000", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011785099999999999 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979367+0000", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0438999999999999e-05 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979374+0000", 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0400000000000004e-06 2024-09-17T08:20:18.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979415+0000", 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1279999999999998e-05 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979502+0000", 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6149999999999993e-05 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.979586+0000", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4332999999999994e-05 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:28.980492+0000", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090632099999999997 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.033104+0000", 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.052611766999999997 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039310+0000", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0062060930000000002 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039339+0000", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8484999999999999e-05 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.039372+0000", 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3337000000000003e-05 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.496 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29213 7.c 7:3e440d7d:::benchmark_data_smithi079_83551_object29212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 469.23175941099998, 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057673473000000003, 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.497 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.498 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.498 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29213 2024-09-17T08:20:18.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248531+0000", 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248530+0000", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248900+0000", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000370066 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248909+0000", 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9749999999999996e-06 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248914+0000", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8389999999999997e-06 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.248950+0000", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.595e-05 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249061+0000", 2024-09-17T08:20:18.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110703 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249260+0000", 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000199182 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252421+0000", 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031614400000000002 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252454+0000", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3216999999999997e-05 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306147+0000", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.053692036999999998 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306179+0000", 2024-09-17T08:20:18.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2332000000000001e-05 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.306204+0000", 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5429999999999999e-05 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:29221 7.c 7:3e2e93ed:::benchmark_data_smithi079_83551_object29220:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:20:18.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 469.230361969, 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.058164050000000002, 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 29221 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:20:18.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249928+0000", 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.249927+0000", 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250103+0000", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581399999999999 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250116+0000", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3127e-05 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.250124+0000", 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7419999999999997e-06 2024-09-17T08:20:18.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251107+0000", 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098316799999999998 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251265+0000", 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015859700000000001 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:20:18.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.251404+0000", 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000138496 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252513+0000", 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011087200000000001 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.252648+0000", 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000135081 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308041+0000", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055393851000000001 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308072+0000", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0790999999999997e-05 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:29.308092+0000", 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0032000000000001e-05 2024-09-17T08:20:18.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30807 7.7 7:e1a9b60f:::benchmark_data_smithi079_83551_object30806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45926193499997, 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15767790800000001, 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30807 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021028+0000", 2024-09-17T08:20:18.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021027+0000", 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021092+0000", 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5506000000000006e-05 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021104+0000", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1472e-05 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021115+0000", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1080000000000001e-05 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021160+0000", 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5367000000000001e-05 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021239+0000", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8437999999999997e-05 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.021321+0000", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2540999999999995e-05 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022432+0000", 2024-09-17T08:20:18.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001110457 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025758+0000", 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003326552 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178623+0000", 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.152864151 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178669+0000", 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6896e-05 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178706+0000", 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6767000000000002e-05 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30809 7.d 7:b03b464e:::benchmark_data_smithi079_83551_object30808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.517 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45769572099999, 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15486507099999999, 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30809 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022595+0000", 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022593+0000", 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022677+0000", 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3442999999999994e-05 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022694+0000", 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7031e-05 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022702+0000", 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.028e-06 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022744+0000", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2491999999999999e-05 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022838+0000", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3326999999999996e-05 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.022969+0000", 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131068 2024-09-17T08:20:18.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.023921+0000", 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00095237200000000005 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175140+0000", 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15121870300000001 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177383+0000", 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022433729999999999 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177423+0000", 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9823000000000001e-05 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177460+0000", 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6634999999999997e-05 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30813 7.7 7:e02af656:::benchmark_data_smithi079_83551_object30812:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45623809900002, 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15475839, 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30813 2024-09-17T08:20:18.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024052+0000", 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024046+0000", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999938 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024174+0000", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000128133 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024192+0000", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8287999999999998e-05 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024200+0000", 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3900000000000004e-06 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024259+0000", 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9525999999999999e-05 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024392+0000", 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132946 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024549+0000", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015661000000000001 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178587+0000", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.154037904 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178743+0000", 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015638800000000001 2024-09-17T08:20:18.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178750+0000", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5570000000000001e-06 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178783+0000", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3241999999999998e-05 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178811+0000", 2024-09-17T08:20:18.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7668999999999999e-05 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30816 7.d 7:bfa0a9cd:::benchmark_data_smithi079_83551_object30815:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45607745199999, 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.153399859, 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.530 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30816 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024213+0000", 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024212+0000", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024268+0000", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5563000000000003e-05 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024283+0000", 2024-09-17T08:20:18.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5176e-05 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024290+0000", 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0079999999999996e-06 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024333+0000", 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2933999999999999e-05 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024447+0000", 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114708 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024706+0000", 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025843399999999999 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177353+0000", 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15264659 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177528+0000", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017581300000000001 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177538+0000", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6139999999999998e-06 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177581+0000", 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2889999999999998e-05 2024-09-17T08:20:18.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177613+0000", 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1819999999999997e-05 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45604196900001, 2024-09-17T08:20:18.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084112399999999, 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024248+0000", 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024247+0000", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024253+0000", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3569999999999997e-06 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024263+0000", 2024-09-17T08:20:18.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0562e-05 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024279+0000", 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5896000000000002e-05 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025745+0000", 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001465767 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025753+0000", 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9100000000000005e-06 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175089+0000", 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.149336513 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30809 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:20:18.540 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.455719053, 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15057779399999999, 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:18.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:20:18.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024571+0000", 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024570+0000", 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024602+0000", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1566000000000002e-05 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024612+0000", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9510000000000001e-06 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024618+0000", 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2609999999999999e-06 2024-09-17T08:20:18.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175046+0000", 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.150427808 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175130+0000", 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3523000000000007e-05 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.545 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175149+0000", 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9442000000000001e-05 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30817 7.c 7:3940e3c2:::benchmark_data_smithi079_83551_object30816:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45568902299999, 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15284112799999999, 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.546 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30817 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.547 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024601+0000", 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024590+0000", 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999989 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024982+0000", 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000392063 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.548 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.024994+0000", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1858000000000001e-05 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025005+0000", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0288999999999999e-05 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025057+0000", 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2791999999999999e-05 2024-09-17T08:20:18.549 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025172+0000", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011491499999999999 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,7", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025294+0000", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012128 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174906+0000", 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961192100000001 2024-09-17T08:20:18.550 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177330+0000", 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002424538 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177361+0000", 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1254999999999998e-05 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.551 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177406+0000", 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4196000000000002e-05 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177442+0000", 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6826999999999998e-05 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30807 7.7 e98/91 ondisk, result = 0)", 2024-09-17T08:20:18.552 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45449007399998, 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15291164700000001, 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025800+0000", 2024-09-17T08:20:18.553 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025799+0000", 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.025805+0000", 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1129999999999997e-06 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.554 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175419+0000", 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14961376500000001 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175431+0000", 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2131e-05 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178612+0000", 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031804939999999999 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.555 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178618+0000", 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9100000000000002e-06 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.178712+0000", 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4178999999999994e-05 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24698.0:30816 7.d e98/91 ondisk, result = 0)", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45377799099998, 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15084441900000001, 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.557 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026512+0000", 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026517+0000", 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5160000000000002e-06 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.558 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174997+0000", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14847996599999999 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.175009+0000", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1552999999999999e-05 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177340+0000", 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023309110000000002 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.559 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177347+0000", 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3560000000000004e-06 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177357+0000", 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7340000000000003e-06 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.560 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:30820 7.d 7:b5459376:::benchmark_data_smithi079_83551_object30819:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 468.45342799000002, 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15420378100000001, 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 30820 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.561 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026862+0000", 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026861+0000", 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.562 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.026944+0000", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3053999999999995e-05 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174921+0000", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14797671100000001 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.174933+0000", 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1926000000000001e-05 2024-09-17T08:20:18.563 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176058+0000", 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011258220000000001 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176161+0000", 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102187 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:20:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.176275+0000", 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114832 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.177625+0000", 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001349996 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.179664+0000", 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002038467 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.565 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.180981+0000", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013172679999999999 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181030+0000", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9128999999999997e-05 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:30.181066+0000", 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5720999999999999e-05 2024-09-17T08:20:18.566 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:137290 8.6 e193/165 8:6be15e8a:::benchmark_data_smithi079_115040_object2270:head v 193'8528, mlcod=192'8526)", 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 137.022668778, 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.078277779000000006, 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.567 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.568 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457631+0000", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0086e-05 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457661+0000", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0505999999999999e-05 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457674+0000", 2024-09-17T08:20:18.569 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2907e-05 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457787+0000", 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112409 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457801+0000", 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4423000000000001e-05 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.570 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535864+0000", 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.078062520999999996 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535899+0000", 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5722999999999997e-05 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:137294 8.7 e193/165 8:ec500667:::benchmark_data_smithi079_115040_object2274:head v 193'8664, mlcod=192'8662)", 2024-09-17T08:20:18.571 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 137.01065957099999, 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069534631, 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:20:18.572 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469630+0000", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469634+0000", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7230000000000002e-06 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535475+0000", 2024-09-17T08:20:18.573 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.065841199000000003 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535491+0000", 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5633000000000001e-05 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535544+0000", 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2677999999999999e-05 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.574 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535564+0000", 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0066999999999999e-05 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.539079+0000", 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003515308 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.575 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.539165+0000", 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6195999999999999e-05 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:144399 8.7 e196/165 8:eb4c6be9:::benchmark_data_smithi079_115040_object9379:head v 196'9087, mlcod=196'9083)", 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 131.389057296, 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091227230000000006, 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.576 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.577 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091232+0000", 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091240+0000", 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2179999999999999e-06 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091248+0000", 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4850000000000003e-06 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.578 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091253+0000", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0170000000000004e-06 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091286+0000", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3386000000000002e-05 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091300+0000", 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3349999999999999e-05 2024-09-17T08:20:18.579 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182423+0000", 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091123327000000004 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182460+0000", 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7219999999999999e-05 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.580 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:144403 8.5 e196/165 8:a8e6c9af:::benchmark_data_smithi079_115040_object9383:head v 196'9006, mlcod=196'9004)", 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 131.38745005300001, 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099445130000000007, 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.581 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092839+0000", 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.582 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092847+0000", 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9810000000000003e-06 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182416+0000", 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.089568712999999994 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182427+0000", 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1003e-05 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.583 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182468+0000", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1108e-05 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182485+0000", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6549999999999999e-05 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.192241+0000", 2024-09-17T08:20:18.584 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0097565510000000005 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.192285+0000", 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4254e-05 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24377.0:575 4.a 4:55b0279d:reshard::reshard.0000000009:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:20:18.585 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.849536144999998, 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.87298637199999995, 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24377", 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3352227429", 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 575 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.586 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630753+0000", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630758+0000", 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6380000000000001e-06 2024-09-17T08:20:18.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630769+0000", 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1270999999999999e-05 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630774+0000", 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0549999999999999e-06 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.588 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630825+0000", 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0308e-05 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for peered", 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630832+0000", 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4100000000000002e-06 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.496920+0000", 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.86608754099999996 2024-09-17T08:20:18.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497058+0000", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013873499999999999 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497248+0000", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018980700000000001 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497885+0000", 2024-09-17T08:20:18.590 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00063677199999999997 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497916+0000", 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1127999999999997e-05 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503609+0000", 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056926040000000004 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.591 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503683+0000", 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4165999999999994e-05 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503740+0000", 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7522000000000003e-05 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.592 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24377.0:594 4.a 4:50d64734:::obj_delete_at_hint.0000000009:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.591069117, 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.61468610300000004, 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24377", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3352227429", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 594 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:20:18.593 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889219+0000", 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.594 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889225+0000", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7610000000000004e-06 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889236+0000", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0321e-05 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889240+0000", 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8509999999999998e-06 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.595 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889280+0000", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9335000000000002e-05 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for peered", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889286+0000", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0650000000000004e-06 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497469+0000", 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.60818299799999997 2024-09-17T08:20:18.596 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497557+0000", 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.776e-05 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497664+0000", 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000107094 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:20:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497949+0000", 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028577799999999999 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.498102+0000", 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000152256 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503812+0000", 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005710499 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.598 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503870+0000", 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8198000000000001e-05 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503907+0000", 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6820000000000003e-05 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:20:18.599 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:18.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:18.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:18.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:18.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:18.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:18.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:18.902 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:18.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:18.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:19.004 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:19.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.039 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:19.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:19.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:19.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:19.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:19.195 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 1 16 3935 3919 244.989 244.938 0.00508469 0.00406191 2024-09-17T08:20:19.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:19.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:19.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:19.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:19.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:19.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:19 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:20:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:19 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:20:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:19 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:20:19.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:19 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:20:19.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:19.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:19.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:20:19.596 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:19 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:20:19.685 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:19.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:19.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:19.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:19.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:19.853 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:19.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:19.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:20:19.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:20.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:20.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:20.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:20.109 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:20:20.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.147 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:20.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:20.196 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 2 15 6487 6472 202.193 159.562 0.0064448 0.00492923 2024-09-17T08:20:20.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:20.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.289 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:20.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:20.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:20.408 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:20.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.428 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:20.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:20.473 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[40649]: pgmap v447: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 677 KiB/s wr, 11 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:20.473 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:20:20.473 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:20 smithi173 ceph-mon[33187]: pgmap v447: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 677 KiB/s wr, 11 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:20 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:20 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[29565]: pgmap v447: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 677 KiB/s wr, 11 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:20:20.474 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:20:20.475 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:20 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:20:20.362+0000 7fd5ec7a5640 -1 osd.2 204 log_to_monitors true 2024-09-17T08:20:20.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:20.562 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:20.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:20.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:20.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.703 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:20.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:20.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:20.837 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:no valid command found; 10 closest matches: 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:0 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:1 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:2 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:abort 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:assert 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs debug_inject_read_zeros 2024-09-17T08:20:20.920 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs files list 2024-09-17T08:20:20.921 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs stats 2024-09-17T08:20:20.921 INFO:teuthology.orchestra.run.smithi079.stderr:bluestore allocator dump block 2024-09-17T08:20:20.921 INFO:teuthology.orchestra.run.smithi079.stderr:bluestore allocator fragmentation block 2024-09-17T08:20:20.921 INFO:teuthology.orchestra.run.smithi079.stderr:admin_socket: invalid command 2024-09-17T08:20:20.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:20.974 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:20.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:20.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:21.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:21.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:21.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:21.196 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 3 16 9308 9292 193.541 176.25 0.00348949 0.00515875 2024-09-17T08:20:21.212 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:21.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:21.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:21.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:21.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.363 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:21.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:21.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:20:21.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 59/882 objects degraded (6.689%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:21.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:20:21.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:21 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:20:21.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:21.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:21.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:21.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:21.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:21.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:20:21.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:20:21.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 59/882 objects degraded (6.689%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:21.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 59/882 objects degraded (6.689%), 13 pgs degraded, 40 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:20:21.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:20:21.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:21.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:21.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:21.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:21.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:21.892 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:21.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:21.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:22.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:22.064 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:22.119 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:20:21 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:20:21.804+0000 7fd5e0547640 -1 osd.2 204 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:20:22.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:22.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.180 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:22.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:22.197 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 4 16 10949 10933 170.761 102.562 0.0049805 0.00584775 2024-09-17T08:20:22.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:22.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:22.377 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:20:22.377 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:20:22.377 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:20:22.377 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: osdmap e226: 8 total, 7 up, 8 in 2024-09-17T08:20:22.377 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: pgmap v449: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 784 KiB/s wr, 12 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:22.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2918293874' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.378 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-mon[33187]: from='osd.2 ' entity='osd.2' 2024-09-17T08:20:22.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:22.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:22.476 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:20:22.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: osdmap e226: 8 total, 7 up, 8 in 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.478 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: pgmap v449: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 784 KiB/s wr, 12 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2918293874' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[40649]: from='osd.2 ' entity='osd.2' 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:20:22.479 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: osdmap e226: 8 total, 7 up, 8 in 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: pgmap v449: 148 pgs: 31 active+undersized, 13 active+undersized+degraded, 104 active+clean; 6.7 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 784 KiB/s wr, 12 op/s; 59/882 objects degraded (6.689%) 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2918293874' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:22.480 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:22 smithi079 ceph-mon[29565]: from='osd.2 ' entity='osd.2' 2024-09-17T08:20:22.535 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:20:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:20:22.535 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:22.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:22.572 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:22.674 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:22.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.696 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:22.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:22.819 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:22.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:22.942 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:22.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:22.953 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T08:20:22.953 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T08:20:23.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:23.070 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:23.172 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:23.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:23.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:23.197 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 5 16 11758 11742 146.727 50.5625 0.00156746 0.00586005 2024-09-17T08:20:23.218 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:23.231 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:23.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:20:23.295 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:23.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.347 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:23.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:23.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:23.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:20:23.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:20:23.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:20:23.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782] boot 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T08:20:23.476 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:23 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:23.489 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782] boot 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T08:20:23.510 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: osd.2 [v2:172.21.15.79:6818/359954782,v1:172.21.15.79:6819/359954782] boot 2024-09-17T08:20:23.511 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T08:20:23.512 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:23 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T08:20:23.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:23.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.599 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:23.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:23.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:23.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:23.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:23.824 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:23.846 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:23.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:23.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:23.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:23.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:24.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:24.093 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:24.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:24.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:24.197 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 6 16 11758 11742 122.277 0 - 0.00586005 2024-09-17T08:20:24.216 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:24.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:24.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:24.339 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:24.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:24.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:24.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:24.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:24.547 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:24.547 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:24.547 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: pgmap v451: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T08:20:24.548 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:20:24.549 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: pgmap v451: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:20:24.550 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:24 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: pgmap v451: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:24.555 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T08:20:24.556 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:20:24.556 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:24 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:20:24.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:24.586 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:24.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:24.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:24.726 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:24.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:24.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:24.857 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:24.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:24.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:24.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:25.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:25.109 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:25.141 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:25.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:25.197 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 7 16 13192 13176 117.613 44.8125 0.00592545 0.008494 2024-09-17T08:20:25.212 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:20:25.212 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:20:25.212 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:20:25.213 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:20:25.213 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:20:25.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:25.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:25.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:25.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:25.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:25.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:25.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:25 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:20:25.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:25 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:20:25.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:25 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:20:25.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:25 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:20:25.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:25 smithi173 ceph-mon[33187]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T08:20:25.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:25.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:25.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:25.587 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:20:25.587 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:20:25.587 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:20:25.587 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[40649]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:20:25.588 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:25 smithi079 ceph-mon[29565]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T08:20:25.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:20:25.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:20:25.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:25.675 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:20:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:20:25.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:25.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:25.706 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:20:25.707 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:20:25.745 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:20:25.745 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:20:25.776 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:20:25.776 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:20:25.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:25.809 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:20:25.809 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:20:25.822 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:25.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:25.841 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:20:25.841 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:20:25.872 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:20:25.872 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:20:25.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:20:25.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:25.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:20:25.960 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:25.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:25.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:26.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:26.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.109 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:26.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:26.197 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 8 16 15319 15303 119.527 132.938 0.0097614 0.00835834 2024-09-17T08:20:26.206 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:20:26.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:26.247 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:26.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:26.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:26.375 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:26.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:26.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:26.516 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:26.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:26.517 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:26 smithi173 ceph-mon[33187]: 6.9 deep-scrub starts 2024-09-17T08:20:26.518 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:26 smithi173 ceph-mon[33187]: 6.9 deep-scrub ok 2024-09-17T08:20:26.518 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:26 smithi173 ceph-mon[33187]: pgmap v454: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 731 B/s rd, 161 MiB/s wr, 2.58k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:26.518 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:26 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:20:26.518 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2984929148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:26.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[29565]: 6.9 deep-scrub starts 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[29565]: 6.9 deep-scrub ok 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[29565]: pgmap v454: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 731 B/s rd, 161 MiB/s wr, 2.58k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2984929148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[40649]: 6.9 deep-scrub starts 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[40649]: 6.9 deep-scrub ok 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[40649]: pgmap v454: 148 pgs: 19 peering, 20 active+undersized, 5 active+undersized+degraded, 104 active+clean; 683 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 731 B/s rd, 161 MiB/s wr, 2.58k op/s; 18/33324 objects degraded (0.054%) 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:20:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2984929148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:26.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:26.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:26.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:26.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:26.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:26.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:26.934 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:26.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:26.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:27.036 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:27.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:27.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:27.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:27.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 9 16 17591 17575 122.016 142 0.00544659 0.008187 2024-09-17T08:20:27.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.219 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:27.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:27.221 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:27.321 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:27.350 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:27.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:27.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:27 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:20:27.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:27 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:27.356 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:27 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:20:27.356 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:27 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:27.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:27.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:27 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:20:27.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:27 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized (PG_DEGRADED) 2024-09-17T08:20:27.490 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:27.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:27.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:27.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:27.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:27.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:27.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:27.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:27.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:27.899 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:27.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:27.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:27.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:20:28.001 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:28.032 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:28.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:28.164 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:28.189 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:20:28.189 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:20:28.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 10 16 19754 19738 123.332 135.188 0.0102315 0.00810224 2024-09-17T08:20:28.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:28.295 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:28.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:28.418 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:28.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:28 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:20:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:28 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:20:28.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:28 smithi173 ceph-mon[33187]: pgmap v455: 148 pgs: 19 peering, 1 active+undersized, 128 active+clean; 788 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 130 MiB/s wr, 2.08k op/s; 0 B/s, 1 objects/s recovering 2024-09-17T08:20:28.520 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:28.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:28.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:20:28.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:20:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[40649]: pgmap v455: 148 pgs: 19 peering, 1 active+undersized, 128 active+clean; 788 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 130 MiB/s wr, 2.08k op/s; 0 B/s, 1 objects/s recovering 2024-09-17T08:20:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:20:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:20:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:28 smithi079 ceph-mon[29565]: pgmap v455: 148 pgs: 19 peering, 1 active+undersized, 128 active+clean; 788 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 130 MiB/s wr, 2.08k op/s; 0 B/s, 1 objects/s recovering 2024-09-17T08:20:28.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:28.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:28.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:28.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:28.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:20:28.812 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:28.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:28.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:28.944 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:28.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:29.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:29.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:29.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:20:29.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:29.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 11 16 21330 21314 121.074 98.5 0.0136736 0.00825026 2024-09-17T08:20:29.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:29.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:29.318 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:29.346 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:29.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:29.447 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:29.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:29.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:29.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:29.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:29.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:29.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:20:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[40649]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized) 2024-09-17T08:20:29.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:20:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:20:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[29565]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized) 2024-09-17T08:20:29.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:29 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:20:29.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:29 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:20:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:29 smithi173 ceph-mon[33187]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/33324 objects degraded (0.054%), 5 pgs degraded, 23 pgs undersized) 2024-09-17T08:20:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:29 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:20:29.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:29.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:29.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:29.863 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:29.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:29.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:29.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:30.013 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:30.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:30.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:30.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:30.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:30.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 12 16 23230 23214 120.88 118.75 0.010895 0.00826698 2024-09-17T08:20:30.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:30.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:30.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:30.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:30.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:30.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:30 smithi173 ceph-mon[33187]: 6.0 deep-scrub starts 2024-09-17T08:20:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:30 smithi173 ceph-mon[33187]: pgmap v456: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 147 MiB/s wr, 2.36k op/s; 1011/58737 objects degraded (1.721%); 11 MiB/s, 177 objects/s recovering 2024-09-17T08:20:30.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:30.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:30.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:30.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:30 smithi079 ceph-mon[40649]: 6.0 deep-scrub starts 2024-09-17T08:20:30.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:30 smithi079 ceph-mon[40649]: pgmap v456: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 147 MiB/s wr, 2.36k op/s; 1011/58737 objects degraded (1.721%); 11 MiB/s, 177 objects/s recovering 2024-09-17T08:20:30.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:30 smithi079 ceph-mon[29565]: 6.0 deep-scrub starts 2024-09-17T08:20:30.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:30 smithi079 ceph-mon[29565]: pgmap v456: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 147 MiB/s wr, 2.36k op/s; 1011/58737 objects degraded (1.721%); 11 MiB/s, 177 objects/s recovering 2024-09-17T08:20:30.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:30.680 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:30.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:30.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:30.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:30.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:30.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:30.984 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:30.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:31.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:31.118 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:31.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 13 15 25391 25376 121.974 135.125 0.0115303 0.00819256 2024-09-17T08:20:31.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:31.249 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:31.267 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:31.350 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:31.376 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:31.413 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:31 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:20:31.413 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:31 smithi079 ceph-mon[40649]: Health check failed: Degraded data redundancy: 1011/58737 objects degraded (1.721%), 3 pgs degraded (PG_DEGRADED) 2024-09-17T08:20:31.413 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:31 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:20:31.414 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:31 smithi079 ceph-mon[29565]: Health check failed: Degraded data redundancy: 1011/58737 objects degraded (1.721%), 3 pgs degraded (PG_DEGRADED) 2024-09-17T08:20:31.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:31.501 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:31.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:31.603 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:31.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:31 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:20:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:31 smithi173 ceph-mon[33187]: Health check failed: Degraded data redundancy: 1011/58737 objects degraded (1.721%), 3 pgs degraded (PG_DEGRADED) 2024-09-17T08:20:31.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:31.749 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:31.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:31.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:31.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:31.974 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:31.997 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:31.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:31.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:32.099 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:32.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:32.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:32.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 14 16 25695 25679 114.615 18.9375 0.00174585 0.00818546 2024-09-17T08:20:32.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:32.246 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:32.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:32.297 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:20:32.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:32.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: pgmap v458: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 67 MiB/s wr, 1.07k op/s; 1011/58737 objects degraded (1.721%); 9.3 MiB/s, 154 objects/s recovering 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: 2.12 deep-scrub starts 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: 2.12 deep-scrub ok 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: pgmap v458: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 67 MiB/s wr, 1.07k op/s; 1011/58737 objects degraded (1.721%); 9.3 MiB/s, 154 objects/s recovering 2024-09-17T08:20:32.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: 2.12 deep-scrub starts 2024-09-17T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: 2.12 deep-scrub ok 2024-09-17T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:32.375 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:32.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.377 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:32.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:32.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:32.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:32.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:32.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:32.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: pgmap v458: 148 pgs: 3 active+clean+scrubbing, 3 active+recovering+undersized+degraded+remapped, 142 active+clean; 1.2 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 67 MiB/s wr, 1.07k op/s; 1011/58737 objects degraded (1.721%); 9.3 MiB/s, 154 objects/s recovering 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: 2.12 deep-scrub starts 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: 2.12 deep-scrub ok 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:32.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:32.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:32.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:32.856 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:32.878 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:32.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:32.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:32.980 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:33.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:33.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:33.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:33.129 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:33.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:33.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 15 16 26073 26057 108.549 23.625 0.00172105 0.00875862 2024-09-17T08:20:33.224 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:33.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:33.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:33.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:33.295 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:33.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:33.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.393 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:33.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:33.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:33.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:33.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:33.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:20:33.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[40649]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:33.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:20:33.602 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:33 smithi079 ceph-mon[29565]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T08:20:33.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:33.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:33.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:33.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2127939194' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:20:33.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:33 smithi173 ceph-mon[33187]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T08:20:33.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:33.806 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:33.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:33.908 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:33.940 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:33.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:33.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:34.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:34.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:34.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:34.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 16 16 28216 28200 110.135 133.938 0.00832439 0.00907427 2024-09-17T08:20:34.206 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:34.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:34.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:34.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:34.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:34.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:34.605 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.605 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:34.605 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[40649]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T08:20:34.606 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[40649]: pgmap v461: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.17k op/s; 344/77430 objects degraded (0.444%); 19 MiB/s, 314 objects/s recovering 2024-09-17T08:20:34.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:34.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[29565]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T08:20:34.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:34 smithi079 ceph-mon[29565]: pgmap v461: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.17k op/s; 344/77430 objects degraded (0.444%); 19 MiB/s, 314 objects/s recovering 2024-09-17T08:20:34.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:34.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:34 smithi173 ceph-mon[33187]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T08:20:34.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:34 smithi173 ceph-mon[33187]: pgmap v461: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.17k op/s; 344/77430 objects degraded (0.444%); 19 MiB/s, 314 objects/s recovering 2024-09-17T08:20:34.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:34.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.747 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:34.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:34.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:34.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:34.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:34.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.018 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:35.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:35.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:35.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:35.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:35.198 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 17 16 30626 30610 112.516 150.625 0.00482407 0.00888184 2024-09-17T08:20:35.252 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:35.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:35.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:35.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:35.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[40649]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:35.440 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:35.441 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[29565]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T08:20:35.441 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:35.441 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:35.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:35.564 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:35.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:35.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:35.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:35 smithi173 ceph-mon[33187]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T08:20:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:35.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:35.841 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:35.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:35.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:35.979 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:35.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:35.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:36.082 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:36.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:36.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:36.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 18 16 32870 32854 114.056 140.25 0.00293252 0.00876174 2024-09-17T08:20:36.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:36.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.246 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:36.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:36.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:36.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:36.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:36.476 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:36.503 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:36.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:36.600 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:20:36.600 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:20:36.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:36 smithi079 ceph-mon[40649]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T08:20:36.600 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:36 smithi079 ceph-mon[40649]: pgmap v464: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 344/77430 objects degraded (0.444%); 10 MiB/s, 166 objects/s recovering 2024-09-17T08:20:36.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:36 smithi079 ceph-mon[29565]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T08:20:36.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:36 smithi079 ceph-mon[29565]: pgmap v464: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 344/77430 objects degraded (0.444%); 10 MiB/s, 166 objects/s recovering 2024-09-17T08:20:36.605 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:36.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:36.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:36.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:20:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:36 smithi173 ceph-mon[33187]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T08:20:36.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:36 smithi173 ceph-mon[33187]: pgmap v464: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.6 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 344/77430 objects degraded (0.444%); 10 MiB/s, 166 objects/s recovering 2024-09-17T08:20:36.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:36.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:36.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:36.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:36.920 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:36.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:36.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:37.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:37.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:37.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:37.152 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:37.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:37.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:37.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 19 15 34890 34875 114.7 126.312 0.00524869 0.00871326 2024-09-17T08:20:37.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:37.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:37.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:37.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:37.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.444 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:37.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:37.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:37.585 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:37.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:37 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded (PG_DEGRADED) 2024-09-17T08:20:37.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:37 smithi079 ceph-mon[40649]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T08:20:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:37 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded (PG_DEGRADED) 2024-09-17T08:20:37.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:37 smithi079 ceph-mon[29565]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T08:20:37.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:37.706 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:37.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:37.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:37 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded (PG_DEGRADED) 2024-09-17T08:20:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:37 smithi173 ceph-mon[33187]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T08:20:37.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:37.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:37.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:37.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:37.977 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:37.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:37.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:38.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:38.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:38.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:38.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:20:38.199671+0000 min lat: 0.00129253 max lat: 2.12531 avg lat: 0.00868074 2024-09-17T08:20:38.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:20:38.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 20 16 36862 36846 115.124 123.188 0.0045574 0.00868074 2024-09-17T08:20:38.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:38.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:38.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:38.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:38.377 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:38.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:38.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:38.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.501 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:38.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:38.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:38.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:38 smithi079 ceph-mon[29565]: pgmap v466: 148 pgs: 148 active+clean; 1.8 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s wr, 1.05k op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-17T08:20:38.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:38 smithi079 ceph-mon[40649]: pgmap v466: 148 pgs: 148 active+clean; 1.8 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s wr, 1.05k op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-17T08:20:38.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:38.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:38.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:38 smithi173 ceph-mon[33187]: pgmap v466: 148 pgs: 148 active+clean; 1.8 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s wr, 1.05k op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-17T08:20:38.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:38.771 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:38.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:38.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:38.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:38.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:38.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:38.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:39.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.030 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:39.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:39.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:39.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:39.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 21 16 38443 38427 114.347 98.8125 0.00222257 0.00873533 2024-09-17T08:20:39.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:39.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:39.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:39.399 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:39.430 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:39.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:39.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:39.561 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[29565]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded) 2024-09-17T08:20:39.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:39.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[40649]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded) 2024-09-17T08:20:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:39.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:39 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:39.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:39.696 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:39.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:39 smithi173 ceph-mon[33187]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 344/77430 objects degraded (0.444%), 1 pg degraded) 2024-09-17T08:20:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:39 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:39.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:39 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:39.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:39.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:39.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:39.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:39.975 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:39.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:39.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:40.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:40.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:40.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:40.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 22 16 40322 40306 114.487 117.438 0.00822739 0.00872979 2024-09-17T08:20:40.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:40.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.251 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:40.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:40.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:40.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:40.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:40.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:40.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:40.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:40.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:20:40.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:40 smithi079 ceph-mon[29565]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T08:20:40.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:40 smithi079 ceph-mon[29565]: pgmap v468: 148 pgs: 148 active+clean; 2.2 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1002 B/s rd, 130 MiB/s wr, 2.08k op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-17T08:20:40.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:40 smithi079 ceph-mon[40649]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T08:20:40.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:40 smithi079 ceph-mon[40649]: pgmap v468: 148 pgs: 148 active+clean; 2.2 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1002 B/s rd, 130 MiB/s wr, 2.08k op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-17T08:20:40.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:40.654 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:40.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:40.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:40 smithi173 ceph-mon[33187]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T08:20:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:40 smithi173 ceph-mon[33187]: pgmap v468: 148 pgs: 148 active+clean; 2.2 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1002 B/s rd, 130 MiB/s wr, 2.08k op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-17T08:20:40.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:40.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.806 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:40.806 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:40.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:40.943 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:40.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:40.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:41.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:41.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:41.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:41.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:41.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 23 15 42234 42219 114.707 119.562 0.00776562 0.00871271 2024-09-17T08:20:41.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:41.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:41.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:41.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.364 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:41.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:41.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:41.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:41.598 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:41.628 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:41.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:41.720 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:20:41.721 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:20:41.730 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:41.736 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:20:41.737 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:20:41.751 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:41.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:41.753 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:20:41.753 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:20:41.767 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:20:41.768 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:20:41.787 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:20:41.787 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:20:41.819 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:20:41.819 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:20:41.851 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:20:41.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:20:41.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:41.886 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:20:41.886 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:20:41.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:41.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:41.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:41.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:42.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:42.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:42.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:42.199 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 24 16 43935 43919 114.354 106.25 0.00729042 0.00873965 2024-09-17T08:20:42.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:42.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:42.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:42.404 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:42.437 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:42.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.438 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:42.539 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:42.562 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:42.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.572 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:42.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[40649]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T08:20:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[40649]: pgmap v470: 148 pgs: 148 active+clean; 2.2 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.87k op/s; 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:20:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2541395785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[29565]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T08:20:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[29565]: pgmap v470: 148 pgs: 148 active+clean; 2.2 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.87k op/s; 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:20:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2541395785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:42.674 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:42.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.705 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:42.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:42 smithi173 ceph-mon[33187]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T08:20:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:42 smithi173 ceph-mon[33187]: pgmap v470: 148 pgs: 148 active+clean; 2.2 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.87k op/s; 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:20:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2541395785' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:42.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:42.845 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:42.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:42.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:42.978 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:42.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:42.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:43.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:43.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:43.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:43.200 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 25 16 45716 45700 114.228 111.312 0.00637456 0.00874899 2024-09-17T08:20:43.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:43.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:43.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:43.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:43.303 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:43.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:43.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:43.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:43.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:43.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:43.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:43.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:43.582 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:43 smithi079 ceph-mon[29565]: pgmap v471: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.67k op/s 2024-09-17T08:20:43.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:43.582 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:43 smithi079 ceph-mon[40649]: pgmap v471: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.67k op/s 2024-09-17T08:20:43.629 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:43.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:43.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:43.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:43.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:43.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:43.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:43.936 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:43.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:43.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:43 smithi173 ceph-mon[33187]: pgmap v471: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 2.67k op/s 2024-09-17T08:20:44.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:44.069 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:44.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:44.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:44.201 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 26 16 47586 47570 114.326 116.875 0.0099791 0.00874205 2024-09-17T08:20:44.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:44.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:44.306 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:44.337 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:44.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:44.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:44.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.472 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:44.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:44.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:44.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.619 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:44.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:44.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:44.747 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:44.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:48571 9.6 9.4e44f9a6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e238)", 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:44.750698+0000", 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00012643100000000001, 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000154647, 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:20:44.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48571 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750698+0000", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750698+0000", 2024-09-17T08:20:44.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:20:44.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750696+0000", 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750721+0000", 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5182e-05 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750733+0000", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1499e-05 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750740+0000", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6539999999999999e-06 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:44.750824+0000", 2024-09-17T08:20:44.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3442000000000005e-05 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-17T08:20:44.757 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:20:44.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:44.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:44.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:44.989 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:45.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:45.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:45.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:20:45.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.153 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:45.202 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 27 16 49385 49369 114.253 112.438 0.013956 0.00874759 2024-09-17T08:20:45.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:45.450 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.451 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:45.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:45.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:45.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:45.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:45.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:45.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:45.867 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[40649]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[40649]: pgmap v473: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 844 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[29565]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T08:20:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:45 smithi079 ceph-mon[29565]: pgmap v473: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 844 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:20:45.969 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:45 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:45 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:45 smithi173 ceph-mon[33187]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T08:20:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:45 smithi173 ceph-mon[33187]: pgmap v473: 148 pgs: 148 active+clean; 2.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 844 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:20:45.998 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:45.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:45.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:46.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:46.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:46.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:46.202 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 28 16 51161 51145 114.136 111 0.00932016 0.00875667 2024-09-17T08:20:46.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:46.278 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:46.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:46.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:46.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:46.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:46.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:46.556 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:46.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.557 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:46.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:46.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.699 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:46.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:46.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:46.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:46.950 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:46.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:46.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:46.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:47.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:47.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:47.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:47.202 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 29 16 52886 52870 113.918 107.812 0.0105254 0.00877344 2024-09-17T08:20:47.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:47.253 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:47.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:47.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:47.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:47.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:47.485 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:47.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:47.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:47.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[40649]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T08:20:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[29565]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T08:20:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:47.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.653 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:47.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:47.687 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:20:47.687 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:20:47.687 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:20:47.687 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:20:47.687 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:20:47.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:47 smithi173 ceph-mon[33187]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T08:20:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:20:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:20:47.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:47.797 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:47.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:47.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:47.935 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:47.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:47.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:48.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:48.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:48.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:48.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:48.202 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:48.203 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 30 16 54425 54409 113.326 96.1875 0.0171719 0.00881879 2024-09-17T08:20:48.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:48.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:48.340 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:48.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:48.379 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:20:48.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:48.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.476 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:48.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:48.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:48.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-17T08:20:48.606 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:48.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:48 smithi173 ceph-mon[33187]: pgmap v475: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 842 B/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-17T08:20:48.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:48.679 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:48 smithi079 ceph-mon[40649]: pgmap v475: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 842 B/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-17T08:20:48.679 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:48.680 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:48 smithi079 ceph-mon[29565]: pgmap v475: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 842 B/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-17T08:20:48.680 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:20:48.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:48.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:48.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:48.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:48.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:48.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:48.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:48.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:49.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.020 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:49.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:49.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:49.161 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:49.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:49.202 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 31 16 55969 55953 112.783 96.5 0.00820422 0.00886093 2024-09-17T08:20:49.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:49.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:49.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:49.399 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:49.436 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:49.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:49.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:49.577 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:49.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:49.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:20:49.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:49.695 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:49.696 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:49.696 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[29565]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T08:20:49.696 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:49.696 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:49.696 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:49 smithi079 ceph-mon[40649]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T08:20:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:49 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:20:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1860296639' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:20:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:49 smithi173 ceph-mon[33187]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T08:20:49.758 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:49.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.759 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:49.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:49.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:49.909 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:49.909 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:50.010 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:50.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:50.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:50.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:50.187 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:50.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 32 16 57336 57320 111.92 85.4375 0.0140297 0.0089301 2024-09-17T08:20:50.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:50.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:50.278 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:50.281 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:20:50.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:50.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:50.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:50.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:50.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:50.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:50.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:50.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:50.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:50.613 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:50 smithi079 ceph-mon[40649]: pgmap v477: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:20:50.613 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:50 smithi079 ceph-mon[29565]: pgmap v477: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:20:50.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:50 smithi173 ceph-mon[33187]: pgmap v477: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:20:50.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:50.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:50.888 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:50.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:50.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:50.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:51.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:51.120 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:51.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:51.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 33 16 58732 58716 111.172 87.25 0.0056739 0.00899033 2024-09-17T08:20:51.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:51.285 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:51.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:51.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:51.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:51.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:51.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:51.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:51.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:51.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:51.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.831 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:51.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:51.932 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:51.955 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:51.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:51.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:52.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:52.093 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:52.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:52.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:20:52.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:52.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 34 16 60267 60251 110.724 95.9375 0.00781776 0.00902703 2024-09-17T08:20:52.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:52.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:52.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:52.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:52.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:52.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:52.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.501 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:52.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:52.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:52 smithi079 ceph-mon[29565]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T08:20:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:52 smithi079 ceph-mon[29565]: pgmap v479: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-17T08:20:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:52 smithi079 ceph-mon[40649]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T08:20:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:52 smithi079 ceph-mon[40649]: pgmap v479: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-17T08:20:52.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:52.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:52 smithi173 ceph-mon[33187]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T08:20:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:52 smithi173 ceph-mon[33187]: pgmap v479: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-17T08:20:52.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:52.758 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:52.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.759 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:52.860 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:52.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:52.897 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:52.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:52.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:53.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:53.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:53.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:53.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:53.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 35 16 61759 61743 110.225 93.25 0.0118706 0.00906814 2024-09-17T08:20:53.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:20:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:20:53.279 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:53.309 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:53.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:20:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:20:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:20:53.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:53.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.451 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:53.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:53.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:53.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:53.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:53.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:53.722 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:53.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:53.824 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:53.856 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:53.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:53 smithi079 ceph-mon[29565]: pgmap v480: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.4 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-17T08:20:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:53 smithi079 ceph-mon[40649]: pgmap v480: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.4 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-17T08:20:53.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:20:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:53 smithi173 ceph-mon[33187]: pgmap v480: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.4 KiB/s rd, 137 MiB/s wr, 2.19k op/s 2024-09-17T08:20:53.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:53.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:53.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:54.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:54.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.117 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:54.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:54.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 36 16 63020 63004 109.352 78.8125 0.109168 0.00912172 2024-09-17T08:20:54.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:54.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.250 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:54.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:54.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:54.393 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:54.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:54.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:54.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:54.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:54.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:20:54.672 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:54.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:20:54.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:54.806 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:54.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:54.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:54.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:54.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:55.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:55.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:55.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:55.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:55.196 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:55.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:55.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 37 16 64433 64417 108.783 88.3125 0.00729897 0.00918757 2024-09-17T08:20:55.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:55.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:55.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:55.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:55.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.458 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:55.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:55.559 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:55.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:55.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:55.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:55.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:55.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:55.852 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:55.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:55 smithi079 ceph-mon[29565]: pgmap v481: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:20:55.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:55 smithi079 ceph-mon[40649]: pgmap v481: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:20:55.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:55 smithi173 ceph-mon[33187]: pgmap v481: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-17T08:20:55.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:55.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:55.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:56.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:56.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:56.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:56.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 38 16 65708 65692 108.018 79.6875 0.0145261 0.00925242 2024-09-17T08:20:56.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:56.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:56.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:56.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:56.376 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:56.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:56.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:56.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:56.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:56.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:56.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.662 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:56.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:56.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:56.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:56.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.793 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:56 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:20:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:56 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:20:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:56 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:20:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:56 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:20:56.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:56.930 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:56.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:56.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:56 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:20:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:56 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:20:57.032 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:57.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:57.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:57.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:57.187 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:57.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:57.205 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 39 16 67057 67041 107.41 84.3125 0.00812084 0.00930567 2024-09-17T08:20:57.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:57.320 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:57.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:57.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:57.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:57.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:57.601 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:57.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:57.607 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:57 smithi079 ceph-mon[29565]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T08:20:57.608 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:57 smithi079 ceph-mon[29565]: pgmap v483: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:20:57.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:57 smithi079 ceph-mon[40649]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T08:20:57.608 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:57 smithi079 ceph-mon[40649]: pgmap v483: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:20:57.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:57.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.758 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:57.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:57.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:20:57.899 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:57.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:57.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:20:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:57 smithi173 ceph-mon[33187]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T08:20:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:57 smithi173 ceph-mon[33187]: pgmap v483: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:20:58.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:58.035 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:58.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:58.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:58.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.175 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:58.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:58.206 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:20:58.206354+0000 min lat: 0.00129253 max lat: 2.12531 avg lat: 0.00933653 2024-09-17T08:20:58.206 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:20:58.206 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 40 15 68537 68522 107.039 92.5625 0.00600653 0.00933653 2024-09-17T08:20:58.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:58.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:58.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:58.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:58.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:58.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:58.489 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:20:58.489 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:20:58.535 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:20:58.535 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:20:58.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:20:58.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:20:58.567 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:20:58.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:58.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:20:58.606 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:20:58.607 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:20:58.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:20:58.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:20:58.668 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:58 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4269898954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:58.669 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:20:58.670 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:20:58.670 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:58 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4269898954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:58.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:20:58.701 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:20:58.702 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:20:58.726 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:58.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:20:58.732 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:20:58.733 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:20:58.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:58.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:58.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:58.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:58 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4269898954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:20:58.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:58.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:58.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:59.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:59.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.131 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:59.206 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 41 16 69967 69951 106.606 89.3125 0.00766662 0.00937589 2024-09-17T08:20:59.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:59.264 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:59.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:20:59.403 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:20:59.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:20:59.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:20:59.538 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:20:59.639 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:59.670 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:59.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:20:59.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:20:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:20:59 smithi079 ceph-mon[40649]: pgmap v484: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-17T08:20:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:20:59 smithi079 ceph-mon[29565]: pgmap v484: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-17T08:20:59.916 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:20:59.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:20:59.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:20:59.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:20:59.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:20:59 smithi173 ceph-mon[33187]: pgmap v484: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-17T08:21:00.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:00.086 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:00.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:00.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:00.209 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 42 16 71454 71438 106.272 92.9375 0.0109385 0.0094045 2024-09-17T08:21:00.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.217 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:00.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:00.318 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:00.352 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:00.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:00.454 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:00.487 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:00.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:00.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:00.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.630 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:00.630 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:00.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:00.759 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:00.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:00.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:00.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:00.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:00.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:01.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:01.047 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:01.050 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:21:01.050 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:21:01.149 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:01.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:01.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:01.209 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 43 16 72810 72794 105.772 84.75 0.0113296 0.00944907 2024-09-17T08:21:01.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:01.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:01.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:01.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:01.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:01.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:01.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.597 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:01.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:01.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:01.732 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:01.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:01.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:21:01.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:01.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:01.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:01.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:01.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:02.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:02.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:02.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:02.156 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:02.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.157 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:02.209 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 44 16 74120 74104 105.228 81.875 0.00715534 0.00949823 2024-09-17T08:21:02.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:02.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:02.387 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T08:21:02.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:02.429 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:02.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:02.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:02.566 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:02.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[40649]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T08:21:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[40649]: pgmap v486: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 910 op/s 2024-09-17T08:21:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2496203191' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[29565]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T08:21:02.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[29565]: pgmap v486: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 910 op/s 2024-09-17T08:21:02.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2496203191' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:02.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:02.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:02.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-mon[33187]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T08:21:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-mon[33187]: pgmap v486: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 910 op/s 2024-09-17T08:21:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2496203191' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:21:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:02.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:02.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:02.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:02.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:02.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:02.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:02.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:03.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:03.091 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:03.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:03.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:03.210 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 45 16 75451 75435 104.735 83.1875 0.0106904 0.00954267 2024-09-17T08:21:03.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:03.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:03.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:03.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:03.376 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:03.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:03.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.516 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:03.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:03.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:03.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:03.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:03 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:21:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:03 smithi173 ceph-mon[33187]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T08:21:03.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:03.780 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:03.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:03 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:21:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:03 smithi079 ceph-mon[29565]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T08:21:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:03 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:21:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:03 smithi079 ceph-mon[40649]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T08:21:03.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:03.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:03.914 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:03.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:04.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:04.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.049 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:04.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:04.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:04.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.181 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:04.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:04.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 46 16 76778 76762 104.262 82.9375 0.00910131 0.00958675 2024-09-17T08:21:04.282 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:04.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:04.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:04.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:04.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:04.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:04.486 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:04.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:21:04.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:04.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.621 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:04.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:04.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:04.661 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:04 smithi079 ceph-mon[29565]: pgmap v488: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:21:04.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:04.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:04 smithi079 ceph-mon[40649]: pgmap v488: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:21:04.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:04 smithi173 ceph-mon[33187]: pgmap v488: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:21:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:04.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:04.867 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:04.896 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:04.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:04.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:04.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:05.044 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:05.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:05.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:05.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:05.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:05.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 47 16 78010 77994 103.682 77 0.00340222 0.009638 2024-09-17T08:21:05.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:05.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:05.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:05.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:05.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.455 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:05.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:05.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:05.595 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:05.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:05.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:05 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:05 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:05 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:21:05.724 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:05.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:05.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:05.858 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:05.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:05.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:05.990 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:05.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:05.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:06.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:06.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:06.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:06.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 48 16 79220 79204 103.097 75.625 0.00844461 0.00969451 2024-09-17T08:21:06.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:06.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:06.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:06.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:06.415 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:06.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:06.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:06.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:06.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:06.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: pgmap v489: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:06.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[40649]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: pgmap v489: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:06.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:06 smithi079 ceph-mon[29565]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T08:21:06.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:06.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:06.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: pgmap v489: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:06 smithi173 ceph-mon[33187]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T08:21:06.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:06.817 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:06.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.818 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:06.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-17T08:21:06.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:06.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:06.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:06.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:07.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:07.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.100 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:07.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:07.193 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:07.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:07.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 49 16 80551 80535 102.691 83.1875 0.0212955 0.00973284 2024-09-17T08:21:07.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:07.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:07.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:07.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:07.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:07.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:07.493 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:07.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:07.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:07.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:07 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi173 to 3248M 2024-09-17T08:21:07.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:07 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi173 to 3248M 2024-09-17T08:21:07.627 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:07.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:07 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi173 to 3248M 2024-09-17T08:21:07.729 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:07.762 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:07.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:07.864 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:07.889 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:07.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:07.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:07.992 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:08.026 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:08.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:08.128 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:08.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.163 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:08.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:08.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 50 16 81830 81814 102.236 79.9375 0.0104344 0.00977608 2024-09-17T08:21:08.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:08.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:08.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:08.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:08.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:08.582 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:08.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:08.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:08.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:08.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: pgmap v491: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 842 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: Adjusting osd_memory_target on smithi079 to 3248M 2024-09-17T08:21:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:08.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:08.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: pgmap v491: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 842 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: Adjusting osd_memory_target on smithi079 to 3248M 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: pgmap v491: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 842 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: Adjusting osd_memory_target on smithi079 to 3248M 2024-09-17T08:21:08.734 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:21:08.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:21:08.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:08.824 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:08.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:08.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:08.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:08.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:08.995 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:08.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:09.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:09.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.133 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:09.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:09.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 51 16 83068 83052 101.748 77.375 0.0215959 0.00982352 2024-09-17T08:21:09.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:09.269 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:09.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.270 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:09.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:09.406 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:09.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:21:09.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:09.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.538 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:09.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:09.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:21:09.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:21:09.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:09.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:09.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:09.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:21:09.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:09.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:09.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:09.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:09.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:09.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:09.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:10.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:10.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:10.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:10.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:10.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 52 16 84412 84396 101.407 84 0.0121903 0.00985651 2024-09-17T08:21:10.217 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:10.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:10.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:10.366 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:10.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:10.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:10.498 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:10.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:10.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:10.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:10.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: pgmap v492: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:21:10.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: pgmap v492: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:21:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:21:10.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:10 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:21:10.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:10.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: pgmap v492: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3299213513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:21:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:10 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:21:10.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:10.749 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:10.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:10.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:10.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:10.910 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:10.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:10.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:11.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:11.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:11.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:11.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:11.184 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:11.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:11.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 53 16 85760 85744 101.083 84.25 0.0116706 0.00988799 2024-09-17T08:21:11.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:11.320 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:11.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:11.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:11.466 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:11.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:11.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:11.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:11.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:11.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:21:11.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:21:11.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:21:11.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[29565]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:21:11.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:11 smithi079 ceph-mon[40649]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T08:21:11.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:11 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:21:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:11 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:21:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:11 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:21:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:11 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:21:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:11 smithi173 ceph-mon[33187]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T08:21:11.747 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:11.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:11.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:11.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:11.885 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:11.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:11.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:12.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:12.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:12.154 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:12.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.156 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:12.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 54 16 87053 87037 100.708 80.8125 0.0119122 0.00992481 2024-09-17T08:21:12.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:12.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:12.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.291 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:12.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:21:12.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:12.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:12.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:12.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:12.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:12.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:12.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:12.706 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:12.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: pgmap v495: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:21:12.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:21:12.808 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:12.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.842 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:12.842 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:21:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: pgmap v495: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: pgmap v495: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:21:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:21:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:12 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:21:12.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:12.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:12.977 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:12.978 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:13.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:13.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:13.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:13.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 55 16 88313 88297 100.308 78.75 0.0109594 0.00996455 2024-09-17T08:21:13.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:13.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:13.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:13.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:13.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:13.367 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:13.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:13.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:13.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:13.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.527 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:13.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:21:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:21:13.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:21:13.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:21:13.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:21:13.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:13 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:21:13.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:13.666 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:13.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:21:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:21:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:21:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:21:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:21:13.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:13 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:21:13.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:13.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:13.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:21:13.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:21:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:21:13.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:13.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:13.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:13.941 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:14.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:14.069 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:14.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:14.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:14.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.205 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:14.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:14.211 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 56 16 89595 89579 99.9479 80.125 0.00823074 0.0100006 2024-09-17T08:21:14.306 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:14.342 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:14.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:14.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:14.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:14.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:14.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.620 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:14.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:14.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:14 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:21:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:14 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:21:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:14 smithi173 ceph-mon[33187]: pgmap v496: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:21:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:14 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1799515766' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:14.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:14.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:21:14.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:21:14.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[40649]: pgmap v496: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1799515766' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[29565]: pgmap v496: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:21:14.760 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:14 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1799515766' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:14.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:14.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:14.863 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:14.899 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:14.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:14.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:15.001 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:15.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:15.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:15.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:15.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:15.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:15.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 57 16 90925 90909 99.6517 83.125 0.0112053 0.0100291 2024-09-17T08:21:15.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:15.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:15.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:15.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:15.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:15.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:15.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:15.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:15.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:15.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:21:15.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:21:15.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:21:15.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:21:15.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:15 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:21:15.749 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:15.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:15.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:15.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:15.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:15.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:15.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:21:16.189 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:21:16.190 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:21:16.190 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:21:16.190 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:21:16.190 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:15 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:21:16.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:16.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 58 16 92240 92224 99.3505 82.1875 0.0123171 0.0100608 2024-09-17T08:21:16.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:16.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:16.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:16.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:16.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:16.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:16.504 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:16.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:92522 9.0 9:0661fd42:::benchmark_data_smithi079_143589_object92521:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.447907+0000", 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0050120149999999999, 2024-09-17T08:21:16.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050460720000000004, 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 92522 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447907+0000", 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447907+0000", 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447905+0000", 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447927+0000", 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2008e-05 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447939+0000", 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1491999999999999e-05 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447946+0000", 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447998+0000", 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1836999999999999e-05 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.448132+0000", 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013453200000000001 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:21:16.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.448279+0000", 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146852 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:92524 9.e e247/221 9:7f6542c7:::benchmark_data_smithi079_143589_object92523:head v 247'5742, mlcod=247'5739)", 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.449163+0000", 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0037560990000000002, 2024-09-17T08:21:16.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038732940000000002, 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449163+0000", 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449163+0000", 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449162+0000", 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449213+0000", 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1351999999999999e-05 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449224+0000", 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1542000000000001e-05 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449231+0000", 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8800000000000002e-06 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449807+0000", 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00057519600000000004 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449823+0000", 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6799e-05 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:92525 9.3 e247/221 9:c978a34b:::benchmark_data_smithi079_143589_object92524:head v 247'5954, mlcod=247'5952)", 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.449248+0000", 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.003670651, 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003853174, 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449248+0000", 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449248+0000", 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449248+0000", 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449341+0000", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3703999999999996e-05 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449351+0000", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8859999999999999e-06 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449363+0000", 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2126e-05 2024-09-17T08:21:16.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449420+0000", 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6990000000000002e-05 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.449442+0000", 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1362e-05 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15327.0:92522 9.0 e247/230)", 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.451221+0000", 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0016976579999999999, 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019373509999999999, 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451221+0000", 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451221+0000", 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451220+0000", 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:21:16.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451226+0000", 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7919999999999998e-06 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451243+0000", 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6787999999999999e-05 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451250+0000", 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8310000000000001e-06 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:92526 9.a 9:50288257:::benchmark_data_smithi079_143589_object92525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.452615+0000", 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00030345799999999999, 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00059885199999999998, 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:21:16.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 92526 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452615+0000", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452615+0000", 2024-09-17T08:21:16.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452614+0000", 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452635+0000", 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0945999999999999e-05 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452650+0000", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5753e-05 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452658+0000", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8189999999999992e-06 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452701+0000", 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2979000000000003e-05 2024-09-17T08:21:16.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452850+0000", 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000148438 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.452989+0000", 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139331 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:92514 9.a 9:59419c50:::benchmark_data_smithi079_143589_object92513:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.442964+0000", 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0099547609999999995, 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0103289, 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 92514 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.442964+0000", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.442964+0000", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.442962+0000", 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:21:16.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.442990+0000", 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7982000000000001e-05 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.443003+0000", 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2828e-05 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.443012+0000", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4079999999999998e-06 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.443133+0000", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012165199999999999 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.443307+0000", 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017318699999999999 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.443487+0000", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180836 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.447552+0000", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040647119999999998 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.451638+0000", 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040857269999999999 2024-09-17T08:21:16.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:92517 9.2 e247/230 9:4ee96a31:::benchmark_data_smithi079_143589_object92516:head v 247'5962, mlcod=247'5960)", 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:16.446150+0000", 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0067686250000000003, 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072243239999999998, 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446150+0000", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446150+0000", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446149+0000", 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:21:16.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446173+0000", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3677999999999999e-05 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446189+0000", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6430999999999999e-05 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446197+0000", 2024-09-17T08:21:16.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5510000000000002e-06 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446327+0000", 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013045900000000001 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:16.446347+0000", 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9216e-05 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:16.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:16.547 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:21:16.547 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 7 2024-09-17T08:21:16.547 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:21:16.606 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:21:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:21:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[40649]: pgmap v497: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-17T08:21:16.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[29565]: pgmap v497: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:21:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:16 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:21:16.641 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:16.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:16.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:21:16.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:16.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.798 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:16.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:16.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:16.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:16.953 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:16.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:16.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:16 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:21:16.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:16 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:21:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:16 smithi173 ceph-mon[33187]: pgmap v497: 148 pgs: 148 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-17T08:21:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:16 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:21:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:16 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:21:17.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:17.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:17.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:17.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:17.214 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 59 16 93457 93441 98.9558 76.0625 0.00848744 0.0101003 2024-09-17T08:21:17.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:17.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:17.337 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:17.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:17.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:17.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:17.509 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:17.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:17.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:21:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:21:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:21:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:21:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:17.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:17.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:17.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:17.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:17.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:17.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:17.914 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:17.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:17.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:17 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:21:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:17 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:21:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:18.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:21:18.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:18.058 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:18.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:18.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:18.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.196 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:18.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:18.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:21:18.213203+0000 min lat: 0.00129253 max lat: 2.12531 avg lat: 0.010141 2024-09-17T08:21:18.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:21:18.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 60 16 94663 94647 98.5627 75.375 0.0149449 0.010141 2024-09-17T08:21:18.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:18.331 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:18.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.332 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:18.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:18.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:18.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:18.508 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T08:21:18.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:18.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:18.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:18.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:18.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:18.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[40649]: pgmap v498: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.06k op/s 2024-09-17T08:21:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:21:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:21:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1711884959' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[29565]: pgmap v498: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.06k op/s 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1711884959' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:18 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:18 smithi173 ceph-mon[33187]: pgmap v498: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 67 MiB/s wr, 1.06k op/s 2024-09-17T08:21:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:18 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:21:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:18 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:21:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1711884959' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:18 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:21:18.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:18.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:18.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:18.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:19.013 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:19.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:19.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:19.155 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:19.212 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 61 16 95901 95885 98.2153 77.375 0.0106041 0.0101772 2024-09-17T08:21:19.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:19.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:19.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:19.442 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.443 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:19.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:19.580 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:19.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:19.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.721 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[29565]: pgmap v500: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:19.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:19.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:21:19.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:21:19.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:21:19.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T08:21:19.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:19 smithi079 ceph-mon[40649]: pgmap v500: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:21:19.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:19.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:19.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T08:21:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:19 smithi173 ceph-mon[33187]: pgmap v500: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:21:19.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:19.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:19.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:20.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:20.129 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:20.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:20.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 62 16 96995 96979 97.734 68.375 0.0101794 0.0102269 2024-09-17T08:21:20.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:20.260 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:20.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:20.362 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:20.392 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:20.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:20.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:20.534 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:20.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:20.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:20.666 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:20.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.667 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:20.751 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:21:20.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:21:20.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:21:20.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:21:20.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:21:20.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:21:20.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:20 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:21:20.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:20.811 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:20.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:20.913 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:20.945 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:20.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:20.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:21:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:21:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:21:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:21:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:21:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:20 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:21:21.047 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:21.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:21.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:21.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:21.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 63 15 98088 98073 97.268 68.375 0.0104853 0.010275 2024-09-17T08:21:21.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:21.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:21.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-17T08:21:21.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:21.346 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:21.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.347 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:21.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:21.490 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:21:21.501 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:21.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:21.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:21.641 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:21.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:21.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/466506347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:21:21.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[40649]: pgmap v502: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 841 op/s 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/466506347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:21:21.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:21.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T08:21:21.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:21 smithi079 ceph-mon[29565]: pgmap v502: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 841 op/s 2024-09-17T08:21:21.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:21.926 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:21.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:21.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/466506347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T08:21:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:21 smithi173 ceph-mon[33187]: pgmap v502: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 841 op/s 2024-09-17T08:21:22.028 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:22.057 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:22.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.058 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:22.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:22.201 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:22.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:22.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 64 16 99336 99320 96.9657 77.9375 0.00685809 0.0103083 2024-09-17T08:21:22.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:22.347 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:22.352 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:22.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:22.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:22.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:22.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:22.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:21:22.524 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:21:22.525 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:22 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:22.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.647 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:22.648 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:22.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:22.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:22.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:21:22.868 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:21:22.869 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:21:22.869 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:21:22.887 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:22.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:22.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:22.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:23.010 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:23.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:23.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:23.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:23.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:23.182 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:23.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:23.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 65 15 100417 100402 96.5143 67.625 0.0262498 0.0103564 2024-09-17T08:21:23.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:23.284 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:23.317 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:23.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:23.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:23.454 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:23.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:23.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:23.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:23.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:23.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:23.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:23.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:23.833 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:23.867 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:23.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:23.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:23 smithi079 ceph-mon[29565]: pgmap v503: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:21:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:23 smithi079 ceph-mon[40649]: pgmap v503: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:21:23.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:23 smithi173 ceph-mon[33187]: pgmap v503: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-17T08:21:24.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:24.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:24.138 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:24.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 66 15 101759 101744 96.3227 83.875 0.0100259 0.0103774 2024-09-17T08:21:24.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:24.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:24.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:21:24.374 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:24.404 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:24.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:24.533 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:21:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:21:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:24 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:21:24.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:24.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:24.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:24.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:24.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:24.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:24.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:24.943 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:24.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:24 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:21:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:24 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:21:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:24 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:21:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:24 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:21:25.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:25.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:25.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:25.179 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:25.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 67 16 103060 103044 96.0977 81.25 0.0130123 0.0104014 2024-09-17T08:21:25.222 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:25.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:25.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:25.365 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:25.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:25.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:25.498 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:25.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:25.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:25.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:25.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:25.735 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:25.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:25.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:21:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:21:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[40649]: pgmap v504: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:21:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:21:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:25 smithi079 ceph-mon[29565]: pgmap v504: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:25.874 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:25.908 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:25.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:25.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:25 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:21:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:25 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:21:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:25 smithi173 ceph-mon[33187]: pgmap v504: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:26.011 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:26.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:26.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:26.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:26.181 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:26.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:26.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 68 16 104221 104205 95.7515 72.5625 0.00383321 0.0104353 2024-09-17T08:21:26.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:26.316 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:26.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:26.418 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:26.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:26.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:26.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:26.571 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:26.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:26.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:26.809 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:26.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.837 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:26.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:26.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:26 smithi079 ceph-mon[40649]: 2.1a deep-scrub starts 2024-09-17T08:21:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:26 smithi079 ceph-mon[40649]: 2.1a deep-scrub ok 2024-09-17T08:21:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:26 smithi079 ceph-mon[29565]: 2.1a deep-scrub starts 2024-09-17T08:21:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:26 smithi079 ceph-mon[29565]: 2.1a deep-scrub ok 2024-09-17T08:21:26.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:26.973 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:26 smithi173 ceph-mon[33187]: 2.1a deep-scrub starts 2024-09-17T08:21:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:26 smithi173 ceph-mon[33187]: 2.1a deep-scrub ok 2024-09-17T08:21:26.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:26.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:27.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:27.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:27.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:27.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:27.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 69 16 105334 105318 95.372 69.5625 0.00938618 0.0104808 2024-09-17T08:21:27.247 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:27.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:27.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:27.378 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:27.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.379 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:27.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:27.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.511 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:27.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:27.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:27.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:21:27.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:21:27.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:21:27.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:21:27.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:21:27.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[29565]: pgmap v505: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 572 B/s rd, 53 MiB/s wr, 850 op/s 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:21:27.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:21:27.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:27 smithi079 ceph-mon[40649]: pgmap v505: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 572 B/s rd, 53 MiB/s wr, 850 op/s 2024-09-17T08:21:27.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:27.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:27.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:27.790 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:27.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:27.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:27.923 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:27.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:27.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:21:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:27 smithi173 ceph-mon[33187]: pgmap v505: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 572 B/s rd, 53 MiB/s wr, 850 op/s 2024-09-17T08:21:28.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:28.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:28.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:28.214 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 70 16 106632 106616 95.1684 81.125 0.0138828 0.0105032 2024-09-17T08:21:28.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:28.260 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:28.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:28.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.358 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:28.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:28.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:28.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:28.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.497 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:28.598 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:28.632 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:28.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:28.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:28.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:28.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.764 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:21:28.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:21:28.852 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:21:28.852 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:28 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:21:28.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:28.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:28.897 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:28.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:28 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:21:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:28 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:21:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:28 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:21:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:28 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:21:28.999 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.031 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:29.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:29.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:29.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:29.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:29.213 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 71 16 107879 107863 94.9256 77.9375 0.0110632 0.01053 2024-09-17T08:21:29.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:29.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:29.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:29.406 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:21:29.407 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:21:29.419 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:29.442 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:21:29.442 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:21:29.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:29.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:29.475 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:21:29.475 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:21:29.510 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:21:29.510 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:21:29.540 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:21:29.541 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:21:29.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:29.571 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:21:29.572 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:21:29.597 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:29.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:29.603 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:21:29.603 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:21:29.638 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:21:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:21:29.662 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[40649]: 5.11 deep-scrub starts 2024-09-17T08:21:29.662 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[40649]: 5.11 deep-scrub ok 2024-09-17T08:21:29.662 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3121247131' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:29.662 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[40649]: pgmap v506: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:21:29.663 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[29565]: 5.11 deep-scrub starts 2024-09-17T08:21:29.663 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[29565]: 5.11 deep-scrub ok 2024-09-17T08:21:29.663 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3121247131' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:29.663 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:29 smithi079 ceph-mon[29565]: pgmap v506: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:21:29.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-mon[33187]: 5.11 deep-scrub starts 2024-09-17T08:21:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-mon[33187]: 5.11 deep-scrub ok 2024-09-17T08:21:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3121247131' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-mon[33187]: pgmap v506: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:21:29.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore.transactions t=2024-09-17T08:21:29.427045963Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:21:29.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:21:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=cleanup t=2024-09-17T08:21:29.427480557Z level=info msg="Completed cleanup jobs" duration=12.52712ms 2024-09-17T08:21:29.735 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:29.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:29.837 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:29.865 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:29.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:29.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:29.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:30.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:30.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:30.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:30.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:30.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:30.214 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 72 16 109041 109025 94.6159 72.625 0.0120657 0.0105643 2024-09-17T08:21:30.260 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:30.285 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:30.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:30.344 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:21:30.344 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:21:30.344 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:21:30.344 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:21:30.344 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:21:30.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:30.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:30.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:30.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:30.558 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:30.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:30.598 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T08:21:30.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:30.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:30.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:30.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:30.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:30.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:21:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:21:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:21:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:21:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:21:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:21:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:21:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:21:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:21:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:21:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:21:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:21:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:21:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:21:30.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:21:30.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:21:30.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:21:30.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:30 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:21:30.929 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:30.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:30.962 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:30.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:30.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:21:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:30 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:21:31.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:31.101 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:31.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:31.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:21:31.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:31.214 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 73 16 110267 110251 94.3693 76.625 0.0165844 0.0105923 2024-09-17T08:21:31.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:31.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:31.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:31.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:31.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:31.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:31.535 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:31.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:31.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:31.671 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:31.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:31.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:31.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.810 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:31.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: 6.14 deep-scrub starts 2024-09-17T08:21:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: 6.14 deep-scrub ok 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T08:21:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[40649]: pgmap v509: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 813 op/s 2024-09-17T08:21:31.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: 6.14 deep-scrub starts 2024-09-17T08:21:31.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: 6.14 deep-scrub ok 2024-09-17T08:21:31.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:21:31.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T08:21:31.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:21:31.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:21:31.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T08:21:31.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:31 smithi079 ceph-mon[29565]: pgmap v509: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 813 op/s 2024-09-17T08:21:31.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:31.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:31.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:31.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: 6.14 deep-scrub starts 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: 6.14 deep-scrub ok 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/380036934' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T08:21:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:31 smithi173 ceph-mon[33187]: pgmap v509: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 813 op/s 2024-09-17T08:21:32.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:32.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:32.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:32.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:32.214 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 74 16 111567 111551 94.192 81.25 0.0125522 0.0106121 2024-09-17T08:21:32.228 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:32.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:32.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:32.366 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:32.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:32.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:32.511 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:32.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:32.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:32.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.648 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:32.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:32.783 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:32.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: 2.19 deep-scrub starts 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: 2.19 deep-scrub ok 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: 2.19 deep-scrub starts 2024-09-17T08:21:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: 2.19 deep-scrub ok 2024-09-17T08:21:32.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:32.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: 2.19 deep-scrub starts 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: 2.19 deep-scrub ok 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:32.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:32.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:32.921 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:32.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:32.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:33.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:33.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-17T08:21:33.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:33.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:33.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 75 16 112848 112832 94.0021 80.0625 0.0200195 0.0106335 2024-09-17T08:21:33.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:33.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:33.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:33.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:33.280 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:33.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:33.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:33.390 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:33.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:33.531 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:33.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:33.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:33.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:33.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:33.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:33.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.801 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:33.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[40649]: pgmap v510: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:33 smithi079 ceph-mon[29565]: pgmap v510: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:21:33.902 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:33.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:33.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:33.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:33 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:21:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:33 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:21:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:33 smithi173 ceph-mon[33187]: pgmap v510: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:21:34.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:34.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:34.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:34.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:34.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:34.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:34.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 76 16 114015 113999 93.7249 72.9375 0.00582248 0.0106644 2024-09-17T08:21:34.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:34.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:34.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:34.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:34.457 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:34.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:34.559 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:34.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.586 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:34.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:34.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:34.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.723 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:34.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:34.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:34.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:34.863 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:34.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:21:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:21:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:21:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:21:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:34 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:21:34.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:21:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:34 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:21:35.003 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:35.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:35.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:35.144 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:35.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 77 16 115149 115133 93.4282 70.875 0.00669612 0.0106988 2024-09-17T08:21:35.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:35.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:35.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:35.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:35.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:35.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:35.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:35.544 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:35.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:35.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:35.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:35.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:35.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:35.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:35.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:21:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:21:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[40649]: pgmap v512: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:21:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:21:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:35 smithi079 ceph-mon[29565]: pgmap v512: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:35.902 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:35.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:35.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:35.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:35 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:21:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:35 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:21:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:35 smithi173 ceph-mon[33187]: pgmap v512: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:36.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:36.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:36.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:36.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:36.209 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:36.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 78 16 116160 116144 93.0405 63.1875 0.00826402 0.0107435 2024-09-17T08:21:36.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:36.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:36.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:21:36.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:36.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:36.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:36.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:36.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:36.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:36.628 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:21:36.628 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:21:36.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:36.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.762 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:36.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:36.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:36.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:36.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:36.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3621465345' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:21:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:36 smithi173 ceph-mon[33187]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T08:21:36.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:37.030 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:37.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3621465345' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[29565]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3621465345' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:21:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:36 smithi079 ceph-mon[40649]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T08:21:37.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:37.172 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:37.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:37.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 79 15 117428 117413 92.8666 79.3125 0.0123347 0.0107632 2024-09-17T08:21:37.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:37.292 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:37.293 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:37.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:37.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:37.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:37.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:37.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:37.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:37.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.617 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:37.618 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:37.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:37.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:37.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: 5.e deep-scrub starts 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: 5.e deep-scrub ok 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[40649]: pgmap v515: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-17T08:21:37.812 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: 5.e deep-scrub starts 2024-09-17T08:21:37.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: 5.e deep-scrub ok 2024-09-17T08:21:37.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:37.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:37.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T08:21:37.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:37 smithi079 ceph-mon[29565]: pgmap v515: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-17T08:21:37.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:37.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:37.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:37.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: 5.e deep-scrub starts 2024-09-17T08:21:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: 5.e deep-scrub ok 2024-09-17T08:21:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T08:21:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:37 smithi173 ceph-mon[33187]: pgmap v515: 148 pgs: 148 active+clean; 7.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-17T08:21:37.986 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:38.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:38.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:38.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:38.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.172 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:38.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:38.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:21:38.216222+0000 min lat: 0.00129253 max lat: 2.12531 avg lat: 0.0107853 2024-09-17T08:21:38.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:21:38.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 80 16 118672 118656 92.6768 77.6875 0.00863074 0.0107853 2024-09-17T08:21:38.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:38.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:38.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:38.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:38.447 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:38.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:38.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:38.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:38.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:38.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:38.714 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:38.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:38.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:38.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:38.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:38.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:38.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:39.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:39.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:39.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:39.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:39.215 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 81 16 119977 119961 92.5395 81.5625 0.0134525 0.0108019 2024-09-17T08:21:39.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:39.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:21:39.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:39.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:39.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:39.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:39.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:39.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:39.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:39.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:39.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:39.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:39.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:39.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:39.823 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:39.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:39.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:21:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:39 smithi079 ceph-mon[40649]: pgmap v516: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 982 op/s 2024-09-17T08:21:39.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:39 smithi079 ceph-mon[29565]: pgmap v516: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 982 op/s 2024-09-17T08:21:39.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:39.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:39.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:39.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:39 smithi173 ceph-mon[33187]: pgmap v516: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 982 op/s 2024-09-17T08:21:40.056 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:40.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:40.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:40.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:40.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 82 16 121212 121196 92.3523 77.1875 0.00810128 0.0108237 2024-09-17T08:21:40.217 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:40.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:40.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:40.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:40.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:40.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:40.491 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:40.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:40.593 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:40.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:40.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:40.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:40.762 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:40.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:40.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:40.913 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:40.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:40.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:41.015 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:41.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:41.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.053 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:41.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:41.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.187 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:41.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:41.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 83 16 122418 122402 92.1477 75.375 0.00376888 0.010847 2024-09-17T08:21:41.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:41.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:41.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:41.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:41.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:41.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:41.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:41.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.593 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:41.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:41.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:41.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.744 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:41.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:41.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:41.883 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:21:41.883 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:21:41.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:41.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:41.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:41.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:42.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:42.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:42.110 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:42.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:42.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:42.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:42.217 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 84 16 123651 123635 91.968 77.0625 0.00852784 0.010869 2024-09-17T08:21:42.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:42.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:42.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:42.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:42.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:42.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:42.522 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:42 smithi079 ceph-mon[29565]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T08:21:42.522 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:42 smithi079 ceph-mon[29565]: pgmap v518: 148 pgs: 148 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:42.522 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:42 smithi079 ceph-mon[40649]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T08:21:42.523 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:42 smithi079 ceph-mon[40649]: pgmap v518: 148 pgs: 148 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:42.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:42.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:21:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:42.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:42.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:42.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:42.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:42 smithi173 ceph-mon[33187]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T08:21:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:42 smithi173 ceph-mon[33187]: pgmap v518: 148 pgs: 148 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:21:42.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:42.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:42.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:42.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:42.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:42.998 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:42.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:43.099 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:43.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:43.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:43.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 85 16 124860 124844 91.775 75.5625 0.0122034 0.0108913 2024-09-17T08:21:43.217 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:43.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:43.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:43.273 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:43.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:43.342 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:43.375 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T08:21:43.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:43.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:43.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:43 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3812166278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:43 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3812166278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:43 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.655 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:43.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:43.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:43 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3812166278' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:43 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:21:43.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:43.822 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:43.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:43.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:43.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:43.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:43.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:44.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:44.092 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:44.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:44.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:44.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 86 16 126141 126125 91.6384 80.0625 0.0110533 0.0109078 2024-09-17T08:21:44.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:44.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:44.338 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:44.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.371 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:44.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:44.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:44.504 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:44.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:44.565 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:44.565 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:21:44.565 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[40649]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T08:21:44.565 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[40649]: pgmap v520: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 845 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:21:44.566 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:44.566 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:21:44.566 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[29565]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T08:21:44.566 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:44 smithi079 ceph-mon[29565]: pgmap v520: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 845 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:21:44.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:44.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:44.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:44 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:21:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:44 smithi173 ceph-mon[33187]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T08:21:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:44 smithi173 ceph-mon[33187]: pgmap v520: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 845 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:21:44.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:44.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.783 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:44.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:44.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:44.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:44.924 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:44.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:45.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:45.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-17T08:21:45.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:45.165 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:45.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:45.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 87 15 127403 127388 91.4923 78.9375 0.0149911 0.0109251 2024-09-17T08:21:45.295 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:45.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:45.431 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:45.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.468 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:45.569 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:45.600 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:45.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:45 smithi079 ceph-mon[29565]: pgmap v521: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:45.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:45 smithi079 ceph-mon[40649]: pgmap v521: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:45.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:45.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.739 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:45.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:45.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:45.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:45.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:45.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:45 smithi173 ceph-mon[33187]: pgmap v521: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-17T08:21:45.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:46.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:46.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:46.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:46.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:46.216 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 88 16 128733 128717 91.3964 83.0625 0.0130612 0.0109366 2024-09-17T08:21:46.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:46.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:46.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:46.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:46.515 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:46.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:46.645 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:46.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:46.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:46.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:46.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:46.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:46.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:46.944 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:47.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:47.217 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 89 16 130015 129999 91.2688 80.125 0.0114491 0.0109525 2024-09-17T08:21:47.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:47.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:47.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:47.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:47.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:47.482 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:47.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:47.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:47.549 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:47.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[40649]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T08:21:47.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:47.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:47.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:47.568 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:47.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[29565]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T08:21:47.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:47.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:47.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:47.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:47.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:47.648 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:47.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:47 smithi173 ceph-mon[33187]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T08:21:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:47 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:47 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:21:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:21:47.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:47.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:47.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:47.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:47.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:47.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:48.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:48.093 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:48.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total time run: 90.0187 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total writes made: 131196 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Write size: 65536 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object size: 65536 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Bandwidth (MB/sec): 91.0894 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Bandwidth: 32.0715 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max bandwidth (MB/sec): 244.938 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min bandwidth (MB/sec): 0 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average IOPS: 1457 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev IOPS: 513.145 2024-09-17T08:21:48.218 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max IOPS: 3919 2024-09-17T08:21:48.219 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min IOPS: 0 2024-09-17T08:21:48.219 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average Latency(s): 0.0109735 2024-09-17T08:21:48.219 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Latency(s): 0.0284733 2024-09-17T08:21:48.219 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max latency(s): 2.12531 2024-09-17T08:21:48.219 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min latency(s): 0.00129253 2024-09-17T08:21:48.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:48.223 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T08:21:48.249 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:21:48.249 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:21:48.250 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:21:48.250 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:21:48.287 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:21:48.287 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:21:48.288 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:48.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:48.329 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:21:48.330 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:21:48.366 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:48.368 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:21:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:21:48.409 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:21:48.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:21:48.452 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:21:48.453 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:21:48.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:48.489 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:21:48.490 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[40649]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[40649]: pgmap v524: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1401391949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[29565]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[29565]: pgmap v524: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:21:48.490 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1401391949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:48.512 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:48.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:48.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:48.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:48 smithi173 ceph-mon[33187]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T08:21:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:48 smithi173 ceph-mon[33187]: pgmap v524: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:21:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1401391949' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:21:48.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:48.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.782 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:48.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:48.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:48.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:48.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:49.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:49.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.072 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:49.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:49.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:49.204 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:49.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:49.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:49.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:49.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:49.429 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:21:49.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:49.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:49.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:49.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:49.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:49.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:49.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:49.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1526845600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:49 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:49.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:49.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:49.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1526845600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:49 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1526845600' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:49 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:21:49.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:49.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:49.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:49.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:50.018 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:50.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:50.145 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:50.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:50.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:50.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:50.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:50.417 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:50.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:50.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:50.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:50.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:50.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:50 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:50 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:50 smithi173 ceph-mon[33187]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T08:21:50.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:50 smithi173 ceph-mon[33187]: pgmap v526: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.31k op/s 2024-09-17T08:21:50.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:50.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:50.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[40649]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T08:21:50.735 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[40649]: pgmap v526: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.31k op/s 2024-09-17T08:21:50.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:21:50.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:21:50.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[29565]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T08:21:50.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:50 smithi079 ceph-mon[29565]: pgmap v526: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.31k op/s 2024-09-17T08:21:50.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:50.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:50.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:50.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:50.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:50.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:51.061 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:51.097 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:51.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:51.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:51.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.237 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:51.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:51.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:51.372 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:51.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:51.474 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:51.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.510 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:51.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:51.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:51.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:51.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:51.779 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:51.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:51.916 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:51.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:51.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:52.018 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:52.053 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:52.155 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:52.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:52.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:52.317 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:52.419 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:52.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.457 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:52.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:52.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:52.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:52.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:52 smithi173 ceph-mon[33187]: pgmap v527: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.12k op/s 2024-09-17T08:21:52.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.729 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:52.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:52.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:52.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:52.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:52 smithi079 ceph-mon[40649]: pgmap v527: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.12k op/s 2024-09-17T08:21:52.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:52 smithi079 ceph-mon[29565]: pgmap v527: 148 pgs: 148 active+clean; 8.0 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.12k op/s 2024-09-17T08:21:52.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:52.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:52.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:53.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:53.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.130 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:53.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:53.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:21:53.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:21:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:21:53.226 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:21:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:21:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:21:53.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:53.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:53.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:53.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:53.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:53.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:53.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:53.565 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:53.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:53.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:53.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.698 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:53.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:53.800 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:53.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:53.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:53 smithi079 ceph-mon[40649]: pgmap v528: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 867 B/s rd, 55 MiB/s wr, 1.50k op/s 2024-09-17T08:21:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:53 smithi079 ceph-mon[29565]: pgmap v528: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 867 B/s rd, 55 MiB/s wr, 1.50k op/s 2024-09-17T08:21:53.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:53.962 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:53.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:53.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:21:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:53 smithi173 ceph-mon[33187]: pgmap v528: 148 pgs: 148 active+clean; 7.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 867 B/s rd, 55 MiB/s wr, 1.50k op/s 2024-09-17T08:21:54.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:54.094 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:54.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:54.230 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:54.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:54.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:54.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:54.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:54.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:54.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.514 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:54.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:54.515 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:21:54.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:54.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:54.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:54.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:54.779 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:54 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:54.779 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:54 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:54.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:54 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:54.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:54 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:54.805 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:54.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:54.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:54.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:54.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:54 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:54 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:55.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:55.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:55.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:55.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:55.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.218 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:55.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:140268 9.2 9:41289c96:::benchmark_data_smithi079_143589_object9069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:55.293479+0000", 2024-09-17T08:21:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00060169800000000001, 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062860800000000001, 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 140268 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293479+0000", 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293479+0000", 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293478+0000", 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:21:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293484+0000", 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.959e-06 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293496+0000", 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2201999999999999e-05 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293502+0000", 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1800000000000001e-06 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293544+0000", 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2008999999999997e-05 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293645+0000", 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101149 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:21:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293744+0000", 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8573999999999995e-05 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:140266 9.7 e260/221)", 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:55.293777+0000", 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00030320299999999999, 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040076199999999998, 2024-09-17T08:21:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293777+0000", 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293777+0000", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293777+0000", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293794+0000", 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6772000000000001e-05 2024-09-17T08:21:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293804+0000", 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0624e-05 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293811+0000", 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6309999999999997e-06 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:140270 9.3 e260/221)", 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:55.293821+0000", 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.000259926, 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039762899999999998, 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293821+0000", 2024-09-17T08:21:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293821+0000", 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293820+0000", 2024-09-17T08:21:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293825+0000", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2010000000000001e-06 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293831+0000", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1e-06 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.293835+0000", 2024-09-17T08:21:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1119999999999998e-06 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:140259 9.3 e260/221 9:ca46197a:::benchmark_data_smithi079_143589_object9060:head v 260'8979, mlcod=260'8977)", 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:55.291834+0000", 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0022463660000000001, 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024294770000000002, 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291834+0000", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291834+0000", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291834+0000", 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:21:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291839+0000", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1379999999999998e-06 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291846+0000", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8109999999999997e-06 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291850+0000", 2024-09-17T08:21:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999996e-06 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291923+0000", 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3041999999999996e-05 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291943+0000", 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0145000000000001e-05 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:140260 9.7 e260/221 9:ec462918:::benchmark_data_smithi079_143589_object9061:head v 260'8867, mlcod=260'8862)", 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:21:55.291860+0000", 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0022203119999999999, 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024570429999999999, 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291860+0000", 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291860+0000", 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291860+0000", 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967296 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:21:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291864+0000", 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1409999999999998e-06 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291870+0000", 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9279999999999998e-06 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291874+0000", 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2010000000000001e-06 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291940+0000", 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5817999999999999e-05 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:21:55.291959+0000", 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8388000000000001e-05 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:21:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:21:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 5 2024-09-17T08:21:55.316 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:21:55.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:55.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:55.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:55.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:55.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:55.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:55.592 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:55.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:55.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:55.724 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:55.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:55.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:55.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:55.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:55 smithi079 ceph-mon[40649]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T08:21:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:55 smithi079 ceph-mon[40649]: pgmap v530: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-09-17T08:21:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:55 smithi079 ceph-mon[29565]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T08:21:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:55 smithi079 ceph-mon[29565]: pgmap v530: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-09-17T08:21:55.898 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:55.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:55.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:55.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:21:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:55 smithi173 ceph-mon[33187]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T08:21:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:55 smithi173 ceph-mon[33187]: pgmap v530: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-09-17T08:21:56.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:56.029 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:56.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:56.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.160 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:56.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:56.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:56.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:56.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:56.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:56.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:56.597 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T08:21:56.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:56.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:56.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:56.802 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:56.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:56 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2657525179' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:56 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2657525179' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:56.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:56.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:56.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:56 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2657525179' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:21:57.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:57.104 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:57.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:57.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:57.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:57.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:57.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:57.392 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:57.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:57.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:57.532 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:57.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:57.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:57.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:57.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:57.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:21:57.799 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:57.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:21:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:21:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[40649]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T08:21:57.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[40649]: pgmap v532: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 767 B/s rd, 3.0 MiB/s wr, 1.05k op/s 2024-09-17T08:21:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:21:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[29565]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T08:21:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:57 smithi079 ceph-mon[29565]: pgmap v532: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 767 B/s rd, 3.0 MiB/s wr, 1.05k op/s 2024-09-17T08:21:57.900 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:57.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:57.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:57.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:21:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:57 smithi173 ceph-mon[33187]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T08:21:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:57 smithi173 ceph-mon[33187]: pgmap v532: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 767 B/s rd, 3.0 MiB/s wr, 1.05k op/s 2024-09-17T08:21:58.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:58.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.068 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:58.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:58.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.203 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:58.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:21:58.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:21:58.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:21:58.445 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:58.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:58.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:58.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:58.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:58.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:58.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:58.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:58.886 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:58.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:58.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:58.989 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:21:59.020 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:59.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:21:59.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:21:59.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:59.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:21:59.259 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:21:59.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.296 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:59.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:21:59.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:59.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.431 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:59.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:21:59.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:21:59.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:21:59.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:59.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:59.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:59.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:21:59.844 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:21:59.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:21:59.933 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:21:59.933 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:21:59.933 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[29565]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T08:21:59.933 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[29565]: pgmap v534: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:21:59.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:21:59.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:21:59.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[40649]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T08:21:59.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:21:59 smithi079 ceph-mon[40649]: pgmap v534: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:21:59.946 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:21:59.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:21:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:21:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:59 smithi173 ceph-mon[33187]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T08:21:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:21:59 smithi173 ceph-mon[33187]: pgmap v534: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:21:59.978 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:21:59.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:21:59.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:00.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:00.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:00.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:00.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:00.254 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:00.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:00.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:00.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.393 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:00.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:00.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:00.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.529 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:00.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:00.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-17T08:22:00.625 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:00.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:00.659 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:00.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.660 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:00.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:00.795 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:00.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:00.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:00.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:00.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:00.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:01.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:01.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.078 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:01.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:01.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:01.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:01.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:01.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:01.354 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:01.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:01.456 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:01.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:01.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:01.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:01.616 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:01.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:01.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:01.749 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:01.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:01.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:01.875 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:01.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:01.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:01.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:02.005 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:02.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:02.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:02.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:02.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:02.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:02.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:02.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:02.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:02.421 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:02.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[29565]: pgmap v535: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[40649]: pgmap v535: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:22:02.504 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:22:02.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:02.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:02.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:02.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:02.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:02.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:02.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:02.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:02 smithi173 ceph-mon[33187]: pgmap v535: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:02 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:22:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:02 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:22:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:02.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:02.826 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:02.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:02.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:02.965 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:02.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:02.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:03.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:03.101 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:03.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:03.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:03.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:03.240 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:03.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:03.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:03.377 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:03.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:03.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:03.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.501 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:03.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:03.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:03.634 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:03.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:03.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:03 smithi173 ceph-mon[33187]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T08:22:03.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:03.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:03.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:03.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:03 smithi079 ceph-mon[29565]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T08:22:03.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:03 smithi079 ceph-mon[40649]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T08:22:03.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:03.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:03.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:04.021 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:04.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.057 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:04.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:04.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:04.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:04.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:04.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:04.327 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:04.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:04.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:04.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:04.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:04.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:04.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:04.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:04.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:04.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:04 smithi173 ceph-mon[33187]: pgmap v537: 148 pgs: 148 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-17T08:22:04.747 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:04.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:04.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:04.782 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:04 smithi079 ceph-mon[40649]: pgmap v537: 148 pgs: 148 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-17T08:22:04.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:04.782 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:04 smithi079 ceph-mon[29565]: pgmap v537: 148 pgs: 148 active+clean; 6.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-17T08:22:04.849 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:04.879 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:04.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:04.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:04.936 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:04.937 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:04.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:05.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.029 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:05.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:05.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:05.160 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:05.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:05.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:05.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:05.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:05.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:05.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:05.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:05.429 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:22:05.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:05.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:05.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:22:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:22:05.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:05.697 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:05.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.698 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:05.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:22:05.799 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:05.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:05.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:05.875 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:22:05.875 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:22:05.875 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:22:05.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:22:05.888 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:22:05.888 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:22:05.921 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:22:05.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:22:05.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:05.954 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:22:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:22:05.976 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:05.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:05.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:06.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:06.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:06.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:06.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:06.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:06.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:06.358 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:06.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:06.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:06.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:06.526 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:06.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:06.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:06.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.660 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:06.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: pgmap v539: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 974 op/s 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2822602683' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:22:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:06 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:22:06.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:06.798 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:06.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: pgmap v539: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 974 op/s 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2822602683' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1212294204' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T08:22:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: pgmap v539: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 974 op/s 2024-09-17T08:22:06.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2822602683' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:06.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:22:06.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:06 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:22:06.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:06.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:06.938 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:06.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:07.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:07.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:07.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:07.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:07.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:07.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:07.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:07.427 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:07.456 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:07.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:07.585 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:07.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:07.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:07.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:07 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:22:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:07 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:22:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:07 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:22:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:07 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:22:07.824 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:07.864 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:22:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:22:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:22:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:22:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:22:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:22:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:22:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:07 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:22:07.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:07.998 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:07.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:07.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:08.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:08.133 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:08.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:08.234 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:08.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:08.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:08.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:08.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.413 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:08.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:08.514 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:08.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:08.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: pgmap v540: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 881 op/s 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:22:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: pgmap v540: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 881 op/s 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:22:08.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:08.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:08.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:08.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: pgmap v540: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 881 op/s 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:22:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:08.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:08.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:08.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:08.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:08.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:08.957 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:08.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:09.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:09.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.095 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:09.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:09.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:09.232 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:09.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:09.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:09.361 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:09.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:09.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:09.502 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:09.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:09.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:09 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:22:09.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:09 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:22:09.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:09 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:22:09.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:09 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:22:09.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:09.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:09 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:22:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:09 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:22:09.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:09.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:09.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:09.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:09.912 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:09.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:09.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:10.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:10.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:10.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:10.156 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:10.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:10.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:10.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:10.314 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:10.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:10.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:22:10.415 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:10.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.443 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:10.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:10.466 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:22:10.466 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:22:10.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:22:10.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:10.584 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:10.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:10.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: pgmap v541: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: pgmap v541: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-17T08:22:10.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:22:10.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:22:10.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:22:10.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:10 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:22:10.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: pgmap v541: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:22:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:10 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:22:10.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:10.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:10.842 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:10.878 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:10.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:10.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:10.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:11.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.014 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:11.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:11.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:11.153 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:11.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:11.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:11.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:11.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.290 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:11.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:11.421 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:11.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.422 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:11.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:11.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:22:11.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:11.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:11.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:11.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.688 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:11.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:11 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:22:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:11 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:22:11.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:11 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:22:11.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:11 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:22:11.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:11 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:22:11.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:11 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:22:11.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:11.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:11.822 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:11.921 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:11.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:11.924 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:12.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:12.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:12.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:12.151 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:12.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:12.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:12.296 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:12.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:12.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:12.432 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:12.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:12.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:12.538 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T08:22:12.573 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:12.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:12.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:12.713 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:12.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:12.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:12.849 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:12.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:12.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:22:12.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: pgmap v542: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:22:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: pgmap v542: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:22:12.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:22:12.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:22:12.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:12 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:22:12.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:22:12.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:22:12.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:22:12.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:22:12.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: pgmap v542: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-17T08:22:12.925 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:22:12.925 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:22:12.925 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:22:12.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:12.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:12.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:12.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:13.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:13.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:13.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:13.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:13.258 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:13.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:13.363 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:13.402 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:13.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:13.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:13.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[29565]: pgmap v544: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:22:13.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:13 smithi079 ceph-mon[40649]: pgmap v544: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:22:13.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:13.687 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:13.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:13.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:13.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.824 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:13.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:13.951 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:13.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:13.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2622867205' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:13 smithi173 ceph-mon[33187]: pgmap v544: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 636 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:22:14.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:14.086 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:14.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:14.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:14.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.227 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:14.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:14.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:14.360 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:14.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:14.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:14.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:14.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:14.621 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:14.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:14.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:14.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.744 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:14.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:14.845 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:22:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:22:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:14 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:22:14.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:14.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:14 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:22:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:14 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:22:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:14 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:22:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:14 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:22:14.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:15.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:15.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:15.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:15.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.155 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:15.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:15.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:15.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:15.432 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:15.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:15.571 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:15.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:15.710 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:15.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:15.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:22:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:22:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[29565]: pgmap v545: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:22:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:15 smithi079 ceph-mon[40649]: pgmap v545: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:15.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:22:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:15 smithi173 ceph-mon[33187]: pgmap v545: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:15.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:15.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:15.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:16.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:16.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.127 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:16.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:16.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:16.263 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:16.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:16.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:16.403 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:16.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:16.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:16.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.542 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:16.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:16.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:16.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:16.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.677 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:16.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:16.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:16.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:22:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:22:16.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:16 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:22:16.920 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:16.949 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:16.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:16.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:16 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:22:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:16 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:22:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:16 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:22:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:16 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:22:16.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-17T08:22:17.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:17.108 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:17.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:17.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:17.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.251 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:17.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:17.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:17.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:17.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:17.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:17.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:17.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:17.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:17.671 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:17.849 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:22:17.850 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:22:17.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.851 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:22:17.851 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:17.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:22:17.851 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:17.852 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:22:17.852 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:22:17.852 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:22:17.852 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[29565]: 9.7 scrub starts 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[29565]: pgmap v546: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:17.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[40649]: 9.7 scrub starts 2024-09-17T08:22:17.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:22:17.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:22:17.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:17.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:17 smithi079 ceph-mon[40649]: pgmap v546: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:17.854 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:22:17.854 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:22:17.878 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:22:17.879 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:22:17.914 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:22:17.915 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:22:17.946 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:22:17.946 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:22:17.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:17 smithi173 ceph-mon[33187]: 9.7 scrub starts 2024-09-17T08:22:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:17 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:22:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:17 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:22:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:17 smithi173 ceph-mon[33187]: pgmap v546: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-17T08:22:17.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:17.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:17.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:18.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:18.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:18.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:18.224 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:18.248 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:18.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:18.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:18.393 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:18.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:18.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:18.523 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:18.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:18.590 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:22:18.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2412363040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:22:18.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:22:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2412363040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:22:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:22:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:18.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:18 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:22:18.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:18.661 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:18.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:18.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:18.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:18.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:18.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:22:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:22:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2412363040' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:18.876 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:22:18.876 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:22:18.876 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:18.876 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:18 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:22:18.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:18.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:18.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:18.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:19.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:19.074 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:19.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:19.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:19.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:19.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:19.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:19.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:19.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:19.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:19.502 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:19.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:19.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:19.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:19.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:19.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:19.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:19.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: 9.7 scrub ok 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T08:22:19.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[40649]: pgmap v548: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: 9.7 scrub ok 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T08:22:19.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:19 smithi079 ceph-mon[29565]: pgmap v548: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-17T08:22:19.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:19.894 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:19.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:19.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: 9.7 scrub ok 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3773961866' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T08:22:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:19 smithi173 ceph-mon[33187]: pgmap v548: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-17T08:22:19.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:20.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:20.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:20.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:20.157 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:20.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:20.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:20.293 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:20.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:20.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:20.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:20.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:20.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:20.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.559 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:20.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:20.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:20.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:20.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:20.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:20.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:20.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:20.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:22:20.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:22:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:20 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:22:20.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:20.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:20.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:20.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:20 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:22:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:20 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:22:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:20 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:22:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:20 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:22:21.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:21.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:21.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:21.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:21.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:21.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.376 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:21.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:21.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.499 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:21.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:21.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:21.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:21.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:21.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:22:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[29565]: pgmap v549: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:22:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:21 smithi079 ceph-mon[40649]: pgmap v549: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:21.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:21.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:21.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:21.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:22:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:22:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:22:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:22:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:22:21.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:22:21.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:21 smithi173 ceph-mon[33187]: pgmap v549: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:22.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:22.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:22.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:22.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:22.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.185 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:22.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:22.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:22.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:22.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:22.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:22.457 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:22.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:22.559 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:22.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:22.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:22.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:22.716 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:22.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:22.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:22.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:22.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:22.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:22:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:22:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:22:22.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:22 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:22:22.936 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:22:22.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:22.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:22.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:22.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:23.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:23.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.120 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:23.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:23.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:23.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:23.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.257 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:23.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:23.378 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:23.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:23.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:23.511 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:23.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:23.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:23.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.647 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:23.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:23.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:23.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.781 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:23.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: 5.1a deep-scrub starts 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: 5.1a deep-scrub ok 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: 5.16 deep-scrub starts 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: 5.16 deep-scrub ok 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[40649]: pgmap v550: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: 5.1a deep-scrub starts 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: 5.1a deep-scrub ok 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: 5.16 deep-scrub starts 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: 5.16 deep-scrub ok 2024-09-17T08:22:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:23.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:23 smithi079 ceph-mon[29565]: pgmap v550: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:22:23.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:23.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:23.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:23.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: 5.1a deep-scrub starts 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: 5.1a deep-scrub ok 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: 5.16 deep-scrub starts 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: 5.16 deep-scrub ok 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:23 smithi173 ceph-mon[33187]: pgmap v550: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:22:24.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:24.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.039 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:24.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:24.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.161 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:24.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:24.295 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:24.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.296 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:24.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:24.429 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:24.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:24.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.556 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:24.657 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:24.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.693 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:24.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:24.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:24.826 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:24.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:24.975 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:24.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:24.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:25.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:25.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:25.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:25.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:25.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:25.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:25.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:25.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.391 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:25.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:25.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:25.528 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:25.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:25.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:25.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:25.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:25.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:25.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:25.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:25.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:25.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:25.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:25.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:25.994 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:22:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:22:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:22:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:22:25.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:938 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94434111864704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 430.57863153699998, 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85810919399999996, 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 938 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411880+0000", 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411879+0000", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411885+0000", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3550000000000001e-06 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411902+0000", 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6849000000000001e-05 2024-09-17T08:22:25.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.411910+0000", 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4070000000000004e-06 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.412139+0000", 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229559 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-17T08:22:25.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.412144+0000", 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1739999999999999e-06 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.432270+0000", 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020125219 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.432280+0000", 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.006e-05 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:25.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.260117+0000", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.82783703900000005 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for degraded object", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.260397+0000", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000280563 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.266158+0000", 2024-09-17T08:22:26.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057612200000000001 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.266254+0000", 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5954999999999999e-05 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.269990+0000", 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003735258 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218569 7.2 7:44085301:::benchmark_data_smithi079_83551_object91169:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.450076+0000", 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 430.54043653600002, 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0270830379999998, 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218569 2024-09-17T08:22:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450076+0000", 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450076+0000", 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450075+0000", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450078+0000", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6460000000000002e-06 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450084+0000", 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.579e-06 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.450089+0000", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1379999999999998e-06 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.452919+0000", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028294489999999999 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.452931+0000", 2024-09-17T08:22:26.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2415999999999999e-05 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.234698+0000", 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.78176677900000002 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.234712+0000", 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4395e-05 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.467270+0000", 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.232557763 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.467403+0000", 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132371 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2,7", 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.467878+0000", 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000475055 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.474320+0000", 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006442606 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.476909+0000", 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00258858 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.476921+0000", 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2539e-05 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.476955+0000", 2024-09-17T08:22:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4091000000000003e-05 2024-09-17T08:22:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477008+0000", 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2116e-05 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477159+0000", 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015096 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.011 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218577 7.2 7:4f286863:::benchmark_data_smithi079_83551_object91177:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.472813+0000", 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 430.51769936699998, 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0129505480000001, 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218577 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472813+0000", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472813+0000", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472811+0000", 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472817+0000", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6720000000000001e-06 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472828+0000", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1276e-05 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472835+0000", 2024-09-17T08:22:26.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4119999999999998e-06 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472940+0000", 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104851 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.472948+0000", 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8459999999999997e-06 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.234740+0000", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.76179123699999995 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.234747+0000", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6149999999999999e-06 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.474346+0000", 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395985270000001 2024-09-17T08:22:26.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.474460+0000", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114003 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2,7", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.474601+0000", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014162800000000001 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477224+0000", 2024-09-17T08:22:26.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026226449999999998 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.482826+0000", 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005601516 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.482892+0000", 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6061000000000003e-05 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485614+0000", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002722571 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485662+0000", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8168999999999999e-05 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485763+0000", 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100777 2024-09-17T08:22:26.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218587 7.2 7:4b099950:::benchmark_data_smithi079_83551_object91187:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.484815+0000", 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 430.50569695399997, 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0011359390000001, 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:22:26.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218587 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484815+0000", 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484815+0000", 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484813+0000", 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484821+0000", 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8830000000000006e-06 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484837+0000", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5971999999999999e-05 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484849+0000", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.189e-05 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484898+0000", 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9215000000000002e-05 2024-09-17T08:22:26.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.484905+0000", 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.235150+0000", 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.75024495599999996 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.235161+0000", 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0794e-05 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477251+0000", 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2420898600000001 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477356+0000", 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104559 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2,7", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.477476+0000", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120024 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.479925+0000", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024494170000000002 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.482924+0000", 2024-09-17T08:22:26.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002999406 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.482954+0000", 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9869000000000002e-05 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485817+0000", 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028629509999999999 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485860+0000", 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2769999999999999e-05 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.485951+0000", 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1049000000000002e-05 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24698.0:218589 7.0 7:02e079c7:::benchmark_data_smithi079_83551_object91189:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-17T08:22:26.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:15:15.485707+0000", 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 430.50480465999999, 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8904339829999999, 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24698", 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 218589 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485707+0000", 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485707+0000", 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485707+0000", 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485710+0000", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1190000000000001e-06 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485721+0000", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.06e-05 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485725+0000", 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4499999999999997e-06 2024-09-17T08:22:26.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485798+0000", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3289999999999998e-05 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:15.485810+0000", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1338999999999999e-05 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.235459+0000", 2024-09-17T08:22:26.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.74964894100000001 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:16.235470+0000", 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1729000000000001e-05 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.354444+0000", 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1189739430000001 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.356118+0000", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001673414 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2,5", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.356349+0000", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023104700000000001 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.365165+0000", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008816216 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.365175+0000", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0497e-05 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.365468+0000", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029268099999999999 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.375997+0000", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010528449 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.376046+0000", 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9230000000000001e-05 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:15:17.376141+0000", 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5618999999999994e-05 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:74605 8.b 8:df41b961:::benchmark_data_smithi079_115040_object74604:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:17:08.716801+0000", 2024-09-17T08:22:26.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.27371104500003, 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.159992736, 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74605 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716801+0000", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716801+0000", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716800+0000", 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716822+0000", 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2001000000000002e-05 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716838+0000", 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6458000000000001e-05 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716865+0000", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6755999999999999e-05 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.716922+0000", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6674e-05 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717061+0000", 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139104 2024-09-17T08:22:26.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717195+0000", 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013428 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.722800+0000", 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056048540000000003 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.722836+0000", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6019000000000002e-05 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.876526+0000", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15368973699999999 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.876703+0000", 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017766 2024-09-17T08:22:26.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.876794+0000", 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0539999999999997e-05 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:74606 8.f 8:f53a941f:::benchmark_data_smithi079_115040_object74605:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-17T08:22:26.043 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:17:08.717151+0000", 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.27336142299998, 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15786518799999999, 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74606 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717151+0000", 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717151+0000", 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717150+0000", 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717225+0000", 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5099999999999996e-05 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717237+0000", 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1868999999999999e-05 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717244+0000", 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1019999999999999e-06 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717375+0000", 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013145799999999999 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717488+0000", 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112426 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717655+0000", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016769199999999999 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.720375+0000", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002719885 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.720426+0000", 2024-09-17T08:22:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0473999999999998e-05 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.874934+0000", 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15450782900000001 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.874979+0000", 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5522000000000002e-05 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.875016+0000", 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6770000000000002e-05 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15096.0:74608 8.b 8:d9253d67:::benchmark_data_smithi079_115040_object74607:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:17:08.717257+0000", 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.27325516000002, 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17166329299999999, 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15096", 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74608 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717257+0000", 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717257+0000", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717256+0000", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717307+0000", 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0259000000000001e-05 2024-09-17T08:22:26.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717315+0000", 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.473e-06 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.717321+0000", 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8490000000000002e-06 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.719931+0000", 2024-09-17T08:22:26.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002610013 2024-09-17T08:22:26.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:26.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.720242+0000", 2024-09-17T08:22:26.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000311453 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.720396+0000", 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015361899999999999 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.723005+0000", 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026085840000000002 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.723246+0000", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024121400000000001 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.888833+0000", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.165587225 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.888881+0000", 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8189000000000002e-05 2024-09-17T08:22:26.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:17:08.888920+0000", 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8980000000000003e-05 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11661 9.2 9:4a142cba:::benchmark_data_smithi079_143589_object11660:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e226)", 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.52350978600001, 2024-09-17T08:22:26.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1206362190000001, 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11661 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467028+0000", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6152e-05 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467039+0000", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0975000000000001e-05 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467046+0000", 2024-09-17T08:22:26.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7519999999999996e-06 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469412+0000", 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00236514 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469537+0000", 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012545899999999999 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469795+0000", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025807199999999998 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.473388+0000", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035933499999999999 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491273+0000", 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017884118000000001 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491283+0000", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477118+0000", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98583439100000003 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477182+0000", 2024-09-17T08:22:26.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3971000000000001e-05 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.587537+0000", 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1103552290000001 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.587638+0000", 2024-09-17T08:22:26.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101394 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11677 9.2 9:4d7c77eb:::benchmark_data_smithi079_143589_object11676:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e226)", 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.515365428, 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.117803313, 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11677 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475145+0000", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475171+0000", 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5361e-05 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475181+0000", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311e-05 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475188+0000", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8210000000000004e-06 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491728+0000", 2024-09-17T08:22:26.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01654015 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491736+0000", 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2849999999999995e-06 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477216+0000", 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98547963500000002 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477224+0000", 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6859999999999995e-06 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588074+0000", 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1108491460000001 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588183+0000", 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109447 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588290+0000", 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107101 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590678+0000", 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023879299999999999 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591490+0000", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081148099999999996 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592352+0000", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086275299999999996 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592413+0000", 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0192999999999998e-05 2024-09-17T08:22:26.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592950+0000", 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053737399999999997 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11689 9.2 9:44ca1e6e:::benchmark_data_smithi079_143589_object11688:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.498556128, 2024-09-17T08:22:26.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1013493219999999, 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11689 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.078 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491954+0000", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491975+0000", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0701999999999999e-05 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491996+0000", 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.056e-05 2024-09-17T08:22:26.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492007+0000", 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1165000000000001e-05 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492171+0000", 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016403199999999999 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492179+0000", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3280000000000006e-06 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477251+0000", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.985072155 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477260+0000", 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7460000000000006e-06 2024-09-17T08:22:26.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589001+0000", 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1117409949999999 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589129+0000", 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012832800000000001 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:22:26.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589302+0000", 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000172177 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591544+0000", 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022423090000000001 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591633+0000", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9308000000000006e-05 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592991+0000", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013578959999999999 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593025+0000", 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4350000000000001e-05 2024-09-17T08:22:26.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593305+0000", 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027980099999999999 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11704 9.2 9:41d452d7:::benchmark_data_smithi079_143589_object11703:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.489172631, 2024-09-17T08:22:26.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0927345320000001, 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11704 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:22:26.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501338+0000", 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501363+0000", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5066999999999999e-05 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501375+0000", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1566000000000001e-05 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501382+0000", 2024-09-17T08:22:26.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0129999999999996e-06 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501433+0000", 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1002999999999999e-05 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501442+0000", 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8230000000000006e-06 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477286+0000", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97584350399999997 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477294+0000", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6529999999999994e-06 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589843+0000", 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.112549104 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589966+0000", 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012277800000000001 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590114+0000", 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014866099999999999 2024-09-17T08:22:26.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591640+0000", 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015257809999999999 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591683+0000", 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3460000000000001e-05 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593630+0000", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001946779 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593684+0000", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3808999999999999e-05 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594074+0000", 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038991200000000002 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.094 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11749 9.2 9:4b73f331:::benchmark_data_smithi079_143589_object11748:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.46528474199999, 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0692497840000001, 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11749 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525226+0000", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525248+0000", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1557999999999999e-05 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525258+0000", 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5680000000000005e-06 2024-09-17T08:22:26.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525264+0000", 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3810000000000004e-06 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525422+0000", 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015760700000000001 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525431+0000", 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5009999999999993e-06 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477317+0000", 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95188626399999998 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477326+0000", 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4749999999999993e-06 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590708+0000", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113382469 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590842+0000", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013373000000000001 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590997+0000", 2024-09-17T08:22:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015488500000000001 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591711+0000", 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071417900000000001 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591929+0000", 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000218123 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594137+0000", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022075100000000002 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594199+0000", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1870000000000002e-05 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594477+0000", 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027855799999999998 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1625 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94434111870464 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e225)", 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:22:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.03837980500001, 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53841006800000002, 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:22:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1625 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952135+0000", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6540000000000001e-06 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952142+0000", 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9369999999999998e-06 2024-09-17T08:22:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952147+0000", 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1019999999999996e-06 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952242+0000", 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4555999999999994e-05 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952254+0000", 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1989e-05 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490303+0000", 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53804894000000003 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490454+0000", 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015072999999999999 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490542+0000", 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8702000000000006e-05 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1631 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94434111870464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.03767195899999, 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54383248699999998, 2024-09-17T08:22:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1631 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952839+0000", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952846+0000", 2024-09-17T08:22:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6100000000000002e-06 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952865+0000", 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9641999999999999e-05 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952872+0000", 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2840000000000001e-06 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952988+0000", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115812 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952998+0000", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0182e-05 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490843+0000", 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53784467899999999 2024-09-17T08:22:26.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490913+0000", 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0266999999999997e-05 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.496673+0000", 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057591739999999997 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:177040 9.4 e266/221 9:253eef9f:::benchmark_data_smithi079_143589_object45841:head v 266'11071, mlcod=266'11068)", 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:23.992443+0000", 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.998068755, 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18664249099999999, 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992443+0000", 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992443+0000", 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992442+0000", 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992449+0000", 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.131e-06 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992461+0000", 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1785e-05 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.992468+0000", 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.994631+0000", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021633030000000001 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.994650+0000", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8688000000000001e-05 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179044+0000", 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18439449299999999 2024-09-17T08:22:26.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179086+0000", 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1349e-05 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:177045 9.3 e266/221 9:ced929c2:::benchmark_data_smithi079_143589_object45846:head v 266'11322, mlcod=266'11320)", 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:23.997167+0000", 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.993345114, 2024-09-17T08:22:26.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18451585700000001, 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997167+0000", 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997167+0000", 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997166+0000", 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997173+0000", 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0110000000000003e-06 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997182+0000", 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6819999999999992e-06 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997188+0000", 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8000000000000004e-06 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179063+0000", 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18187505900000001 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179088+0000", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5154e-05 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.181615+0000", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025269250000000002 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.181683+0000", 2024-09-17T08:22:26.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7990000000000005e-05 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:177047 9.4 e266/221 9:21a05e5a:::benchmark_data_smithi079_143589_object45848:head v 266'11072, mlcod=266'11068)", 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:23.997241+0000", 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.9932711830000001, 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.19228737700000001, 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997241+0000", 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997241+0000", 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997240+0000", 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.997246+0000", 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4759999999999997e-06 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179033+0000", 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18178724900000001 2024-09-17T08:22:26.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179046+0000", 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3027000000000001e-05 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179136+0000", 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9400000000000005e-05 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179155+0000", 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9732e-05 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.189498+0000", 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010343110000000001 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.189528+0000", 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9828e-05 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:177050 9.1 9:8fbf13d4:::benchmark_data_smithi079_143589_object45851:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:23.999505+0000", 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.991007387, 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.183356465, 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:22:26.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 177050 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999505+0000", 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999505+0000", 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999503+0000", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999511+0000", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7940000000000003e-06 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179038+0000", 2024-09-17T08:22:26.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.179527452 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179053+0000", 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4705000000000001e-05 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179104+0000", 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179245+0000", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014137700000000001 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179362+0000", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117067 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.181887+0000", 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025246859999999999 2024-09-17T08:22:26.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.182141+0000", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000253935 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.182581+0000", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044069600000000002 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.182791+0000", 2024-09-17T08:22:26.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020932200000000001 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.182861+0000", 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0357000000000004e-05 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:177048 9.7 e266/221 9:e39b6dc4:::benchmark_data_smithi079_143589_object45849:head v 266'11174, mlcod=266'11172)", 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:23.999704+0000", 2024-09-17T08:22:26.135 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.9908077579999999, 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18457237500000001, 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999704+0000", 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999704+0000", 2024-09-17T08:22:26.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999703+0000", 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:23.999713+0000", 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6609999999999995e-06 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179041+0000", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.179327554 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179049+0000", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3159999999999997e-06 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179145+0000", 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6025000000000003e-05 2024-09-17T08:22:26.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.179165+0000", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0174999999999999e-05 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.184244+0000", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050790080000000003 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:24.184277+0000", 2024-09-17T08:22:26.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2502000000000002e-05 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:22:26.140 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:22:26.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.145 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:26.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:26.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:26.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:26.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:26.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:26.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.417 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:26.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:26.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:26.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:26.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:22:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[29565]: pgmap v552: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:22:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:26 smithi079 ceph-mon[40649]: pgmap v552: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-17T08:22:26.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:26.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.691 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:26.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:22:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:26 smithi173 ceph-mon[33187]: pgmap v552: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-17T08:22:26.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:26.823 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:26.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:26.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:26.955 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:26.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:26.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:26.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:27.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:27.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.105 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:27.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:27.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:27.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.240 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:27.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:27.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:27.373 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:27.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:27.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[29565]: 2.13 deep-scrub starts 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[29565]: 2.13 deep-scrub ok 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[40649]: 2.13 deep-scrub starts 2024-09-17T08:22:27.472 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:27 smithi079 ceph-mon[40649]: 2.13 deep-scrub ok 2024-09-17T08:22:27.475 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:27.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:27.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:27.607 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:27.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:27.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:27.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:27 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:22:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:27 smithi173 ceph-mon[33187]: 2.13 deep-scrub starts 2024-09-17T08:22:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:27 smithi173 ceph-mon[33187]: 2.13 deep-scrub ok 2024-09-17T08:22:27.735 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:27.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:27.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:27.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:27.892 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:27.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:27.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:27.994 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:28.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:28.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:28.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:28.156 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:28.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:28.258 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:28.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:28.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:28.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:28.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:28.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:28.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:22:28.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:28.562 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:28.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:28.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:22:28.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: pgmap v553: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 576 B/s rd, 0 B/s wr, 707 op/s 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[29565]: 9.c scrub starts 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:22:28.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:22:28.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: pgmap v553: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 576 B/s rd, 0 B/s wr, 707 op/s 2024-09-17T08:22:28.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:28 smithi079 ceph-mon[40649]: 9.c scrub starts 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: pgmap v553: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 576 B/s rd, 0 B/s wr, 707 op/s 2024-09-17T08:22:28.630 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:28 smithi173 ceph-mon[33187]: 9.c scrub starts 2024-09-17T08:22:28.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:28.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:28.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:28.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:28.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:28.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:28.911 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:28.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:28.970 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:28.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:28.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:29.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:29.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:29.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:29.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:29.235 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:29.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:29.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:29.373 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:29.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.374 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:29.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:29.509 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:29.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:29.605 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:22:29.606 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:29 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:22:29.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:29.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.672 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:29.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:29 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:22:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:29 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:22:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:29 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:22:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:29 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:22:29.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:29.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.806 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:29.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:29.907 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:29.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:29.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:29.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:30.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:30.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:30.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:30.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:30.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:30.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:30.347 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:30.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:30.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:22:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:22:30.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 9.c scrub ok 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: pgmap v554: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:30.475 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:30 smithi173 ceph-mon[33187]: 9.a scrub starts 2024-09-17T08:22:30.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:30.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:30.491 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:22:30.491 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:22:30.491 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:22:30.491 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 9.c scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: pgmap v554: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[29565]: 9.a scrub starts 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 9.c scrub ok 2024-09-17T08:22:30.492 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:22:30.493 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:22:30.493 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: pgmap v554: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:30.493 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:30 smithi079 ceph-mon[40649]: 9.a scrub starts 2024-09-17T08:22:30.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:30.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:30.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:30.713 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:30.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:30.792 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:30.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:30.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:30.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:30.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:30.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:31.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:31.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:31.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:185503 9.c e267/221 9:38052f38:::benchmark_data_smithi079_143589_object54304:head v 267'11654, mlcod=267'11652)", 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:31.174983+0000", 2024-09-17T08:22:31.178 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0020154790000000001, 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020392560000000001, 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.174983+0000", 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.174983+0000", 2024-09-17T08:22:31.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.174982+0000", 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.174989+0000", 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1999999999999997e-06 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.175004+0000", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4452e-05 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.175011+0000", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.08e-06 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.175052+0000", 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1026999999999999e-05 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:31.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:31.175073+0000", 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1469000000000001e-05 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2024-09-17T08:22:31.182 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:22:31.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:31.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:31.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:31.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:31.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:31.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:31.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 9.5 scrub starts 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 5.f deep-scrub starts 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 5.f deep-scrub ok 2024-09-17T08:22:31.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:31 smithi173 ceph-mon[33187]: 9.8 scrub starts 2024-09-17T08:22:31.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:31.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:31.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:31.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 9.5 scrub starts 2024-09-17T08:22:31.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:22:31.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:22:31.592 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 5.f deep-scrub starts 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 5.f deep-scrub ok 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[29565]: 9.8 scrub starts 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 9.5 scrub starts 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 5.f deep-scrub starts 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 5.f deep-scrub ok 2024-09-17T08:22:31.593 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:31 smithi079 ceph-mon[40649]: 9.8 scrub starts 2024-09-17T08:22:31.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:31.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:31.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:31.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:31.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:31.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:31.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:31.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:31.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:31.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:22:32.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:32.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:32.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:32.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:32.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.189 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:32.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:32.276 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:32.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:32.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:32.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:32.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:32.466 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:32.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.467 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:32.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:32.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.604 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:32.604 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: 9.a scrub ok 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: pgmap v555: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: 9.5 scrub ok 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[29565]: 9.8 scrub ok 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: 9.a scrub ok 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: pgmap v555: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:32.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: 9.5 scrub ok 2024-09-17T08:22:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:22:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:22:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:32.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:32 smithi079 ceph-mon[40649]: 9.8 scrub ok 2024-09-17T08:22:32.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: 9.a scrub ok 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: pgmap v555: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: 9.5 scrub ok 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:32.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-mon[33187]: 9.8 scrub ok 2024-09-17T08:22:32.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:32.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.740 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:32.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:32.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:32.875 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:32.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:32.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:33.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:33.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.137 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:33.200 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:33.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:33.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:33.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:33.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:33.392 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:33.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:33 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:22:33.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:33 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:22:33.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:33 smithi173 ceph-mon[33187]: 9.e deep-scrub starts 2024-09-17T08:22:33.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:33 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:22:33.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:33 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:22:33.493 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:33.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.532 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:33.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:33.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:33.666 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:33.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:33.790 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[40649]: 9.e deep-scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[29565]: 9.e deep-scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:22:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:33 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:22:33.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:33.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:33.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:33.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:34.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:34.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:34.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:34.165 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:34.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:34.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:34.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:34.312 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:22:34.312 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:22:34.312 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:22:34.313 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:22:34.313 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:22:34.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:34.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:34.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:34.452 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:34.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:34.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:34.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:34.688 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:34.688 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:22:34.688 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:22:34.688 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: 5.9 deep-scrub starts 2024-09-17T08:22:34.689 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: 5.9 deep-scrub ok 2024-09-17T08:22:34.689 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: pgmap v556: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:22:34.689 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:34 smithi173 ceph-mon[33187]: 9.0 scrub starts 2024-09-17T08:22:34.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:34.723 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:34.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:34.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:34.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:34.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: 5.9 deep-scrub starts 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: 5.9 deep-scrub ok 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: pgmap v556: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[40649]: 9.0 scrub starts 2024-09-17T08:22:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: 5.9 deep-scrub starts 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: 5.9 deep-scrub ok 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: pgmap v556: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-17T08:22:34.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:34 smithi079 ceph-mon[29565]: 9.0 scrub starts 2024-09-17T08:22:34.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:34.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:34.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:34.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:35.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:35.126 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:35.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:35.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:35.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.260 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:35.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:35.303 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:22:35.361 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:35.400 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:35.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:35.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:35.535 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:35.601 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:35 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:22:35.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:35 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:22:35.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:35 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:22:35.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:35 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:22:35.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:35.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.675 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:35.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:35.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:35 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:22:35.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:35 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:22:35.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:35.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:35.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:35.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:35.936 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:35.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:35.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:36.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:36.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:36.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:36.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:36.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.211 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:36.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:36.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:36.351 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:36.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:36.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:36.487 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:36.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:36.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:36.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[40649]: 9.0 scrub ok 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[40649]: pgmap v557: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[40649]: 9.1 scrub starts 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[29565]: 9.0 scrub ok 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[29565]: pgmap v557: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:22:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:36 smithi079 ceph-mon[29565]: 9.1 scrub starts 2024-09-17T08:22:36.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:36 smithi173 ceph-mon[33187]: 9.0 scrub ok 2024-09-17T08:22:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:36 smithi173 ceph-mon[33187]: pgmap v557: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:22:36.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:36 smithi173 ceph-mon[33187]: 9.1 scrub starts 2024-09-17T08:22:36.752 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:36.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:36.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:36.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:36.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:36.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:37.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:37.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:37.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:37.166 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:37.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:37.268 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:37.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:37.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:37.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:37.437 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:37.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:37.453 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T08:22:37.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:37.581 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:37.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:37.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:37.711 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:37.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: 4.1a deep-scrub starts 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: 4.1a deep-scrub ok 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/731975483' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:37 smithi173 ceph-mon[33187]: 9.1 scrub ok 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: 4.1a deep-scrub starts 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: 4.1a deep-scrub ok 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/731975483' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[29565]: 9.1 scrub ok 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:22:37.777 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: 4.1a deep-scrub starts 2024-09-17T08:22:37.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: 4.1a deep-scrub ok 2024-09-17T08:22:37.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/731975483' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:22:37.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:37 smithi079 ceph-mon[40649]: 9.1 scrub ok 2024-09-17T08:22:37.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:37.847 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:37.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:37.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:37.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:38.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.006 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:38.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:38.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:38.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68437 7.1 7:8794a2e1:::benchmark_data_smithi079_83551_object68436:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 583.40457861300001, 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11749947199999999, 2024-09-17T08:22:38.171 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68437 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.172 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762023+0000", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762022+0000", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762153+0000", 2024-09-17T08:22:38.173 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131729 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762168+0000", 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4851999999999999e-05 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762174+0000", 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6439999999999997e-06 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.174 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762209+0000", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5030999999999997e-05 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762299+0000", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0492e-05 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.762395+0000", 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5126999999999999e-05 2024-09-17T08:22:38.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764253+0000", 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0018581089999999999 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.764600+0000", 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034766899999999999 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.176 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879354+0000", 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11475331499999999 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879444+0000", 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0832e-05 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.879522+0000", 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7851999999999999e-05 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.177 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68447 7.1 7:81e8cead:::benchmark_data_smithi079_83551_object68446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 583.39841865999995, 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.116018159, 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:38.178 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68447 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768183+0000", 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.179 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768181+0000", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768276+0000", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5210999999999994e-05 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768289+0000", 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2994e-05 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.180 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768296+0000", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8510000000000001e-06 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768336+0000", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9638999999999998e-05 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768469+0000", 2024-09-17T08:22:38.181 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132881 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.768636+0000", 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016691399999999999 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771127+0000", 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002491643 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.182 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771774+0000", 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064712199999999998 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884106+0000", 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11233185 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884160+0000", 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3644999999999999e-05 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.183 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.884201+0000", 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1152000000000002e-05 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:68453 7.1 7:847278dd:::benchmark_data_smithi079_83551_object68452:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 583.39466999399997, 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.115527932, 2024-09-17T08:22:38.184 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 68453 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.185 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771931+0000", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.771930+0000", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772008+0000", 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7849999999999995e-05 2024-09-17T08:22:38.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772019+0000", 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1488000000000001e-05 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772025+0000", 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1270000000000002e-06 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.187 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772067+0000", 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2552000000000002e-05 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772178+0000", 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110838 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.772272+0000", 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3548999999999995e-05 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.188 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774206+0000", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001934727 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.774634+0000", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00042775400000000002 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887387+0000", 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.112753373 2024-09-17T08:22:38.189 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887423+0000", 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5494999999999998e-05 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:12:54.887459+0000", 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6498999999999997e-05 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183015 7.1 7:881aa8e5:::benchmark_data_smithi079_83551_object55615:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 476.416653967, 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.110799543, 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183015 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.191 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749947+0000", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749946+0000", 2024-09-17T08:22:38.192 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749953+0000", 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1389999999999998e-06 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749968+0000", 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4815e-05 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.193 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.749975+0000", 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7070000000000003e-06 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750019+0000", 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4474000000000001e-05 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750110+0000", 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0523e-05 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.194 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.750189+0000", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9081999999999999e-05 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752660+0000", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024708320000000001 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.752849+0000", 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018942199999999999 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.195 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860669+0000", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10781995900000001 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860711+0000", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2150000000000001e-05 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.860747+0000", 2024-09-17T08:22:38.196 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5818999999999997e-05 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24698.0:183025 7.1 7:819884a4:::benchmark_data_smithi079_83551_object55625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 476.40757271500001, 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10716666699999999, 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24698", 2024-09-17T08:22:38.197 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1398222769", 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 183025 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759029+0000", 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.198 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759027+0000", 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.199 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759036+0000", 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9169999999999995e-06 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759053+0000", 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6585000000000001e-05 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.200 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759062+0000", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0010000000000007e-06 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759107+0000", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5754000000000003e-05 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759217+0000", 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000110092 2024-09-17T08:22:38.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,6", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.759305+0000", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7967999999999995e-05 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761460+0000", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002154885 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.761625+0000", 2024-09-17T08:22:38.202 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000164461 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866070+0000", 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.104445353 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866155+0000", 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5018000000000005e-05 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.203 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:14:41.866195+0000", 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0274000000000001e-05 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35137 8.2 8:468fa297:::benchmark_data_smithi079_115040_object35136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 354.823354585, 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.131430191, 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.204 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35137 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343247+0000", 2024-09-17T08:22:38.205 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343245+0000", 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343349+0000", 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103587 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.206 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343366+0000", 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7232999999999999e-05 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343373+0000", 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3030000000000002e-06 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343441+0000", 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7742000000000003e-05 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.207 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343563+0000", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012195499999999999 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.343696+0000", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132664 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345071+0000", 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013757260000000001 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.208 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.345717+0000", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00064516599999999997 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474616+0000", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12889890800000001 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474651+0000", 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5364000000000002e-05 2024-09-17T08:22:38.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.474677+0000", 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6166e-05 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:35143 8.2 8:48b7532d:::benchmark_data_smithi079_115040_object35142:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 354.82031262499999, 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.13410802799999999, 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.210 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 35143 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.211 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346289+0000", 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346288+0000", 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346342+0000", 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3986000000000001e-05 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.212 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346353+0000", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1019000000000001e-05 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346361+0000", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7889999999999995e-06 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346403+0000", 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2738999999999999e-05 2024-09-17T08:22:38.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346519+0000", 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115482 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.346671+0000", 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015182600000000001 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.214 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.347889+0000", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012181220000000001 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.349481+0000", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001591631 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480320+0000", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.130839172 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.215 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480362+0000", 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1878000000000001e-05 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:43.480397+0000", 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5224e-05 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:36507 8.2 8:46403afa:::benchmark_data_smithi079_115040_object36506:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e176)", 2024-09-17T08:22:38.216 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 354.01806745200003, 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14247363399999999, 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 36507 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:22:38.217 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148534+0000", 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148533+0000", 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.218 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148580+0000", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7367999999999998e-05 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148593+0000", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2622e-05 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148598+0000", 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6359999999999999e-06 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148634+0000", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5598e-05 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148725+0000", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0792999999999996e-05 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.148820+0000", 2024-09-17T08:22:38.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5533999999999997e-05 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.149972+0000", 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001151727 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.150108+0000", 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013644899999999999 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290944+0000", 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.14083580900000001 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.290980+0000", 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5593000000000003e-05 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:16:44.291008+0000", 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7813e-05 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.222 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:139803 8.4 8:2f12791c:::benchmark_data_smithi079_115040_object4783:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 274.68912955000002, 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.103915984, 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 139803 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477472+0000", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477471+0000", 2024-09-17T08:22:38.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477474+0000", 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4329999999999998e-06 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477481+0000", 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0249999999999997e-06 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477485+0000", 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5539999999999999e-06 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477562+0000", 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6897999999999995e-05 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477666+0000", 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103719 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477769+0000", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102635 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.480755+0000", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002986107 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.486665+0000", 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0059098589999999999 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581308+0000", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.094643730999999995 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581355+0000", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6374e-05 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.581388+0000", 2024-09-17T08:22:38.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3050999999999999e-05 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:139805 8.5 8:a4d9555f:::benchmark_data_smithi079_115040_object4785:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 274.688877386, 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.102633397, 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.229 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 139805 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477724+0000", 2024-09-17T08:22:38.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477723+0000", 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477728+0000", 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2950000000000004e-06 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477737+0000", 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6590000000000002e-06 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477743+0000", 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9479999999999996e-06 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477778+0000", 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5317000000000001e-05 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.477914+0000", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013540199999999999 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.478013+0000", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9325000000000002e-05 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.484825+0000", 2024-09-17T08:22:38.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0068114040000000001 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.488487+0000", 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036620569999999998 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580242+0000", 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091755428 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580316+0000", 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3919999999999997e-05 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:03.580357+0000", 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1443999999999998e-05 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:205896 8.2 8:48183680:::benchmark_data_smithi079_115040_object70876:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:22:38.235 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 217.015295219, 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12536080999999999, 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 205896 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151306+0000", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151305+0000", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151312+0000", 2024-09-17T08:22:38.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7580000000000003e-06 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151325+0000", 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3677e-05 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151332+0000", 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1840000000000002e-06 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151449+0000", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116208 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151569+0000", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120178 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151680+0000", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000111673 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155861+0000", 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041803700000000001 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155894+0000", 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3634999999999999e-05 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276563+0000", 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12066801000000001 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276626+0000", 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3219999999999994e-05 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.276667+0000", 2024-09-17T08:22:38.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1294999999999997e-05 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:205897 8.2 8:4ad2b373:::benchmark_data_smithi079_115040_object70877:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 217.01524993000001, 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12573102799999999, 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.243 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 205897 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151352+0000", 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151351+0000", 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151354+0000", 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.841e-06 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151363+0000", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5570000000000001e-06 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.151368+0000", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5870000000000003e-06 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.153985+0000", 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026173799999999999 2024-09-17T08:22:38.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.154064+0000", 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9078000000000004e-05 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.154138+0000", 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3314999999999999e-05 2024-09-17T08:22:38.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155918+0000", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001779751 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.155991+0000", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3767e-05 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277006+0000", 2024-09-17T08:22:38.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12101434899999999 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277047+0000", 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1094000000000003e-05 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:01.277083+0000", 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5809999999999998e-05 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212175 8.2 8:42f488fd:::benchmark_data_smithi079_115040_object77155:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.53144086500001, 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10989600099999999, 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212175 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635161+0000", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635159+0000", 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635168+0000", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8929999999999994e-06 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635182+0000", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3987e-05 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.635190+0000", 2024-09-17T08:22:38.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8939999999999993e-06 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636415+0000", 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012254799999999999 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636516+0000", 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010071199999999999 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636616+0000", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010017399999999999 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639458+0000", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028418710000000002 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.645864+0000", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064057109999999997 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.744926+0000", 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099062219000000007 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745018+0000", 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1884000000000003e-05 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745057+0000", 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8574e-05 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.255 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212180 8.2 8:49a422e1:::benchmark_data_smithi079_115040_object77160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.53012295799999, 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.108731759, 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212180 2024-09-17T08:22:38.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636478+0000", 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636477+0000", 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636483+0000", 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8939999999999998e-06 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636495+0000", 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2156e-05 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636502+0000", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.725e-06 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639480+0000", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029782770000000001 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639577+0000", 2024-09-17T08:22:38.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7027000000000005e-05 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.639676+0000", 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8287000000000003e-05 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642500+0000", 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002824769 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648886+0000", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0063861100000000004 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745137+0000", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096250496000000005 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745177+0000", 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0241000000000003e-05 2024-09-17T08:22:38.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745210+0000", 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3050999999999999e-05 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212181 8.2 8:47e87e5d:::benchmark_data_smithi079_115040_object77161:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.53009351, 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10882399, 2024-09-17T08:22:38.262 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212181 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636508+0000", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636507+0000", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636510+0000", 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.323e-06 2024-09-17T08:22:38.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636515+0000", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2089999999999996e-06 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.636519+0000", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2459999999999997e-06 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642517+0000", 2024-09-17T08:22:38.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00599783 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642629+0000", 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011173800000000001 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.642749+0000", 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000120536 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648894+0000", 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0061442880000000004 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648925+0000", 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1751999999999997e-05 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745256+0000", 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.096330490000000005 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745291+0000", 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5256999999999998e-05 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745332+0000", 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0760999999999998e-05 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212193 8.4 8:2d0a4b0d:::benchmark_data_smithi079_115040_object77173:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.268 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.522710577, 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.106655758, 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212193 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:22:38.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643891+0000", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643890+0000", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643893+0000", 2024-09-17T08:22:38.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7329999999999999e-06 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643899+0000", 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1759999999999998e-06 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.643904+0000", 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.028e-06 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.271 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.649054+0000", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0051491879999999999 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738041+0000", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.088987219000000006 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738172+0000", 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000130827 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750356+0000", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012184053 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750399+0000", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3609000000000002e-05 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750452+0000", 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2630000000000003e-05 2024-09-17T08:22:38.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.750547+0000", 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4753000000000006e-05 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212197 8.2 8:4ffe1ab7:::benchmark_data_smithi079_115040_object77177:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.52039581899999, 2024-09-17T08:22:38.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.107564755, 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212197 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646206+0000", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646204+0000", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646211+0000", 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2430000000000002e-06 2024-09-17T08:22:38.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646220+0000", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8119999999999997e-06 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.646227+0000", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7640000000000005e-06 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648948+0000", 2024-09-17T08:22:38.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00272098 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.649078+0000", 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012928700000000001 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.738858+0000", 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.089780486000000007 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745099+0000", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0062409049999999997 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745222+0000", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000123495 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753521+0000", 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0082982349999999993 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753639+0000", 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118638 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.753770+0000", 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000131023 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.280 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15096.0:212199 8.2 8:4d5a0918:::benchmark_data_smithi079_115040_object77179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 212.51848619699999, 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11034735599999999, 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15096", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/811211303", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 212199 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648115+0000", 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648114+0000", 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648122+0000", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3650000000000004e-06 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648132+0000", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0039e-05 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.648138+0000", 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.699e-06 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741641+0000", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093503257000000006 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741745+0000", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010346 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,5", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.741848+0000", 2024-09-17T08:22:38.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102626 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.745342+0000", 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0034946619999999999 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.751505+0000", 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0061626529999999997 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:22:38.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758277+0000", 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0067723999999999996 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758427+0000", 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015008299999999999 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:19:05.758463+0000", 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5253000000000003e-05 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.286 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(osd.7.0:80799 9.e e267/221 9:713c9782:::benchmark_data_smithi079_143589_object130794:head v 267'12042, mlcod=267'12034)", 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:34.314246+0000", 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 3.8523555279999999, 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022810904, 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314246+0000", 2024-09-17T08:22:38.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314246+0000", 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314245+0000", 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314253+0000", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5709999999999998e-06 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314266+0000", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.24e-05 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314273+0000", 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1359999999999999e-06 2024-09-17T08:22:38.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314314+0000", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1563999999999997e-05 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.314333+0000", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8941e-05 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.337027+0000", 2024-09-17T08:22:38.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022693231000000001 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:34.337057+0000", 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0229000000000001e-05 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:190512 9.e e267/221 9:795fc3e9:::benchmark_data_smithi079_143589_object59313:head v 267'12441, mlcod=267'12422)", 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:36.227789+0000", 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.938812209, 2024-09-17T08:22:38.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022298259000000001, 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227789+0000", 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227789+0000", 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:38.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227788+0000", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227796+0000", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6140000000000003e-06 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227807+0000", 2024-09-17T08:22:38.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1229e-05 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.227814+0000", 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.725e-06 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.229961+0000", 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021467600000000002 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:38.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.230014+0000", 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3658e-05 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.249908+0000", 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019893714 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.250087+0000", 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017960599999999999 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.295 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:38.296 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:38.296 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:22:38.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:38.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:38.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:38.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.431 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:38.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:38.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:38.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:38.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:38.690 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:38.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:38 smithi173 ceph-mon[33187]: 9.9 scrub starts 2024-09-17T08:22:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:38 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:22:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:38 smithi173 ceph-mon[33187]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T08:22:38.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:38 smithi173 ceph-mon[33187]: pgmap v559: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:22:38.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:38.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.824 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:38.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:38.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[40649]: 9.9 scrub starts 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[40649]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[40649]: pgmap v559: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[29565]: 9.9 scrub starts 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[29565]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T08:22:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:38 smithi079 ceph-mon[29565]: pgmap v559: 148 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 146 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:22:38.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:38.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:38.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:38.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:39.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:39.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:39.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:39.181 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:22:39.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:39.211 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:39.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:39.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:39.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:39.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:39.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:39.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:39.485 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:39.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:39.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:39.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:39.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:39 smithi173 ceph-mon[33187]: 9.9 scrub ok 2024-09-17T08:22:39.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:39 smithi173 ceph-mon[33187]: 9.d scrub starts 2024-09-17T08:22:39.730 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:39.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.731 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:39.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:39 smithi079 ceph-mon[40649]: 9.9 scrub ok 2024-09-17T08:22:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:39 smithi079 ceph-mon[40649]: 9.d scrub starts 2024-09-17T08:22:39.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:39.870 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:39.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:39.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:39 smithi079 ceph-mon[29565]: 9.9 scrub ok 2024-09-17T08:22:39.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:39 smithi079 ceph-mon[29565]: 9.d scrub starts 2024-09-17T08:22:39.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:40.006 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:40.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:40.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.144 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.144 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:40.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-17T08:22:40.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:40.277 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.278 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:40.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:40.403 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:40.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:40.525 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:22:40.525 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:22:40.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.555 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:40.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:40.565 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:22:40.565 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:22:40.604 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:22:40.604 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:22:40.636 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:22:40.636 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:22:40.657 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:40.669 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:22:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:22:40.686 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:40.699 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:22:40.700 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: pgmap v560: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:22:40.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: pgmap v560: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:22:40.701 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:40 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: pgmap v560: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:22:40.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:40 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:22:40.730 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:22:40.730 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:22:40.763 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:22:40.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:22:40.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:40.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:40.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:40.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:40.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:40.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:41.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:41.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:41.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:41.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:41.192 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:41.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:41.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:41.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:41.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:41.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:41.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:41.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:41.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:41.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.588 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:41.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:41.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:41 smithi173 ceph-mon[33187]: 9.f scrub starts 2024-09-17T08:22:41.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:41 smithi173 ceph-mon[33187]: 9.d scrub ok 2024-09-17T08:22:41.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:41 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:22:41.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:41 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:22:41.652 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:41 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/236748030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:41.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:41.698 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:22:41.698 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:22:41.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:41.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:41.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[40649]: 9.f scrub starts 2024-09-17T08:22:41.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[40649]: 9.d scrub ok 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/236748030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[29565]: 9.f scrub starts 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[29565]: 9.d scrub ok 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:22:41.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:41 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/236748030' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:41.823 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:41.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:41.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:41.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:41.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:41.999 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:41.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:42.101 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:42.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:42.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:42.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:42.264 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:42.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:42.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:42.401 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:42.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:42.484 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:22:42.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:22:42.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: 9.f scrub ok 2024-09-17T08:22:42.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: pgmap v561: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:22:42.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: 9.3 scrub starts 2024-09-17T08:22:42.485 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[29565]: 9.6 scrub starts 2024-09-17T08:22:42.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:42.535 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:42.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:42.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:42.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:22:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:22:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: 9.f scrub ok 2024-09-17T08:22:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: pgmap v561: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:22:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: 9.3 scrub starts 2024-09-17T08:22:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-mon[33187]: 9.6 scrub starts 2024-09-17T08:22:42.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:22:42.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:22:42.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: 9.f scrub ok 2024-09-17T08:22:42.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: pgmap v561: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-17T08:22:42.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: 9.3 scrub starts 2024-09-17T08:22:42.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:42 smithi079 ceph-mon[40649]: 9.6 scrub starts 2024-09-17T08:22:42.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:42.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:42.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:22:42.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:42.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:42.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:42.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:43.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:43.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:43.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:43.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:43.179 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:43.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:43.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:43.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:43.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.346 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:43.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:43.447 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:43.481 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:43.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:43.515 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:22:43.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:43.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:43.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:43.659 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:43 smithi173 ceph-mon[33187]: 9.b scrub starts 2024-09-17T08:22:43.659 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:43 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:22:43.659 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:43 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:22:43.659 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:43 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:22:43.659 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:43 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:22:43.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:43.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.749 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:43.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:43.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[40649]: 9.b scrub starts 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[29565]: 9.b scrub starts 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:22:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:22:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:22:43.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:43 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:22:43.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:43.890 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:43.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:43.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:44.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:44.026 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:44.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:44.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:44.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:44.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:44.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:44.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:44.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:22:44.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:44.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:44.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:44.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:44.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:44.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: 9.3 scrub ok 2024-09-17T08:22:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: 9.b scrub ok 2024-09-17T08:22:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: pgmap v562: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 993 op/s 2024-09-17T08:22:44.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:22:44.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[29565]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T08:22:44.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: 9.3 scrub ok 2024-09-17T08:22:44.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:44.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: 9.b scrub ok 2024-09-17T08:22:44.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: pgmap v562: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 993 op/s 2024-09-17T08:22:44.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:44.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:22:44.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:44 smithi079 ceph-mon[40649]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T08:22:44.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:44.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.693 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:44.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: 9.3 scrub ok 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: 9.b scrub ok 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: pgmap v562: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 993 op/s 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/688099395' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:22:44.766 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:44 smithi173 ceph-mon[33187]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T08:22:44.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:44.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:44.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:44.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:44.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:44.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:44.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:45.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:45.087 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:45.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.088 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:45.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:45.226 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:45.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:45.292 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:22:45.293 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:22:45.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:45.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:45.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:45.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:45.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:45.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:45.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:45.629 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:45.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:45.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:45.765 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:45.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:45.867 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:45.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:45.904 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:45.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:46.005 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:46.040 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:46.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:46.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:22:46.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:46.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:46.285 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:46.334 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:46.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:46.414 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:46.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:46.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.498 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:46.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:46.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[29565]: pgmap v564: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-09-17T08:22:46.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:22:46.505 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:22:46.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:46.628 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:46.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:46.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:46.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[40649]: pgmap v564: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-09-17T08:22:46.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:22:46.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:46 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:22:46.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:46.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:46.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:46 smithi173 ceph-mon[33187]: pgmap v564: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s 2024-09-17T08:22:46.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:46 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:22:46.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:46 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:22:46.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:46.907 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:46.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:46.908 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:47.009 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:47.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:47.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:47.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:47.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:47.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:47.201 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:47.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:47.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:47.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:47.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:47.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:47.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:47.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:47.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.590 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:47.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:47.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[40649]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1985113433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[29565]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1985113433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:47 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:47.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.714 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:47.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:47 smithi173 ceph-mon[33187]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T08:22:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:22:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:22:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1985113433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:47 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:22:47.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:47.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:47.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:47.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:47.978 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:47.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:48.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:48.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:48.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:48.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:48.232 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T08:22:48.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.241 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:48.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:48.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:22:48.342 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:48.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:48.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:48.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:48.527 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:48.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:48.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:48.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.663 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:48.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:48.765 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:48.782 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:48 smithi173 ceph-mon[33187]: pgmap v566: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 617 op/s 2024-09-17T08:22:48.782 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:48 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:22:48.782 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:48 smithi173 ceph-mon[33187]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T08:22:48.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.809 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:48.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:48.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[40649]: pgmap v566: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 617 op/s 2024-09-17T08:22:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:22:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[40649]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T08:22:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[29565]: pgmap v566: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 617 op/s 2024-09-17T08:22:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:22:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:48 smithi079 ceph-mon[29565]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T08:22:48.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:48.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:48.949 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:48.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:49.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:49.080 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:49.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:49.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:49.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:49.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.211 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:49.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:49.342 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:49.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:49.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:49.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:49.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:49.607 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:49.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:49.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:49.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:49.784 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:49.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.785 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:49.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:49.919 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:49.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:49.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:50.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:50.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:50.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:50.155 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T08:22:50.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:22:50.156 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:50.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:50.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:50.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:50.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:50.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:50.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:50.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.452 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:50.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:50.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:50.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:50.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:50.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:50 smithi079 ceph-mon[29565]: pgmap v568: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:22:50.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:50 smithi079 ceph-mon[40649]: pgmap v568: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:22:50.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:50.712 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:50.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:50.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:50 smithi173 ceph-mon[33187]: pgmap v568: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:22:50.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:50.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:50.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:50.981 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:50.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:50.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:51.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:51.107 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:22:51.107 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:22:51.117 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:51.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:51.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:51.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:51.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:51.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:51.368 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:51.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:51.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:51.501 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:51.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:51.603 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:51.636 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:51.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:51.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:51.777 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:51.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:51.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:51.904 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:51.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:51.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:52.006 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:52.027 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:52.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:52.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:52.159 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:52.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:52.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:52.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:52.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:22:52.349 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:22:52.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:22:52.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:52.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:52.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:52.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:52.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:52.584 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:52.640 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:52.660 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:52.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:52.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.705 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:52.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:52 smithi173 ceph-mon[33187]: pgmap v569: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-17T08:22:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:52 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/646701180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:52.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:52 smithi079 ceph-mon[29565]: pgmap v569: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-17T08:22:52.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:52 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/646701180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:52.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:52 smithi079 ceph-mon[40649]: pgmap v569: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-17T08:22:52.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:52 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/646701180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:22:52.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:52.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:52.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:52.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:52.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:52.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:52.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:53.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:53.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:22:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:22:53.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:53.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:53.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:53.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:53.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:53.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:22:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:22:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:22:53.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:53.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:53.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:53.519 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:53.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:53.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:53.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:53.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:53.789 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:53.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:53.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:53.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:53.926 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:53.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:54.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:54.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:54.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:54.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:54.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.193 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:54.194 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:54.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:54.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:54.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:54.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:54.490 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:54.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:54.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:54.613 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:54.613 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:54 smithi173 ceph-mon[33187]: pgmap v570: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 865 op/s 2024-09-17T08:22:54.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:54.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:54 smithi079 ceph-mon[40649]: pgmap v570: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 865 op/s 2024-09-17T08:22:54.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:22:54.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:54 smithi079 ceph-mon[29565]: pgmap v570: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 865 op/s 2024-09-17T08:22:54.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:54.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:54.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:54.767 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:54.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:54.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:54.904 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:54.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:54.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:55.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:55.038 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:55.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:209596 9.b e271/221 9:d81fadb1:::benchmark_data_smithi079_143589_object78397:head v 271'12996, mlcod=271'12993)", 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:55.050769+0000", 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0063859930000000004, 2024-09-17T08:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064165489999999997, 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050769+0000", 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050769+0000", 2024-09-17T08:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050769+0000", 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050776+0000", 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7279999999999995e-06 2024-09-17T08:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050787+0000", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0489000000000001e-05 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050792+0000", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0769999999999998e-06 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050872+0000", 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9633000000000001e-05 2024-09-17T08:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.050901+0000", 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9876999999999999e-05 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:209602 9.b e271/221 9:d6ca34e0:::benchmark_data_smithi079_143589_object78403:head v 271'12997, mlcod=271'12994)", 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:55.054439+0000", 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0027164649999999999, 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028090609999999998, 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054439+0000", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054439+0000", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054438+0000", 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054448+0000", 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7929999999999992e-06 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054463+0000", 2024-09-17T08:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5729000000000001e-05 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054472+0000", 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1470000000000001e-06 2024-09-17T08:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054629+0000", 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015746999999999999 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:55.054682+0000", 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2961000000000003e-05 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 2 2024-09-17T08:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:22:55.140 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:55.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:55.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:55.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:55.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.313 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:55.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:55.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:55.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:55.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:55.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:22:55.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:55.579 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:55.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:55.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:55 smithi173 ceph-mon[33187]: 9.e deep-scrub ok 2024-09-17T08:22:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:55 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:22:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:55 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:22:55.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:55.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:55.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[40649]: 9.e deep-scrub ok 2024-09-17T08:22:55.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:22:55.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:22:55.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[29565]: 9.e deep-scrub ok 2024-09-17T08:22:55.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:22:55.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:55 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:22:55.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:55.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:55.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:55.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:22:55.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:55.985 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:55.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:55.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:56.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:56.108 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:56.151 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:22:56.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:56.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.232 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:56.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:56.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:56.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:56.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:56.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:56.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:56.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:56.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:56.774 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:56 smithi079 ceph-mon[40649]: pgmap v571: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:22:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:56 smithi079 ceph-mon[40649]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T08:22:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:56 smithi079 ceph-mon[29565]: pgmap v571: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:22:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:56 smithi079 ceph-mon[29565]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T08:22:56.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:56.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:56.900 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:56.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:56 smithi173 ceph-mon[33187]: pgmap v571: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-17T08:22:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:56 smithi173 ceph-mon[33187]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T08:22:57.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:22:57.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.041 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:57.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:22:57.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:57.170 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:57.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:57.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:22:57.306 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:57.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:22:57.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:57.443 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:57.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:57.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:57.585 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:57.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:57.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:57.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:57.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:57.800 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:57 smithi173 ceph-mon[33187]: pgmap v573: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 963 op/s 2024-09-17T08:22:57.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:57.851 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:57.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:57 smithi079 ceph-mon[40649]: pgmap v573: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 963 op/s 2024-09-17T08:22:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:57 smithi079 ceph-mon[29565]: pgmap v573: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 963 op/s 2024-09-17T08:22:57.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:57.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:57.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:57.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:58.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:58.102 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:22:58.118 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:58.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:58.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:58.261 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:58.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:58.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:22:58.401 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:22:58.402 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:58.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:58.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:58.526 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:58.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:58.549 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:22:58.628 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:22:58.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:58.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:22:58.703 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:58 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2354580480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.703 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:58 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.703 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:58 smithi173 ceph-mon[33187]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:58.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:58.784 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:58.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:58.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2354580480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[29565]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2354580480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:22:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:58 smithi079 ceph-mon[40649]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:22:58.887 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:58.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:58.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:58.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:59.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:59.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.066 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:59.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:59.098 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:22:59.098 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:22:59.099 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:22:59.099 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:22:59.099 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:22:59.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:22:59.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:59.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:22:59.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:22:59.348 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:59.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:22:59.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:22:59.483 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:59.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:22:59.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:59.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.616 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:59.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:59.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:22:59.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:22:59.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:22:59.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:22:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[40649]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T08:22:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[40649]: pgmap v575: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[29565]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T08:22:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:22:59 smithi079 ceph-mon[29565]: pgmap v575: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:59.892 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:22:59.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:22:59.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:22:59.908 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:59 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:22:59.908 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:59 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:22:59.908 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:59 smithi173 ceph-mon[33187]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T08:22:59.908 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:22:59 smithi173 ceph-mon[33187]: pgmap v575: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:22:59.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:22:59.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:00.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:00.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:00.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:00.167 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:00.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.169 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:00.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:00.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.305 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:00.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:00.333 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:23:00.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:00.442 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:00.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:00.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:00.580 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:00.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:00.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:00.719 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:00.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:00.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:00.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:00.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:00.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:00.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:01.003 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:01.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:01.105 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:01.137 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:01.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:01.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:01.274 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:01.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.275 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:01.328 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:23:01.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:23:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:23:01.365 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:23:01.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:01.396 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:23:01.396 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:23:01.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.410 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:01.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:01.429 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:23:01.429 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:23:01.461 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:23:01.462 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:23:01.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:23:01.490 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:23:01.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:01.520 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:23:01.521 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:23:01.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:01.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:01.552 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:23:01.552 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:23:01.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:01.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:01.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:01.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:01.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:01.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:01.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:01.951 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:01.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:01.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:02.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:02.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.087 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:02.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:02.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:02.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:02.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:02.321 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[40649]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3579031296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[40649]: pgmap v577: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s 2024-09-17T08:23:02.339 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:02.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:02.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[29565]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T08:23:02.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3579031296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:02.340 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:02 smithi079 ceph-mon[29565]: pgmap v577: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s 2024-09-17T08:23:02.363 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:02.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:02.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:02.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:02.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-mon[33187]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T08:23:02.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3579031296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:02.381 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-mon[33187]: pgmap v577: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s 2024-09-17T08:23:02.433 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:23:02.433 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:23:02.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:02.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:02.500 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:02.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:02.632 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:02.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:02.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:23:02.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:02.799 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":573,"stamp":"2024-09-17T08:22:57.482562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3586595118,"num_objects":54915,"num_object_clones":0,"num_object_copies":164745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54915,"num_whiteouts":0,"num_read":1787,"num_read_kb":1633,"num_write":208570,"num_write_kb":8399004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2431,"num_bytes_recovered":145889393,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83582,"ondisk_log_size":83582,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13835532,"kb_used_data":10623364,"kb_used_omap":15,"kb_used_meta":3211888,"kb_avail":736125684,"statfs":{"total":767960285184,"available":753792700416,"internally_reserved":0,"allocated":10878324736,"data_stored":10875964034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":3288974101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-583663616,"num_objects":-8906,"num_object_clones":0,"num_object_copies":-26718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8906,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.254183"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":270,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510342+0000","last_change":"2024-09-17T08:22:32.714143+0000","last_active":"2024-09-17T08:22:46.510342+0000","last_peered":"2024-09-17T08:22:46.510342+0000","last_clean":"2024-09-17T08:22:46.510342+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:22:46.510342+0000","last_undegraded":"2024-09-17T08:22:46.510342+0000","last_fullsized":"2024-09-17T08:22:46.510342+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.714032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.714032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.893234+0000","scrub_duration":0.052748915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":288,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503600+0000","last_change":"2024-09-17T08:22:08.213925+0000","last_active":"2024-09-17T08:22:46.503600+0000","last_peered":"2024-09-17T08:22:46.503600+0000","last_clean":"2024-09-17T08:22:46.503600+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:22:46.503600+0000","last_undegraded":"2024-09-17T08:22:46.503600+0000","last_fullsized":"2024-09-17T08:22:46.503600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:08.213764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:22:08.213764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:08.214149+0000","scrub_duration":0.0019928789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":336,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510747+0000","last_change":"2024-09-17T08:21:10.994605+0000","last_active":"2024-09-17T08:22:46.510747+0000","last_peered":"2024-09-17T08:22:46.510747+0000","last_clean":"2024-09-17T08:22:46.510747+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:22:46.510747+0000","last_undegraded":"2024-09-17T08:22:46.510747+0000","last_fullsized":"2024-09-17T08:22:46.510747+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:21:10.993827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:10.993827+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010019035000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":290,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.497838+0000","last_change":"2024-09-17T08:21:26.022703+0000","last_active":"2024-09-17T08:22:46.497838+0000","last_peered":"2024-09-17T08:22:46.497838+0000","last_clean":"2024-09-17T08:22:46.497838+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:22:46.497838+0000","last_undegraded":"2024-09-17T08:22:46.497838+0000","last_fullsized":"2024-09-17T08:22:46.497838+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:26.022562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.022562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081398640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":277,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510344+0000","last_change":"2024-09-17T08:22:34.633621+0000","last_active":"2024-09-17T08:22:46.510344+0000","last_peered":"2024-09-17T08:22:46.510344+0000","last_clean":"2024-09-17T08:22:46.510344+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:22:46.510344+0000","last_undegraded":"2024-09-17T08:22:46.510344+0000","last_fullsized":"2024-09-17T08:22:46.510344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:34.633510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:22:34.633510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:55.117384+0000","scrub_duration":0.0028104390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":403,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.497852+0000","last_change":"2024-09-17T08:22:12.026146+0000","last_active":"2024-09-17T08:22:46.497852+0000","last_peered":"2024-09-17T08:22:46.497852+0000","last_clean":"2024-09-17T08:22:46.497852+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:22:46.497852+0000","last_undegraded":"2024-09-17T08:22:46.497852+0000","last_fullsized":"2024-09-17T08:22:46.497852+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:12.026000+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.026000+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:37.957272+0000","scrub_duration":0.0077245940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":166,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.263644+0000","last_change":"2024-09-17T08:22:11.468233+0000","last_active":"2024-09-17T08:22:48.263644+0000","last_peered":"2024-09-17T08:22:48.263644+0000","last_clean":"2024-09-17T08:22:48.263644+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:22:48.263644+0000","last_undegraded":"2024-09-17T08:22:48.263644+0000","last_fullsized":"2024-09-17T08:22:48.263644+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:11.468116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.468116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:33.231991+0000","scrub_duration":0.0051835620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":299,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498248+0000","last_change":"2024-09-17T08:22:32.148128+0000","last_active":"2024-09-17T08:22:46.498248+0000","last_peered":"2024-09-17T08:22:46.498248+0000","last_clean":"2024-09-17T08:22:46.498248+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:22:46.498248+0000","last_undegraded":"2024-09-17T08:22:46.498248+0000","last_fullsized":"2024-09-17T08:22:46.498248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.147832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.147832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.754489+0000","scrub_duration":0.0067988659999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505793+0000","last_change":"2024-09-17T08:22:14.492261+0000","last_active":"2024-09-17T08:22:46.505793+0000","last_peered":"2024-09-17T08:22:46.505793+0000","last_clean":"2024-09-17T08:22:46.505793+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:22:46.505793+0000","last_undegraded":"2024-09-17T08:22:46.505793+0000","last_fullsized":"2024-09-17T08:22:46.505793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.492185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.492185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:43.260230+0000","scrub_duration":0.057522131999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":319,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498421+0000","last_change":"2024-09-17T08:21:34.973710+0000","last_active":"2024-09-17T08:22:46.498421+0000","last_peered":"2024-09-17T08:22:46.498421+0000","last_clean":"2024-09-17T08:22:46.498421+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:22:46.498421+0000","last_undegraded":"2024-09-17T08:22:46.498421+0000","last_fullsized":"2024-09-17T08:22:46.498421+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:21:34.973570+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.973570+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:48.307897+0000","scrub_duration":0.059354883999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":312,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504821+0000","last_change":"2024-09-17T08:22:22.423974+0000","last_active":"2024-09-17T08:22:46.504821+0000","last_peered":"2024-09-17T08:22:46.504821+0000","last_clean":"2024-09-17T08:22:46.504821+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:22:46.504821+0000","last_undegraded":"2024-09-17T08:22:46.504821+0000","last_fullsized":"2024-09-17T08:22:46.504821+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.423862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.807818+0000","scrub_duration":0.0032447660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":166,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.263556+0000","last_change":"2024-09-17T08:21:24.523252+0000","last_active":"2024-09-17T08:22:48.263556+0000","last_peered":"2024-09-17T08:22:48.263556+0000","last_clean":"2024-09-17T08:22:48.263556+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:22:48.263556+0000","last_undegraded":"2024-09-17T08:22:48.263556+0000","last_fullsized":"2024-09-17T08:22:48.263556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.523136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.523136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078907060000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":302,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503651+0000","last_change":"2024-09-17T08:21:20.504900+0000","last_active":"2024-09-17T08:22:46.503651+0000","last_peered":"2024-09-17T08:22:46.503651+0000","last_clean":"2024-09-17T08:22:46.503651+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:22:46.503651+0000","last_undegraded":"2024-09-17T08:22:46.503651+0000","last_fullsized":"2024-09-17T08:22:46.503651+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.504720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.504720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018477529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":314,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498417+0000","last_change":"2024-09-17T08:22:36.105717+0000","last_active":"2024-09-17T08:22:46.498417+0000","last_peered":"2024-09-17T08:22:46.498417+0000","last_clean":"2024-09-17T08:22:46.498417+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:22:46.498417+0000","last_undegraded":"2024-09-17T08:22:46.498417+0000","last_fullsized":"2024-09-17T08:22:46.498417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:22:36.105550+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:57.914458+0000","scrub_duration":0.012187885000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":186,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501876+0000","last_change":"2024-09-17T08:22:35.817757+0000","last_active":"2024-09-17T08:22:46.501876+0000","last_peered":"2024-09-17T08:22:46.501876+0000","last_clean":"2024-09-17T08:22:46.501876+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:22:46.501876+0000","last_undegraded":"2024-09-17T08:22:46.501876+0000","last_fullsized":"2024-09-17T08:22:46.501876+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:35.817629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.817629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.187574+0000","scrub_duration":0.0023115000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":299,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501598+0000","last_change":"2024-09-17T08:22:28.020605+0000","last_active":"2024-09-17T08:22:46.501598+0000","last_peered":"2024-09-17T08:22:46.501598+0000","last_clean":"2024-09-17T08:22:46.501598+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:22:46.501598+0000","last_undegraded":"2024-09-17T08:22:46.501598+0000","last_fullsized":"2024-09-17T08:22:46.501598+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:28.019265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:28.019265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.020045+0000","scrub_duration":0.0075046389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"271'12740","reported_seq":14366,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.621561+0000","last_change":"2024-09-17T08:22:35.057102+0000","last_active":"2024-09-17T08:22:53.621561+0000","last_peered":"2024-09-17T08:22:53.621561+0000","last_clean":"2024-09-17T08:22:53.621561+0000","last_became_active":"2024-09-17T08:20:32.362328+0000","last_became_peered":"2024-09-17T08:20:32.362328+0000","last_unstale":"2024-09-17T08:22:53.621561+0000","last_undegraded":"2024-09-17T08:22:53.621561+0000","last_fullsized":"2024-09-17T08:22:53.621561+0000","mapping_epoch":230,"log_start":"256'7800","ondisk_log_start":"256'7800","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11650","last_scrub_stamp":"2024-09-17T08:22:35.056760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.056760+0000","objects_scrubbed":4535,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.046356+0000","scrub_duration":1.5582728380000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222560256,"num_objects":3396,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12740,"num_write_kb":516352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":47382528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":409,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498806+0000","last_change":"2024-09-17T08:22:11.061223+0000","last_active":"2024-09-17T08:22:46.498806+0000","last_peered":"2024-09-17T08:22:46.498806+0000","last_clean":"2024-09-17T08:22:46.498806+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:22:46.498806+0000","last_undegraded":"2024-09-17T08:22:46.498806+0000","last_fullsized":"2024-09-17T08:22:46.498806+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:22:11.061047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.061047+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:14.465518+0000","scrub_duration":0.0082017440000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505320+0000","last_change":"2024-09-17T08:21:29.435457+0000","last_active":"2024-09-17T08:22:46.505320+0000","last_peered":"2024-09-17T08:22:46.505320+0000","last_clean":"2024-09-17T08:22:46.505320+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:22:46.505320+0000","last_undegraded":"2024-09-17T08:22:46.505320+0000","last_fullsized":"2024-09-17T08:22:46.505320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:29.435329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.435329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017420199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502083+0000","last_change":"2024-09-17T08:20:16.153180+0000","last_active":"2024-09-17T08:22:46.502083+0000","last_peered":"2024-09-17T08:22:46.502083+0000","last_clean":"2024-09-17T08:22:46.502083+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:22:46.502083+0000","last_undegraded":"2024-09-17T08:22:46.502083+0000","last_fullsized":"2024-09-17T08:22:46.502083+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:16.153101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:16.153101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017057730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":256,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257905+0000","last_change":"2024-09-17T08:21:15.739147+0000","last_active":"2024-09-17T08:22:48.257905+0000","last_peered":"2024-09-17T08:22:48.257905+0000","last_clean":"2024-09-17T08:22:48.257905+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:22:48.257905+0000","last_undegraded":"2024-09-17T08:22:48.257905+0000","last_fullsized":"2024-09-17T08:22:48.257905+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.739010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.739010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013099520999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"271'12819","reported_seq":12980,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.465665+0000","last_change":"2024-09-17T08:22:37.068629+0000","last_active":"2024-09-17T08:22:53.465665+0000","last_peered":"2024-09-17T08:22:53.465665+0000","last_clean":"2024-09-17T08:22:53.465665+0000","last_became_active":"2024-09-17T08:20:07.223417+0000","last_became_peered":"2024-09-17T08:20:07.223417+0000","last_unstale":"2024-09-17T08:22:53.465665+0000","last_undegraded":"2024-09-17T08:22:53.465665+0000","last_fullsized":"2024-09-17T08:22:53.465665+0000","mapping_epoch":221,"log_start":"256'7900","ondisk_log_start":"256'7900","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11785","last_scrub_stamp":"2024-09-17T08:22:37.068424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.068424+0000","objects_scrubbed":4475,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.205656+0000","scrub_duration":1.4410351079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223412224,"num_objects":3409,"num_object_clones":0,"num_object_copies":10227,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":519296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":315,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.499090+0000","last_change":"2024-09-17T08:22:29.100509+0000","last_active":"2024-09-17T08:22:46.499090+0000","last_peered":"2024-09-17T08:22:46.499090+0000","last_clean":"2024-09-17T08:22:46.499090+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:22:46.499090+0000","last_undegraded":"2024-09-17T08:22:46.499090+0000","last_fullsized":"2024-09-17T08:22:46.499090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:29.100364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.100364+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.213754+0000","scrub_duration":0.023929505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.509374+0000","last_change":"2024-09-17T08:22:24.597248+0000","last_active":"2024-09-17T08:22:46.509374+0000","last_peered":"2024-09-17T08:22:46.509374+0000","last_clean":"2024-09-17T08:22:46.509374+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:22:46.509374+0000","last_undegraded":"2024-09-17T08:22:46.509374+0000","last_fullsized":"2024-09-17T08:22:46.509374+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:24.597137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:22:24.597137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.974690+0000","scrub_duration":0.0075741749999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":391,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:52.982174+0000","last_change":"2024-09-17T08:21:11.769223+0000","last_active":"2024-09-17T08:22:52.982174+0000","last_peered":"2024-09-17T08:22:52.982174+0000","last_clean":"2024-09-17T08:22:52.982174+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:22:52.982174+0000","last_undegraded":"2024-09-17T08:22:52.982174+0000","last_fullsized":"2024-09-17T08:22:52.982174+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:21:11.769092+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:21:11.769092+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066652409999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":293,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.512490+0000","last_change":"2024-09-17T08:21:10.972195+0000","last_active":"2024-09-17T08:22:46.512490+0000","last_peered":"2024-09-17T08:22:46.512490+0000","last_clean":"2024-09-17T08:22:46.512490+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:22:46.512490+0000","last_undegraded":"2024-09-17T08:22:46.512490+0000","last_fullsized":"2024-09-17T08:22:46.512490+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:10.972020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:10.972020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011239071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"271'13389","reported_seq":14963,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.487029+0000","last_change":"2024-09-17T08:20:32.368171+0000","last_active":"2024-09-17T08:22:53.487029+0000","last_peered":"2024-09-17T08:22:53.487029+0000","last_clean":"2024-09-17T08:22:53.487029+0000","last_became_active":"2024-09-17T08:20:32.365928+0000","last_became_peered":"2024-09-17T08:20:32.365928+0000","last_unstale":"2024-09-17T08:22:53.487029+0000","last_undegraded":"2024-09-17T08:22:53.487029+0000","last_fullsized":"2024-09-17T08:22:53.487029+0000","mapping_epoch":230,"log_start":"258'8500","ondisk_log_start":"258'8500","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":0,"log_size":4889,"log_dups_size":0,"ondisk_log_size":4889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227606528,"num_objects":3473,"num_object_clones":0,"num_object_copies":10419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":539584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":49152000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":158,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257575+0000","last_change":"2024-09-17T08:22:27.019007+0000","last_active":"2024-09-17T08:22:48.257575+0000","last_peered":"2024-09-17T08:22:48.257575+0000","last_clean":"2024-09-17T08:22:48.257575+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:22:48.257575+0000","last_undegraded":"2024-09-17T08:22:48.257575+0000","last_fullsized":"2024-09-17T08:22:48.257575+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:27.018869+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.018869+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.142241+0000","scrub_duration":0.0034750470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501816+0000","last_change":"2024-09-17T08:22:05.559395+0000","last_active":"2024-09-17T08:22:46.501816+0000","last_peered":"2024-09-17T08:22:46.501816+0000","last_clean":"2024-09-17T08:22:46.501816+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:22:46.501816+0000","last_undegraded":"2024-09-17T08:22:46.501816+0000","last_fullsized":"2024-09-17T08:22:46.501816+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:05.559268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:05.559268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.693611+0000","scrub_duration":0.010154139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":436,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:47.986770+0000","last_change":"2024-09-17T08:21:35.884656+0000","last_active":"2024-09-17T08:22:47.986770+0000","last_peered":"2024-09-17T08:22:47.986770+0000","last_clean":"2024-09-17T08:22:47.986770+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:22:47.986770+0000","last_undegraded":"2024-09-17T08:22:47.986770+0000","last_fullsized":"2024-09-17T08:22:47.986770+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:21:35.884516+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086332900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504153+0000","last_change":"2024-09-17T08:22:20.469387+0000","last_active":"2024-09-17T08:22:46.504153+0000","last_peered":"2024-09-17T08:22:46.504153+0000","last_clean":"2024-09-17T08:22:46.504153+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:22:46.504153+0000","last_undegraded":"2024-09-17T08:22:46.504153+0000","last_fullsized":"2024-09-17T08:22:46.504153+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.469252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.469252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.979091+0000","scrub_duration":0.0085710660000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"271'13262","reported_seq":13418,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:52.614944+0000","last_change":"2024-09-17T08:22:42.795845+0000","last_active":"2024-09-17T08:22:52.614944+0000","last_peered":"2024-09-17T08:22:52.614944+0000","last_clean":"2024-09-17T08:22:52.614944+0000","last_became_active":"2024-09-17T08:20:07.153440+0000","last_became_peered":"2024-09-17T08:20:07.153440+0000","last_unstale":"2024-09-17T08:22:52.614944+0000","last_undegraded":"2024-09-17T08:22:52.614944+0000","last_fullsized":"2024-09-17T08:22:52.614944+0000","mapping_epoch":221,"log_start":"259'8600","ondisk_log_start":"259'8600","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12590","last_scrub_stamp":"2024-09-17T08:22:42.795634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:42.795634+0000","objects_scrubbed":4222,"log_size":4662,"log_dups_size":0,"ondisk_log_size":4662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.287069+0000","scrub_duration":1.2865534140000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230293504,"num_objects":3514,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13262,"num_write_kb":536832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":353,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.513103+0000","last_change":"2024-09-17T08:22:14.033870+0000","last_active":"2024-09-17T08:22:46.513103+0000","last_peered":"2024-09-17T08:22:46.513103+0000","last_clean":"2024-09-17T08:22:46.513103+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:22:46.513103+0000","last_undegraded":"2024-09-17T08:22:46.513103+0000","last_fullsized":"2024-09-17T08:22:46.513103+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:22:14.033529+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.033529+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.942917+0000","scrub_duration":0.0038854010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":294,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504408+0000","last_change":"2024-09-17T08:21:23.559857+0000","last_active":"2024-09-17T08:22:46.504408+0000","last_peered":"2024-09-17T08:22:46.504408+0000","last_clean":"2024-09-17T08:22:46.504408+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:22:46.504408+0000","last_undegraded":"2024-09-17T08:22:46.504408+0000","last_fullsized":"2024-09-17T08:22:46.504408+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:23.559763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:23.559763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052019450000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":176,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.499178+0000","last_change":"2024-09-17T08:22:30.071610+0000","last_active":"2024-09-17T08:22:46.499178+0000","last_peered":"2024-09-17T08:22:46.499178+0000","last_clean":"2024-09-17T08:22:46.499178+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:22:46.499178+0000","last_undegraded":"2024-09-17T08:22:46.499178+0000","last_fullsized":"2024-09-17T08:22:46.499178+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.071480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.161204+0000","scrub_duration":0.0047153050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":259,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510388+0000","last_change":"2024-09-17T08:20:13.845069+0000","last_active":"2024-09-17T08:22:46.510388+0000","last_peered":"2024-09-17T08:22:46.510388+0000","last_clean":"2024-09-17T08:22:46.510388+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:22:46.510388+0000","last_undegraded":"2024-09-17T08:22:46.510388+0000","last_fullsized":"2024-09-17T08:22:46.510388+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:13.844986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:13.844986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001309288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"271'13057","reported_seq":13228,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.461454+0000","last_change":"2024-09-17T08:22:29.304499+0000","last_active":"2024-09-17T08:22:53.461454+0000","last_peered":"2024-09-17T08:22:53.461454+0000","last_clean":"2024-09-17T08:22:53.461454+0000","last_became_active":"2024-09-17T08:20:07.153490+0000","last_became_peered":"2024-09-17T08:20:07.153490+0000","last_unstale":"2024-09-17T08:22:53.461454+0000","last_undegraded":"2024-09-17T08:22:53.461454+0000","last_fullsized":"2024-09-17T08:22:53.461454+0000","mapping_epoch":221,"log_start":"258'8200","ondisk_log_start":"258'8200","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11539","last_scrub_stamp":"2024-09-17T08:22:29.304266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.304266+0000","objects_scrubbed":4996,"log_size":4857,"log_dups_size":0,"ondisk_log_size":4857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:40.689344+0000","scrub_duration":1.700727825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223805440,"num_objects":3415,"num_object_clones":0,"num_object_copies":10245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13057,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":317,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498068+0000","last_change":"2024-09-17T08:22:20.997014+0000","last_active":"2024-09-17T08:22:46.498068+0000","last_peered":"2024-09-17T08:22:46.498068+0000","last_clean":"2024-09-17T08:22:46.498068+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:22:46.498068+0000","last_undegraded":"2024-09-17T08:22:46.498068+0000","last_fullsized":"2024-09-17T08:22:46.498068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:22:20.996746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.996746+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.038376+0000","scrub_duration":0.0061057289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.268380+0000","last_change":"2024-09-17T08:22:20.368530+0000","last_active":"2024-09-17T08:22:48.268380+0000","last_peered":"2024-09-17T08:22:48.268380+0000","last_clean":"2024-09-17T08:22:48.268380+0000","last_became_active":"2024-09-17T08:20:23.487784+0000","last_became_peered":"2024-09-17T08:20:23.487784+0000","last_unstale":"2024-09-17T08:22:48.268380+0000","last_undegraded":"2024-09-17T08:22:48.268380+0000","last_fullsized":"2024-09-17T08:22:48.268380+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.368411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.368411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:27.290703+0000","scrub_duration":0.0016290580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":469,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.261147+0000","last_change":"2024-09-17T08:22:07.339466+0000","last_active":"2024-09-17T08:22:48.261147+0000","last_peered":"2024-09-17T08:22:48.261147+0000","last_clean":"2024-09-17T08:22:48.261147+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:22:48.261147+0000","last_undegraded":"2024-09-17T08:22:48.261147+0000","last_fullsized":"2024-09-17T08:22:48.261147+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:22:07.339335+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:22:07.339335+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.917029+0000","scrub_duration":0.011996602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503921+0000","last_change":"2024-09-17T08:21:12.324228+0000","last_active":"2024-09-17T08:22:46.503921+0000","last_peered":"2024-09-17T08:22:46.503921+0000","last_clean":"2024-09-17T08:22:46.503921+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:22:46.503921+0000","last_undegraded":"2024-09-17T08:22:46.503921+0000","last_fullsized":"2024-09-17T08:22:46.503921+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:12.323711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:21:12.323711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019279740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"271'12909","reported_seq":13065,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:52.638432+0000","last_change":"2024-09-17T08:22:39.712172+0000","last_active":"2024-09-17T08:22:52.638432+0000","last_peered":"2024-09-17T08:22:52.638432+0000","last_clean":"2024-09-17T08:22:52.638432+0000","last_became_active":"2024-09-17T08:20:07.346251+0000","last_became_peered":"2024-09-17T08:20:07.346251+0000","last_unstale":"2024-09-17T08:22:52.638432+0000","last_undegraded":"2024-09-17T08:22:52.638432+0000","last_fullsized":"2024-09-17T08:22:52.638432+0000","mapping_epoch":221,"log_start":"258'8200","ondisk_log_start":"258'8200","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12088","last_scrub_stamp":"2024-09-17T08:22:39.711942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.711942+0000","objects_scrubbed":4288,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.633790+0000","scrub_duration":1.2802566520000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224657432,"num_objects":3429,"num_object_clones":0,"num_object_copies":10287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12909,"num_write_kb":522753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":325,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510034+0000","last_change":"2024-09-17T08:21:26.037150+0000","last_active":"2024-09-17T08:22:46.510034+0000","last_peered":"2024-09-17T08:22:46.510034+0000","last_clean":"2024-09-17T08:22:46.510034+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:22:46.510034+0000","last_undegraded":"2024-09-17T08:22:46.510034+0000","last_fullsized":"2024-09-17T08:22:46.510034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:21:26.036963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.036963+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038814650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.518641+0000","last_change":"2024-09-17T08:22:15.514138+0000","last_active":"2024-09-17T08:22:46.518641+0000","last_peered":"2024-09-17T08:22:46.518641+0000","last_clean":"2024-09-17T08:22:46.518641+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:22:46.518641+0000","last_undegraded":"2024-09-17T08:22:46.518641+0000","last_fullsized":"2024-09-17T08:22:46.518641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:15.514031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.514031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.523751+0000","scrub_duration":0.0041206280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":273,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.497906+0000","last_change":"2024-09-17T08:21:32.922504+0000","last_active":"2024-09-17T08:22:46.497906+0000","last_peered":"2024-09-17T08:22:46.497906+0000","last_clean":"2024-09-17T08:22:46.497906+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:22:46.497906+0000","last_undegraded":"2024-09-17T08:22:46.497906+0000","last_fullsized":"2024-09-17T08:22:46.497906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:32.922405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:21:32.922405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015573643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":286,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498342+0000","last_change":"2024-09-17T08:20:31.986066+0000","last_active":"2024-09-17T08:22:46.498342+0000","last_peered":"2024-09-17T08:22:46.498342+0000","last_clean":"2024-09-17T08:22:46.498342+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:22:46.498342+0000","last_undegraded":"2024-09-17T08:22:46.498342+0000","last_fullsized":"2024-09-17T08:22:46.498342+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:31.985972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:31.985972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018779650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"271'16483","reported_seq":16847,"reported_epoch":271,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-17T08:22:52.689149+0000","last_change":"2024-09-17T08:22:32.399548+0000","last_active":"2024-09-17T08:22:52.689149+0000","last_peered":"2024-09-17T08:22:52.689149+0000","last_clean":"2024-09-17T08:22:52.689149+0000","last_became_active":"2024-09-17T08:20:07.153814+0000","last_became_peered":"2024-09-17T08:20:07.153814+0000","last_unstale":"2024-09-17T08:22:52.689149+0000","last_undegraded":"2024-09-17T08:22:52.689149+0000","last_fullsized":"2024-09-17T08:22:52.689149+0000","mapping_epoch":221,"log_start":"267'11803","ondisk_log_start":"267'11803","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":3640,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 20s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225116160,"num_objects":3435,"num_object_clones":0,"num_object_copies":10305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12843,"num_write_kb":520896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":337,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501799+0000","last_change":"2024-09-17T08:21:26.896828+0000","last_active":"2024-09-17T08:22:46.501799+0000","last_peered":"2024-09-17T08:22:46.501799+0000","last_clean":"2024-09-17T08:22:46.501799+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:22:46.501799+0000","last_undegraded":"2024-09-17T08:22:46.501799+0000","last_fullsized":"2024-09-17T08:22:46.501799+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:21:26.896576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.896576+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056806000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":293,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505860+0000","last_change":"2024-09-17T08:21:27.406621+0000","last_active":"2024-09-17T08:22:46.505860+0000","last_peered":"2024-09-17T08:22:46.505860+0000","last_clean":"2024-09-17T08:22:46.505860+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:22:46.505860+0000","last_undegraded":"2024-09-17T08:22:46.505860+0000","last_fullsized":"2024-09-17T08:22:46.505860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:21:27.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.406373+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:54.576672+0000","scrub_duration":0.0043336249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":239,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501302+0000","last_change":"2024-09-17T08:22:27.034592+0000","last_active":"2024-09-17T08:22:46.501302+0000","last_peered":"2024-09-17T08:22:46.501302+0000","last_clean":"2024-09-17T08:22:46.501302+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:22:46.501302+0000","last_undegraded":"2024-09-17T08:22:46.501302+0000","last_fullsized":"2024-09-17T08:22:46.501302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.034476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.034476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.281308+0000","scrub_duration":0.0053609369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":181,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501908+0000","last_change":"2024-09-17T08:21:13.406948+0000","last_active":"2024-09-17T08:22:46.501908+0000","last_peered":"2024-09-17T08:22:46.501908+0000","last_clean":"2024-09-17T08:22:46.501908+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:22:46.501908+0000","last_undegraded":"2024-09-17T08:22:46.501908+0000","last_fullsized":"2024-09-17T08:22:46.501908+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:13.406818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:13.406818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039848110000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"271'13031","reported_seq":13188,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.626114+0000","last_change":"2024-09-17T08:22:40.874195+0000","last_active":"2024-09-17T08:22:53.626114+0000","last_peered":"2024-09-17T08:22:53.626114+0000","last_clean":"2024-09-17T08:22:53.626114+0000","last_became_active":"2024-09-17T08:20:07.346503+0000","last_became_peered":"2024-09-17T08:20:07.346503+0000","last_unstale":"2024-09-17T08:22:53.626114+0000","last_undegraded":"2024-09-17T08:22:53.626114+0000","last_fullsized":"2024-09-17T08:22:53.626114+0000","mapping_epoch":221,"log_start":"256'8100","ondisk_log_start":"256'8100","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12220","last_scrub_stamp":"2024-09-17T08:22:40.873622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.873622+0000","objects_scrubbed":4349,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.542314+0000","scrub_duration":1.240716989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229048320,"num_objects":3495,"num_object_clones":0,"num_object_copies":10485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13031,"num_write_kb":528832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":198,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.508764+0000","last_change":"2024-09-17T08:21:20.455004+0000","last_active":"2024-09-17T08:22:46.508764+0000","last_peered":"2024-09-17T08:22:46.508764+0000","last_clean":"2024-09-17T08:22:46.508764+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:22:46.508764+0000","last_undegraded":"2024-09-17T08:22:46.508764+0000","last_fullsized":"2024-09-17T08:22:46.508764+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:21:20.454861+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.454861+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:49.187097+0000","scrub_duration":0.0043903680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502411+0000","last_change":"2024-09-17T08:22:17.515205+0000","last_active":"2024-09-17T08:22:46.502411+0000","last_peered":"2024-09-17T08:22:46.502411+0000","last_clean":"2024-09-17T08:22:46.502411+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:22:46.502411+0000","last_undegraded":"2024-09-17T08:22:46.502411+0000","last_fullsized":"2024-09-17T08:22:46.502411+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:17.515085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.515085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.852597+0000","scrub_duration":0.001995199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":449,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.270217+0000","last_change":"2024-09-17T08:21:29.854449+0000","last_active":"2024-09-17T08:22:48.270217+0000","last_peered":"2024-09-17T08:22:48.270217+0000","last_clean":"2024-09-17T08:22:48.270217+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:22:48.270217+0000","last_undegraded":"2024-09-17T08:22:48.270217+0000","last_fullsized":"2024-09-17T08:22:48.270217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:21:29.853904+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:19:34.379477+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.853904+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059980889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501469+0000","last_change":"2024-09-17T08:21:21.799057+0000","last_active":"2024-09-17T08:22:46.501469+0000","last_peered":"2024-09-17T08:22:46.501469+0000","last_clean":"2024-09-17T08:22:46.501469+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:22:46.501469+0000","last_undegraded":"2024-09-17T08:22:46.501469+0000","last_fullsized":"2024-09-17T08:22:46.501469+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.798694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.798694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002756157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"271'13083","reported_seq":13244,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.424823+0000","last_change":"2024-09-17T08:22:37.976919+0000","last_active":"2024-09-17T08:22:53.424823+0000","last_peered":"2024-09-17T08:22:53.424823+0000","last_clean":"2024-09-17T08:22:53.424823+0000","last_became_active":"2024-09-17T08:20:07.345130+0000","last_became_peered":"2024-09-17T08:20:07.345130+0000","last_unstale":"2024-09-17T08:22:53.424823+0000","last_undegraded":"2024-09-17T08:22:53.424823+0000","last_fullsized":"2024-09-17T08:22:53.424823+0000","mapping_epoch":221,"log_start":"252'7200","ondisk_log_start":"252'7200","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12113","last_scrub_stamp":"2024-09-17T08:22:37.976676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.976676+0000","objects_scrubbed":4399,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.418342+0000","scrub_duration":1.1806302419999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222101504,"num_objects":3389,"num_object_clones":0,"num_object_copies":10167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":316,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.264636+0000","last_change":"2024-09-17T08:21:14.458315+0000","last_active":"2024-09-17T08:22:48.264636+0000","last_peered":"2024-09-17T08:22:48.264636+0000","last_clean":"2024-09-17T08:22:48.264636+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:22:48.264636+0000","last_undegraded":"2024-09-17T08:22:48.264636+0000","last_fullsized":"2024-09-17T08:22:48.264636+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:21:14.458167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.458167+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024233291000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.508852+0000","last_change":"2024-09-17T08:20:22.073899+0000","last_active":"2024-09-17T08:22:46.508852+0000","last_peered":"2024-09-17T08:22:46.508852+0000","last_clean":"2024-09-17T08:22:46.508852+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:22:46.508852+0000","last_undegraded":"2024-09-17T08:22:46.508852+0000","last_fullsized":"2024-09-17T08:22:46.508852+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:22.073341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:20:22.073341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048532590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":302,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502430+0000","last_change":"2024-09-17T08:22:18.168279+0000","last_active":"2024-09-17T08:22:46.502430+0000","last_peered":"2024-09-17T08:22:46.502430+0000","last_clean":"2024-09-17T08:22:46.502430+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:22:46.502430+0000","last_undegraded":"2024-09-17T08:22:46.502430+0000","last_fullsized":"2024-09-17T08:22:46.502430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.168157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.168157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.314142+0000","scrub_duration":0.0039078109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":270,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510116+0000","last_change":"2024-09-17T08:21:13.022290+0000","last_active":"2024-09-17T08:22:46.510116+0000","last_peered":"2024-09-17T08:22:46.510116+0000","last_clean":"2024-09-17T08:22:46.510116+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:22:46.510116+0000","last_undegraded":"2024-09-17T08:22:46.510116+0000","last_fullsized":"2024-09-17T08:22:46.510116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:13.022174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:13.022174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005631545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"271'12887","reported_seq":13058,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.615797+0000","last_change":"2024-09-17T08:22:31.345117+0000","last_active":"2024-09-17T08:22:53.615797+0000","last_peered":"2024-09-17T08:22:53.615797+0000","last_clean":"2024-09-17T08:22:53.615797+0000","last_became_active":"2024-09-17T08:20:07.347116+0000","last_became_peered":"2024-09-17T08:20:07.347116+0000","last_unstale":"2024-09-17T08:22:53.615797+0000","last_undegraded":"2024-09-17T08:22:53.615797+0000","last_fullsized":"2024-09-17T08:22:53.615797+0000","mapping_epoch":221,"log_start":"256'7900","ondisk_log_start":"256'7900","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11550","last_scrub_stamp":"2024-09-17T08:22:31.344820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.344820+0000","objects_scrubbed":4856,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:37.031561+0000","scrub_duration":1.7801603639999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227606528,"num_objects":3473,"num_object_clones":0,"num_object_copies":10419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":184,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502183+0000","last_change":"2024-09-17T08:22:08.570430+0000","last_active":"2024-09-17T08:22:46.502183+0000","last_peered":"2024-09-17T08:22:46.502183+0000","last_clean":"2024-09-17T08:22:46.502183+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:22:46.502183+0000","last_undegraded":"2024-09-17T08:22:46.502183+0000","last_fullsized":"2024-09-17T08:22:46.502183+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:22:08.569507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:08.569507+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:24.727398+0000","scrub_duration":0.010760135000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":627,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:51.971174+0000","last_change":"2024-09-17T08:21:21.456425+0000","last_active":"2024-09-17T08:22:51.971174+0000","last_peered":"2024-09-17T08:22:51.971174+0000","last_clean":"2024-09-17T08:22:51.971174+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:22:51.971174+0000","last_undegraded":"2024-09-17T08:22:51.971174+0000","last_fullsized":"2024-09-17T08:22:51.971174+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:21:21.455503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.455503+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015087046,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":441,"num_read_kb":436,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.229486+0000","last_change":"2024-09-17T08:21:21.794874+0000","last_active":"2024-09-17T08:22:48.229486+0000","last_peered":"2024-09-17T08:22:48.229486+0000","last_clean":"2024-09-17T08:22:48.229486+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:22:48.229486+0000","last_undegraded":"2024-09-17T08:22:48.229486+0000","last_fullsized":"2024-09-17T08:22:48.229486+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.794753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.794753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020168569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":270,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257057+0000","last_change":"2024-09-17T08:22:09.870585+0000","last_active":"2024-09-17T08:22:48.257057+0000","last_peered":"2024-09-17T08:22:48.257057+0000","last_clean":"2024-09-17T08:22:48.257057+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:22:48.257057+0000","last_undegraded":"2024-09-17T08:22:48.257057+0000","last_fullsized":"2024-09-17T08:22:48.257057+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:09.870413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.870413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.195536+0000","scrub_duration":0.0021174589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":293,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505452+0000","last_change":"2024-09-17T08:22:40.481597+0000","last_active":"2024-09-17T08:22:46.505452+0000","last_peered":"2024-09-17T08:22:46.505452+0000","last_clean":"2024-09-17T08:22:46.505452+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:22:46.505452+0000","last_undegraded":"2024-09-17T08:22:46.505452+0000","last_fullsized":"2024-09-17T08:22:46.505452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.481384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.481384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.087943+0000","scrub_duration":0.0030329860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"271'12870","reported_seq":13021,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.412616+0000","last_change":"2024-09-17T08:22:43.087899+0000","last_active":"2024-09-17T08:22:53.412616+0000","last_peered":"2024-09-17T08:22:53.412616+0000","last_clean":"2024-09-17T08:22:53.412616+0000","last_became_active":"2024-09-17T08:20:07.346820+0000","last_became_peered":"2024-09-17T08:20:07.346820+0000","last_unstale":"2024-09-17T08:22:53.412616+0000","last_undegraded":"2024-09-17T08:22:53.412616+0000","last_fullsized":"2024-09-17T08:22:53.412616+0000","mapping_epoch":221,"log_start":"251'6900","ondisk_log_start":"251'6900","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12231","last_scrub_stamp":"2024-09-17T08:22:43.087628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:43.087628+0000","objects_scrubbed":4033,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:44.864552+0000","scrub_duration":1.249210038,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219676672,"num_objects":3352,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12870,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":392,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503065+0000","last_change":"2024-09-17T08:20:23.574560+0000","last_active":"2024-09-17T08:22:46.503065+0000","last_peered":"2024-09-17T08:22:46.503065+0000","last_clean":"2024-09-17T08:22:46.503065+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:22:46.503065+0000","last_undegraded":"2024-09-17T08:22:46.503065+0000","last_fullsized":"2024-09-17T08:22:46.503065+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:20:23.574471+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.574471+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":295,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503767+0000","last_change":"2024-09-17T08:20:14.498127+0000","last_active":"2024-09-17T08:22:46.503767+0000","last_peered":"2024-09-17T08:22:46.503767+0000","last_clean":"2024-09-17T08:22:46.503767+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:22:46.503767+0000","last_undegraded":"2024-09-17T08:22:46.503767+0000","last_fullsized":"2024-09-17T08:22:46.503767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:20:14.498043+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:20:14.498043+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017037160000000001,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":306,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504477+0000","last_change":"2024-09-17T08:21:33.434991+0000","last_active":"2024-09-17T08:22:46.504477+0000","last_peered":"2024-09-17T08:22:46.504477+0000","last_clean":"2024-09-17T08:22:46.504477+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:22:46.504477+0000","last_undegraded":"2024-09-17T08:22:46.504477+0000","last_fullsized":"2024-09-17T08:22:46.504477+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.434904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.434904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037511929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502260+0000","last_change":"2024-09-17T08:20:23.088911+0000","last_active":"2024-09-17T08:22:46.502260+0000","last_peered":"2024-09-17T08:22:46.502260+0000","last_clean":"2024-09-17T08:22:46.502260+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:22:46.502260+0000","last_undegraded":"2024-09-17T08:22:46.502260+0000","last_fullsized":"2024-09-17T08:22:46.502260+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:23.088809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.088809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015125119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":166,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502164+0000","last_change":"2024-09-17T08:21:34.401415+0000","last_active":"2024-09-17T08:22:46.502164+0000","last_peered":"2024-09-17T08:22:46.502164+0000","last_clean":"2024-09-17T08:22:46.502164+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:22:46.502164+0000","last_undegraded":"2024-09-17T08:22:46.502164+0000","last_fullsized":"2024-09-17T08:22:46.502164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:34.401099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.401099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010133837999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"271'12941","reported_seq":13110,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.483746+0000","last_change":"2024-09-17T08:22:32.253807+0000","last_active":"2024-09-17T08:22:53.483746+0000","last_peered":"2024-09-17T08:22:53.483746+0000","last_clean":"2024-09-17T08:22:53.483746+0000","last_became_active":"2024-09-17T08:20:07.345502+0000","last_became_peered":"2024-09-17T08:20:07.345502+0000","last_unstale":"2024-09-17T08:22:53.483746+0000","last_undegraded":"2024-09-17T08:22:53.483746+0000","last_fullsized":"2024-09-17T08:22:53.483746+0000","mapping_epoch":221,"log_start":"257'8100","ondisk_log_start":"257'8100","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11656","last_scrub_stamp":"2024-09-17T08:22:32.253689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.253689+0000","objects_scrubbed":4756,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.659626+0000","scrub_duration":1.6774553399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222756864,"num_objects":3399,"num_object_clones":0,"num_object_copies":10197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12941,"num_write_kb":522880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":276,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498729+0000","last_change":"2024-09-17T08:20:20.963771+0000","last_active":"2024-09-17T08:22:46.498729+0000","last_peered":"2024-09-17T08:22:46.498729+0000","last_clean":"2024-09-17T08:22:46.498729+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:22:46.498729+0000","last_undegraded":"2024-09-17T08:22:46.498729+0000","last_fullsized":"2024-09-17T08:22:46.498729+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:20:20.963676+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:20:20.963676+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042143700000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":284,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510142+0000","last_change":"2024-09-17T08:21:34.970246+0000","last_active":"2024-09-17T08:22:46.510142+0000","last_peered":"2024-09-17T08:22:46.510142+0000","last_clean":"2024-09-17T08:22:46.510142+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:22:46.510142+0000","last_undegraded":"2024-09-17T08:22:46.510142+0000","last_fullsized":"2024-09-17T08:22:46.510142+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:21:34.969971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.969971+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:56.438614+0000","scrub_duration":0.0041400559999999996,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":160,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257193+0000","last_change":"2024-09-17T08:22:14.962129+0000","last_active":"2024-09-17T08:22:48.257193+0000","last_peered":"2024-09-17T08:22:48.257193+0000","last_clean":"2024-09-17T08:22:48.257193+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:22:48.257193+0000","last_undegraded":"2024-09-17T08:22:48.257193+0000","last_fullsized":"2024-09-17T08:22:48.257193+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.962019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.962019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.750253+0000","scrub_duration":0.0046754020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":549,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504615+0000","last_change":"2024-09-17T08:22:13.427742+0000","last_active":"2024-09-17T08:22:46.504615+0000","last_peered":"2024-09-17T08:22:46.504615+0000","last_clean":"2024-09-17T08:22:46.504615+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:22:46.504615+0000","last_undegraded":"2024-09-17T08:22:46.504615+0000","last_fullsized":"2024-09-17T08:22:46.504615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:22:13.427622+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_clean_scrub_stamp":"2024-09-17T08:22:13.427622+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:14.950844+0000","scrub_duration":0.0033528830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":288,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503431+0000","last_change":"2024-09-17T08:21:15.369223+0000","last_active":"2024-09-17T08:22:46.503431+0000","last_peered":"2024-09-17T08:22:46.503431+0000","last_clean":"2024-09-17T08:22:46.503431+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:22:46.503431+0000","last_undegraded":"2024-09-17T08:22:46.503431+0000","last_fullsized":"2024-09-17T08:22:46.503431+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.368937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.368937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021146210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":175,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510595+0000","last_change":"2024-09-17T08:21:30.014753+0000","last_active":"2024-09-17T08:22:46.510595+0000","last_peered":"2024-09-17T08:22:46.510595+0000","last_clean":"2024-09-17T08:22:46.510595+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:22:46.510595+0000","last_undegraded":"2024-09-17T08:22:46.510595+0000","last_fullsized":"2024-09-17T08:22:46.510595+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:30.013651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.013651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012423214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.7","version":"271'13039","reported_seq":13232,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:52.667548+0000","last_change":"2024-09-17T08:22:17.960994+0000","last_active":"2024-09-17T08:22:52.667548+0000","last_peered":"2024-09-17T08:22:52.667548+0000","last_clean":"2024-09-17T08:22:52.667548+0000","last_became_active":"2024-09-17T08:20:07.345480+0000","last_became_peered":"2024-09-17T08:20:07.345480+0000","last_unstale":"2024-09-17T08:22:52.667548+0000","last_undegraded":"2024-09-17T08:22:52.667548+0000","last_fullsized":"2024-09-17T08:22:52.667548+0000","mapping_epoch":221,"log_start":"259'8400","ondisk_log_start":"259'8400","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'10727","last_scrub_stamp":"2024-09-17T08:22:17.960812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.960812+0000","objects_scrubbed":5910,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.367005+0000","scrub_duration":1.5674201079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232062976,"num_objects":3541,"num_object_clones":0,"num_object_copies":10623,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":530560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":209,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.267854+0000","last_change":"2024-09-17T08:22:25.349111+0000","last_active":"2024-09-17T08:22:48.267854+0000","last_peered":"2024-09-17T08:22:48.267854+0000","last_clean":"2024-09-17T08:22:48.267854+0000","last_became_active":"2024-09-17T08:20:02.496306+0000","last_became_peered":"2024-09-17T08:20:02.496306+0000","last_unstale":"2024-09-17T08:22:48.267854+0000","last_undegraded":"2024-09-17T08:22:48.267854+0000","last_fullsized":"2024-09-17T08:22:48.267854+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:22:25.348962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.348962+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.315979+0000","scrub_duration":0.010428712999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":251,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257613+0000","last_change":"2024-09-17T08:21:14.706611+0000","last_active":"2024-09-17T08:22:48.257613+0000","last_peered":"2024-09-17T08:22:48.257613+0000","last_clean":"2024-09-17T08:22:48.257613+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:22:48.257613+0000","last_undegraded":"2024-09-17T08:22:48.257613+0000","last_fullsized":"2024-09-17T08:22:48.257613+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:14.706481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.706481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045730429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":257,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257116+0000","last_change":"2024-09-17T08:21:09.680879+0000","last_active":"2024-09-17T08:22:48.257116+0000","last_peered":"2024-09-17T08:22:48.257116+0000","last_clean":"2024-09-17T08:22:48.257116+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:22:48.257116+0000","last_undegraded":"2024-09-17T08:22:48.257116+0000","last_fullsized":"2024-09-17T08:22:48.257116+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:09.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:21:09.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001955245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":285,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504546+0000","last_change":"2024-09-17T08:21:18.452335+0000","last_active":"2024-09-17T08:22:46.504546+0000","last_peered":"2024-09-17T08:22:46.504546+0000","last_clean":"2024-09-17T08:22:46.504546+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:22:46.504546+0000","last_undegraded":"2024-09-17T08:22:46.504546+0000","last_fullsized":"2024-09-17T08:22:46.504546+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.452221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.452221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042305550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"272'12936","reported_seq":14504,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-17T08:22:56.382884+0000","last_change":"2024-09-17T08:22:43.617938+0000","last_active":"2024-09-17T08:22:56.382884+0000","last_peered":"2024-09-17T08:22:56.382884+0000","last_clean":"2024-09-17T08:22:56.382884+0000","last_became_active":"2024-09-17T08:20:33.313140+0000","last_became_peered":"2024-09-17T08:20:33.313140+0000","last_unstale":"2024-09-17T08:22:56.382884+0000","last_undegraded":"2024-09-17T08:22:56.382884+0000","last_fullsized":"2024-09-17T08:22:56.382884+0000","mapping_epoch":231,"log_start":"249'6100","ondisk_log_start":"249'6100","created":221,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":2949,"log_size":6836,"log_dups_size":0,"ondisk_log_size":6836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209059840,"num_objects":3190,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12936,"num_write_kb":516032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":726,"num_bytes_recovered":47579136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":348,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501707+0000","last_change":"2024-09-17T08:22:29.010241+0000","last_active":"2024-09-17T08:22:46.501707+0000","last_peered":"2024-09-17T08:22:46.501707+0000","last_clean":"2024-09-17T08:22:46.501707+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:22:46.501707+0000","last_undegraded":"2024-09-17T08:22:46.501707+0000","last_fullsized":"2024-09-17T08:22:46.501707+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:22:29.010088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.010088+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.705148+0000","scrub_duration":0.0044362949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":177,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502401+0000","last_change":"2024-09-17T08:22:40.621682+0000","last_active":"2024-09-17T08:22:46.502401+0000","last_peered":"2024-09-17T08:22:46.502401+0000","last_clean":"2024-09-17T08:22:46.502401+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:22:46.502401+0000","last_undegraded":"2024-09-17T08:22:46.502401+0000","last_fullsized":"2024-09-17T08:22:46.502401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.621579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.621579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.471410+0000","scrub_duration":0.0048029379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":271,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.256656+0000","last_change":"2024-09-17T08:22:30.062596+0000","last_active":"2024-09-17T08:22:48.256656+0000","last_peered":"2024-09-17T08:22:48.256656+0000","last_clean":"2024-09-17T08:22:48.256656+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:22:48.256656+0000","last_undegraded":"2024-09-17T08:22:48.256656+0000","last_fullsized":"2024-09-17T08:22:48.256656+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.062098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.062098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.782037+0000","scrub_duration":0.0090326120000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":285,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503487+0000","last_change":"2024-09-17T08:20:24.572671+0000","last_active":"2024-09-17T08:22:46.503487+0000","last_peered":"2024-09-17T08:22:46.503487+0000","last_clean":"2024-09-17T08:22:46.503487+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:22:46.503487+0000","last_undegraded":"2024-09-17T08:22:46.503487+0000","last_fullsized":"2024-09-17T08:22:46.503487+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:20:24.572550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027118630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"271'12919","reported_seq":13090,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.490879+0000","last_change":"2024-09-17T08:22:31.507556+0000","last_active":"2024-09-17T08:22:53.490879+0000","last_peered":"2024-09-17T08:22:53.490879+0000","last_clean":"2024-09-17T08:22:53.490879+0000","last_became_active":"2024-09-17T08:20:07.346402+0000","last_became_peered":"2024-09-17T08:20:07.346402+0000","last_unstale":"2024-09-17T08:22:53.490879+0000","last_undegraded":"2024-09-17T08:22:53.490879+0000","last_fullsized":"2024-09-17T08:22:53.490879+0000","mapping_epoch":221,"log_start":"251'7000","ondisk_log_start":"251'7000","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11557","last_scrub_stamp":"2024-09-17T08:22:31.507376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.507376+0000","objects_scrubbed":4853,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.972003+0000","scrub_duration":1.513714271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225509376,"num_objects":3441,"num_object_clones":0,"num_object_copies":10323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12919,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":341,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504292+0000","last_change":"2024-09-17T08:22:29.458182+0000","last_active":"2024-09-17T08:22:46.504292+0000","last_peered":"2024-09-17T08:22:46.504292+0000","last_clean":"2024-09-17T08:22:46.504292+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:22:46.504292+0000","last_undegraded":"2024-09-17T08:22:46.504292+0000","last_fullsized":"2024-09-17T08:22:46.504292+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:29.458054+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.458054+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.090765+0000","scrub_duration":0.0040122279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":272,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257558+0000","last_change":"2024-09-17T08:22:10.889503+0000","last_active":"2024-09-17T08:22:48.257558+0000","last_peered":"2024-09-17T08:22:48.257558+0000","last_clean":"2024-09-17T08:22:48.257558+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:22:48.257558+0000","last_undegraded":"2024-09-17T08:22:48.257558+0000","last_fullsized":"2024-09-17T08:22:48.257558+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:10.889256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:10.889256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.589523+0000","scrub_duration":0.006339183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":447,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:47.981498+0000","last_change":"2024-09-17T08:22:33.388252+0000","last_active":"2024-09-17T08:22:47.981498+0000","last_peered":"2024-09-17T08:22:47.981498+0000","last_clean":"2024-09-17T08:22:47.981498+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:22:47.981498+0000","last_undegraded":"2024-09-17T08:22:47.981498+0000","last_fullsized":"2024-09-17T08:22:47.981498+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.388161+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.477432+0000","scrub_duration":0.002025004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":89,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.267760+0000","last_change":"2024-09-17T08:22:06.237985+0000","last_active":"2024-09-17T08:22:48.267760+0000","last_peered":"2024-09-17T08:22:48.267760+0000","last_clean":"2024-09-17T08:22:48.267760+0000","last_became_active":"2024-09-17T08:20:02.504850+0000","last_became_peered":"2024-09-17T08:20:02.504850+0000","last_unstale":"2024-09-17T08:22:48.267760+0000","last_undegraded":"2024-09-17T08:22:48.267760+0000","last_fullsized":"2024-09-17T08:22:48.267760+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:06.237871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.237871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:17.845181+0000","scrub_duration":0.001444615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"271'12950","reported_seq":13010,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:53.632831+0000","last_change":"2024-09-17T08:20:07.154512+0000","last_active":"2024-09-17T08:22:53.632831+0000","last_peered":"2024-09-17T08:22:53.632831+0000","last_clean":"2024-09-17T08:22:53.632831+0000","last_became_active":"2024-09-17T08:20:07.154215+0000","last_became_peered":"2024-09-17T08:20:07.154215+0000","last_unstale":"2024-09-17T08:22:53.632831+0000","last_undegraded":"2024-09-17T08:22:53.632831+0000","last_fullsized":"2024-09-17T08:22:53.632831+0000","mapping_epoch":221,"log_start":"256'8000","ondisk_log_start":"256'8000","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":0,"log_size":4950,"log_dups_size":0,"ondisk_log_size":4950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220725248,"num_objects":3368,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12950,"num_write_kb":522176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":359,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.497991+0000","last_change":"2024-09-17T08:22:39.204171+0000","last_active":"2024-09-17T08:22:46.497991+0000","last_peered":"2024-09-17T08:22:46.497991+0000","last_clean":"2024-09-17T08:22:46.497991+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:22:46.497991+0000","last_undegraded":"2024-09-17T08:22:46.497991+0000","last_fullsized":"2024-09-17T08:22:46.497991+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:22:39.204047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.204047+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:03.816885+0000","scrub_duration":0.061163012000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":294,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.499411+0000","last_change":"2024-09-17T08:22:15.001277+0000","last_active":"2024-09-17T08:22:46.499411+0000","last_peered":"2024-09-17T08:22:46.499411+0000","last_clean":"2024-09-17T08:22:46.499411+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:22:46.499411+0000","last_undegraded":"2024-09-17T08:22:46.499411+0000","last_fullsized":"2024-09-17T08:22:46.499411+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:22:15.001149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.001149+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.794479+0000","scrub_duration":0.0054858570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":276,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.229428+0000","last_change":"2024-09-17T08:21:17.762815+0000","last_active":"2024-09-17T08:22:48.229428+0000","last_peered":"2024-09-17T08:22:48.229428+0000","last_clean":"2024-09-17T08:22:48.229428+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:22:48.229428+0000","last_undegraded":"2024-09-17T08:22:48.229428+0000","last_fullsized":"2024-09-17T08:22:48.229428+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:17.762302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:21:17.762302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054597830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":91,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257013+0000","last_change":"2024-09-17T08:21:32.565481+0000","last_active":"2024-09-17T08:22:48.257013+0000","last_peered":"2024-09-17T08:22:48.257013+0000","last_clean":"2024-09-17T08:22:48.257013+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:22:48.257013+0000","last_undegraded":"2024-09-17T08:22:48.257013+0000","last_fullsized":"2024-09-17T08:22:48.257013+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:32.565367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:32.565367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022451039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":303,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502988+0000","last_change":"2024-09-17T08:20:12.524600+0000","last_active":"2024-09-17T08:22:46.502988+0000","last_peered":"2024-09-17T08:22:46.502988+0000","last_clean":"2024-09-17T08:22:46.502988+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:22:46.502988+0000","last_undegraded":"2024-09-17T08:22:46.502988+0000","last_fullsized":"2024-09-17T08:22:46.502988+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:20:12.524489+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:20:12.524489+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001784494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":290,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.229466+0000","last_change":"2024-09-17T08:22:21.972879+0000","last_active":"2024-09-17T08:22:48.229466+0000","last_peered":"2024-09-17T08:22:48.229466+0000","last_clean":"2024-09-17T08:22:48.229466+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:22:48.229466+0000","last_undegraded":"2024-09-17T08:22:48.229466+0000","last_fullsized":"2024-09-17T08:22:48.229466+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:21.972582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.972582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.784884+0000","scrub_duration":0.0084090889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":279,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510804+0000","last_change":"2024-09-17T08:20:19.890506+0000","last_active":"2024-09-17T08:22:46.510804+0000","last_peered":"2024-09-17T08:22:46.510804+0000","last_clean":"2024-09-17T08:22:46.510804+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:22:46.510804+0000","last_undegraded":"2024-09-17T08:22:46.510804+0000","last_fullsized":"2024-09-17T08:22:46.510804+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:19.890356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:19.890356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012482635000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":263,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257184+0000","last_change":"2024-09-17T08:21:30.595600+0000","last_active":"2024-09-17T08:22:48.257184+0000","last_peered":"2024-09-17T08:22:48.257184+0000","last_clean":"2024-09-17T08:22:48.257184+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:22:48.257184+0000","last_undegraded":"2024-09-17T08:22:48.257184+0000","last_fullsized":"2024-09-17T08:22:48.257184+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.595456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002658952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":168,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.267932+0000","last_change":"2024-09-17T08:22:27.392439+0000","last_active":"2024-09-17T08:22:48.267932+0000","last_peered":"2024-09-17T08:22:48.267932+0000","last_clean":"2024-09-17T08:22:48.267932+0000","last_became_active":"2024-09-17T08:20:02.510430+0000","last_became_peered":"2024-09-17T08:20:02.510430+0000","last_unstale":"2024-09-17T08:22:48.267932+0000","last_undegraded":"2024-09-17T08:22:48.267932+0000","last_fullsized":"2024-09-17T08:22:48.267932+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.392168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.392168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:41.379368+0000","scrub_duration":0.0041184029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":326,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.515450+0000","last_change":"2024-09-17T08:21:30.956509+0000","last_active":"2024-09-17T08:22:46.515450+0000","last_peered":"2024-09-17T08:22:46.515450+0000","last_clean":"2024-09-17T08:22:46.515450+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:22:46.515450+0000","last_undegraded":"2024-09-17T08:22:46.515450+0000","last_fullsized":"2024-09-17T08:22:46.515450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:21:30.956385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.956385+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034003470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":193,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510036+0000","last_change":"2024-09-17T08:22:22.621469+0000","last_active":"2024-09-17T08:22:46.510036+0000","last_peered":"2024-09-17T08:22:46.510036+0000","last_clean":"2024-09-17T08:22:46.510036+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:22:46.510036+0000","last_undegraded":"2024-09-17T08:22:46.510036+0000","last_fullsized":"2024-09-17T08:22:46.510036+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.621350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:41.649184+0000","scrub_duration":0.011642807999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":281,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505373+0000","last_change":"2024-09-17T08:20:12.398221+0000","last_active":"2024-09-17T08:22:46.505373+0000","last_peered":"2024-09-17T08:22:46.505373+0000","last_clean":"2024-09-17T08:22:46.505373+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:22:46.505373+0000","last_undegraded":"2024-09-17T08:22:46.505373+0000","last_fullsized":"2024-09-17T08:22:46.505373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:12.398134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:12.398134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012127189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":346,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.511134+0000","last_change":"2024-09-17T08:22:12.398758+0000","last_active":"2024-09-17T08:22:46.511134+0000","last_peered":"2024-09-17T08:22:46.511134+0000","last_clean":"2024-09-17T08:22:46.511134+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:22:46.511134+0000","last_undegraded":"2024-09-17T08:22:46.511134+0000","last_fullsized":"2024-09-17T08:22:46.511134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:12.398295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.398295+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:13.689428+0000","scrub_duration":0.011555360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":222,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510010+0000","last_change":"2024-09-17T08:20:31.890892+0000","last_active":"2024-09-17T08:22:46.510010+0000","last_peered":"2024-09-17T08:22:46.510010+0000","last_clean":"2024-09-17T08:22:46.510010+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:22:46.510010+0000","last_undegraded":"2024-09-17T08:22:46.510010+0000","last_fullsized":"2024-09-17T08:22:46.510010+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:20:31.890809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001129559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":261,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.645761+0000","last_change":"2024-09-17T08:21:29.598725+0000","last_active":"2024-09-17T08:22:48.645761+0000","last_peered":"2024-09-17T08:22:48.645761+0000","last_clean":"2024-09-17T08:22:48.645761+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:22:48.645761+0000","last_undegraded":"2024-09-17T08:22:48.645761+0000","last_fullsized":"2024-09-17T08:22:48.645761+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:21:29.598601+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.598601+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021201536999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":281,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502287+0000","last_change":"2024-09-17T08:22:25.070949+0000","last_active":"2024-09-17T08:22:46.502287+0000","last_peered":"2024-09-17T08:22:46.502287+0000","last_clean":"2024-09-17T08:22:46.502287+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:22:46.502287+0000","last_undegraded":"2024-09-17T08:22:46.502287+0000","last_fullsized":"2024-09-17T08:22:46.502287+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:25.070821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.070821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.414437+0000","scrub_duration":0.0076690669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":228,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.505698+0000","last_change":"2024-09-17T08:22:09.487846+0000","last_active":"2024-09-17T08:22:46.505698+0000","last_peered":"2024-09-17T08:22:46.505698+0000","last_clean":"2024-09-17T08:22:46.505698+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:22:46.505698+0000","last_undegraded":"2024-09-17T08:22:46.505698+0000","last_fullsized":"2024-09-17T08:22:46.505698+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:09.487639+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.487639+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.378775+0000","scrub_duration":0.0094462299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501947+0000","last_change":"2024-09-17T08:22:26.034397+0000","last_active":"2024-09-17T08:22:46.501947+0000","last_peered":"2024-09-17T08:22:46.501947+0000","last_clean":"2024-09-17T08:22:46.501947+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:22:46.501947+0000","last_undegraded":"2024-09-17T08:22:46.501947+0000","last_fullsized":"2024-09-17T08:22:46.501947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:22:26.033887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.236921+0000","scrub_duration":0.011752558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":443,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.262193+0000","last_change":"2024-09-17T08:20:23.856731+0000","last_active":"2024-09-17T08:22:48.262193+0000","last_peered":"2024-09-17T08:22:48.262193+0000","last_clean":"2024-09-17T08:22:48.262193+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:22:48.262193+0000","last_undegraded":"2024-09-17T08:22:48.262193+0000","last_fullsized":"2024-09-17T08:22:48.262193+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:20:23.856627+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.856627+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001634627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":275,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.499114+0000","last_change":"2024-09-17T08:21:15.924915+0000","last_active":"2024-09-17T08:22:46.499114+0000","last_peered":"2024-09-17T08:22:46.499114+0000","last_clean":"2024-09-17T08:22:46.499114+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:22:46.499114+0000","last_undegraded":"2024-09-17T08:22:46.499114+0000","last_fullsized":"2024-09-17T08:22:46.499114+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.924766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.924766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057564210000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":198,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.509035+0000","last_change":"2024-09-17T08:21:12.364191+0000","last_active":"2024-09-17T08:22:46.509035+0000","last_peered":"2024-09-17T08:22:46.509035+0000","last_clean":"2024-09-17T08:22:46.509035+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:22:46.509035+0000","last_undegraded":"2024-09-17T08:22:46.509035+0000","last_fullsized":"2024-09-17T08:22:46.509035+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:21:12.364059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:12.364059+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076608910000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":145,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.268090+0000","last_change":"2024-09-17T08:22:12.321397+0000","last_active":"2024-09-17T08:22:48.268090+0000","last_peered":"2024-09-17T08:22:48.268090+0000","last_clean":"2024-09-17T08:22:48.268090+0000","last_became_active":"2024-09-17T08:20:02.492161+0000","last_became_peered":"2024-09-17T08:20:02.492161+0000","last_unstale":"2024-09-17T08:22:48.268090+0000","last_undegraded":"2024-09-17T08:22:48.268090+0000","last_fullsized":"2024-09-17T08:22:48.268090+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:12.321279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.321279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:26.989484+0000","scrub_duration":0.0030511240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":290,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510471+0000","last_change":"2024-09-17T08:22:16.495330+0000","last_active":"2024-09-17T08:22:46.510471+0000","last_peered":"2024-09-17T08:22:46.510471+0000","last_clean":"2024-09-17T08:22:46.510471+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:22:46.510471+0000","last_undegraded":"2024-09-17T08:22:46.510471+0000","last_fullsized":"2024-09-17T08:22:46.510471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:16.495149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:16.495149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:25.181375+0000","scrub_duration":0.001489045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":299,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.501393+0000","last_change":"2024-09-17T08:22:14.202171+0000","last_active":"2024-09-17T08:22:46.501393+0000","last_peered":"2024-09-17T08:22:46.501393+0000","last_clean":"2024-09-17T08:22:46.501393+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:22:46.501393+0000","last_undegraded":"2024-09-17T08:22:46.501393+0000","last_fullsized":"2024-09-17T08:22:46.501393+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.201855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.201855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:29.297433+0000","scrub_duration":0.018711583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":348,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.498495+0000","last_change":"2024-09-17T08:22:43.519295+0000","last_active":"2024-09-17T08:22:46.498495+0000","last_peered":"2024-09-17T08:22:46.498495+0000","last_clean":"2024-09-17T08:22:46.498495+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:22:46.498495+0000","last_undegraded":"2024-09-17T08:22:46.498495+0000","last_fullsized":"2024-09-17T08:22:46.498495+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:20:18.048698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:20:18.048698+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019777420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":166,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.264430+0000","last_change":"2024-09-17T08:22:18.499171+0000","last_active":"2024-09-17T08:22:48.264430+0000","last_peered":"2024-09-17T08:22:48.264430+0000","last_clean":"2024-09-17T08:22:48.264430+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:22:48.264430+0000","last_undegraded":"2024-09-17T08:22:48.264430+0000","last_fullsized":"2024-09-17T08:22:48.264430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.499009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.499009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.753804+0000","scrub_duration":0.020240600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":155,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.509365+0000","last_change":"2024-09-17T08:20:13.128701+0000","last_active":"2024-09-17T08:22:46.509365+0000","last_peered":"2024-09-17T08:22:46.509365+0000","last_clean":"2024-09-17T08:22:46.509365+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:22:46.509365+0000","last_undegraded":"2024-09-17T08:22:46.509365+0000","last_fullsized":"2024-09-17T08:22:46.509365+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:13.128622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:13.128622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013289720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":270,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510572+0000","last_change":"2024-09-17T08:21:33.941971+0000","last_active":"2024-09-17T08:22:46.510572+0000","last_peered":"2024-09-17T08:22:46.510572+0000","last_clean":"2024-09-17T08:22:46.510572+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:22:46.510572+0000","last_undegraded":"2024-09-17T08:22:46.510572+0000","last_fullsized":"2024-09-17T08:22:46.510572+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.941822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.941822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:03.413596+0000","scrub_duration":0.0057366700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":331,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510515+0000","last_change":"2024-09-17T08:22:06.283965+0000","last_active":"2024-09-17T08:22:46.510515+0000","last_peered":"2024-09-17T08:22:46.510515+0000","last_clean":"2024-09-17T08:22:46.510515+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:22:46.510515+0000","last_undegraded":"2024-09-17T08:22:46.510515+0000","last_fullsized":"2024-09-17T08:22:46.510515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:22:06.283841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.283841+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:12.974698+0000","scrub_duration":0.0044434549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":150,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.257094+0000","last_change":"2024-09-17T08:20:24.347336+0000","last_active":"2024-09-17T08:22:48.257094+0000","last_peered":"2024-09-17T08:22:48.257094+0000","last_clean":"2024-09-17T08:22:48.257094+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:22:48.257094+0000","last_undegraded":"2024-09-17T08:22:48.257094+0000","last_fullsized":"2024-09-17T08:22:48.257094+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:24.347238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:20:24.347238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013570769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":141,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.267747+0000","last_change":"2024-09-17T08:21:29.113802+0000","last_active":"2024-09-17T08:22:48.267747+0000","last_peered":"2024-09-17T08:22:48.267747+0000","last_clean":"2024-09-17T08:22:48.267747+0000","last_became_active":"2024-09-17T08:20:02.510678+0000","last_became_peered":"2024-09-17T08:20:02.510678+0000","last_unstale":"2024-09-17T08:22:48.267747+0000","last_undegraded":"2024-09-17T08:22:48.267747+0000","last_fullsized":"2024-09-17T08:22:48.267747+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.113668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00142106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":285,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504756+0000","last_change":"2024-09-17T08:21:14.392541+0000","last_active":"2024-09-17T08:22:46.504756+0000","last_peered":"2024-09-17T08:22:46.504756+0000","last_clean":"2024-09-17T08:22:46.504756+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:22:46.504756+0000","last_undegraded":"2024-09-17T08:22:46.504756+0000","last_fullsized":"2024-09-17T08:22:46.504756+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:14.392384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.392384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021806350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":305,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510797+0000","last_change":"2024-09-17T08:22:31.627432+0000","last_active":"2024-09-17T08:22:46.510797+0000","last_peered":"2024-09-17T08:22:46.510797+0000","last_clean":"2024-09-17T08:22:46.510797+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:22:46.510797+0000","last_undegraded":"2024-09-17T08:22:46.510797+0000","last_fullsized":"2024-09-17T08:22:46.510797+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:31.627297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.627297+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:49.425907+0000","scrub_duration":0.0041108009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":181,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.268333+0000","last_change":"2024-09-17T08:22:21.330021+0000","last_active":"2024-09-17T08:22:48.268333+0000","last_peered":"2024-09-17T08:22:48.268333+0000","last_clean":"2024-09-17T08:22:48.268333+0000","last_became_active":"2024-09-17T08:20:23.487930+0000","last_became_peered":"2024-09-17T08:20:23.487930+0000","last_unstale":"2024-09-17T08:22:48.268333+0000","last_undegraded":"2024-09-17T08:22:48.268333+0000","last_fullsized":"2024-09-17T08:22:48.268333+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:22:21.329888+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.329888+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.638203+0000","scrub_duration":0.0021372819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":283,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504102+0000","last_change":"2024-09-17T08:21:31.458346+0000","last_active":"2024-09-17T08:22:46.504102+0000","last_peered":"2024-09-17T08:22:46.504102+0000","last_clean":"2024-09-17T08:22:46.504102+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:22:46.504102+0000","last_undegraded":"2024-09-17T08:22:46.504102+0000","last_fullsized":"2024-09-17T08:22:46.504102+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.458200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.458200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020112709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":270,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510259+0000","last_change":"2024-09-17T08:22:39.655694+0000","last_active":"2024-09-17T08:22:46.510259+0000","last_peered":"2024-09-17T08:22:46.510259+0000","last_clean":"2024-09-17T08:22:46.510259+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:22:46.510259+0000","last_undegraded":"2024-09-17T08:22:46.510259+0000","last_fullsized":"2024-09-17T08:22:46.510259+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:39.655581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.655581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.268378+0000","scrub_duration":0.0042238659999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":287,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.503350+0000","last_change":"2024-09-17T08:21:19.480328+0000","last_active":"2024-09-17T08:22:46.503350+0000","last_peered":"2024-09-17T08:22:46.503350+0000","last_clean":"2024-09-17T08:22:46.503350+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:22:46.503350+0000","last_undegraded":"2024-09-17T08:22:46.503350+0000","last_fullsized":"2024-09-17T08:22:46.503350+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:21:19.480200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:19.480200+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035346700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":176,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.497940+0000","last_change":"2024-09-17T08:20:19.999322+0000","last_active":"2024-09-17T08:22:46.497940+0000","last_peered":"2024-09-17T08:22:46.497940+0000","last_clean":"2024-09-17T08:22:46.497940+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:22:46.497940+0000","last_undegraded":"2024-09-17T08:22:46.497940+0000","last_fullsized":"2024-09-17T08:22:46.497940+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:19.999232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:20:19.999232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001724136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":316,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502907+0000","last_change":"2024-09-17T08:22:20.129709+0000","last_active":"2024-09-17T08:22:46.502907+0000","last_peered":"2024-09-17T08:22:46.502907+0000","last_clean":"2024-09-17T08:22:46.502907+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:22:46.502907+0000","last_undegraded":"2024-09-17T08:22:46.502907+0000","last_fullsized":"2024-09-17T08:22:46.502907+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:20.129556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.129556+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:40.752500+0000","scrub_duration":0.010294539,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":360,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:47.983295+0000","last_change":"2024-09-17T08:20:10.960103+0000","last_active":"2024-09-17T08:22:47.983295+0000","last_peered":"2024-09-17T08:22:47.983295+0000","last_clean":"2024-09-17T08:22:47.983295+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:22:47.983295+0000","last_undegraded":"2024-09-17T08:22:47.983295+0000","last_fullsized":"2024-09-17T08:22:47.983295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:20:10.959993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:10.959993+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014768190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":165,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504073+0000","last_change":"2024-09-17T08:20:15.122143+0000","last_active":"2024-09-17T08:22:46.504073+0000","last_peered":"2024-09-17T08:22:46.504073+0000","last_clean":"2024-09-17T08:22:46.504073+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:22:46.504073+0000","last_undegraded":"2024-09-17T08:22:46.504073+0000","last_fullsized":"2024-09-17T08:22:46.504073+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:15.122049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:15.122049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016647579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":276,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510916+0000","last_change":"2024-09-17T08:21:31.941666+0000","last_active":"2024-09-17T08:22:46.510916+0000","last_peered":"2024-09-17T08:22:46.510916+0000","last_clean":"2024-09-17T08:22:46.510916+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:22:46.510916+0000","last_undegraded":"2024-09-17T08:22:46.510916+0000","last_fullsized":"2024-09-17T08:22:46.510916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.941534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:52.896325+0000","scrub_duration":0.0066151480000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":174,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:48.268307+0000","last_change":"2024-09-17T08:22:39.435438+0000","last_active":"2024-09-17T08:22:48.268307+0000","last_peered":"2024-09-17T08:22:48.268307+0000","last_clean":"2024-09-17T08:22:48.268307+0000","last_became_active":"2024-09-17T08:20:02.495177+0000","last_became_peered":"2024-09-17T08:20:02.495177+0000","last_unstale":"2024-09-17T08:22:48.268307+0000","last_undegraded":"2024-09-17T08:22:48.268307+0000","last_fullsized":"2024-09-17T08:22:48.268307+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:39.435291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.435291+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:42.016755+0000","scrub_duration":0.005519843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":301,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.502821+0000","last_change":"2024-09-17T08:21:19.815826+0000","last_active":"2024-09-17T08:22:46.502821+0000","last_peered":"2024-09-17T08:22:46.502821+0000","last_clean":"2024-09-17T08:22:46.502821+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:22:46.502821+0000","last_undegraded":"2024-09-17T08:22:46.502821+0000","last_fullsized":"2024-09-17T08:22:46.502821+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:19.815475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:21:19.815475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061512909999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":271,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.512534+0000","last_change":"2024-09-17T08:22:33.121286+0000","last_active":"2024-09-17T08:22:46.512534+0000","last_peered":"2024-09-17T08:22:46.512534+0000","last_clean":"2024-09-17T08:22:46.512534+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:22:46.512534+0000","last_undegraded":"2024-09-17T08:22:46.512534+0000","last_fullsized":"2024-09-17T08:22:46.512534+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:33.121209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.121209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.285318+0000","scrub_duration":0.0012454370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":228,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:47.539656+0000","last_change":"2024-09-17T08:21:25.000057+0000","last_active":"2024-09-17T08:22:47.539656+0000","last_peered":"2024-09-17T08:22:47.539656+0000","last_clean":"2024-09-17T08:22:47.539656+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:22:47.539656+0000","last_undegraded":"2024-09-17T08:22:47.539656+0000","last_fullsized":"2024-09-17T08:22:47.539656+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.999967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038102280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":394,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-17T08:22:51.379388+0000","last_change":"2024-09-17T08:22:20.983964+0000","last_active":"2024-09-17T08:22:51.379388+0000","last_peered":"2024-09-17T08:22:51.379388+0000","last_clean":"2024-09-17T08:22:51.379388+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:22:51.379388+0000","last_undegraded":"2024-09-17T08:22:51.379388+0000","last_fullsized":"2024-09-17T08:22:51.379388+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:20.983823+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.983823+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.519128+0000","scrub_duration":0.010983009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":165,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.504224+0000","last_change":"2024-09-17T08:22:22.507255+0000","last_active":"2024-09-17T08:22:46.504224+0000","last_peered":"2024-09-17T08:22:46.504224+0000","last_clean":"2024-09-17T08:22:46.504224+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:22:46.504224+0000","last_undegraded":"2024-09-17T08:22:46.504224+0000","last_fullsized":"2024-09-17T08:22:46.504224+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.507142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.507142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:52.303965+0000","scrub_duration":0.0043685180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":260,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510727+0000","last_change":"2024-09-17T08:20:20.860711+0000","last_active":"2024-09-17T08:22:46.510727+0000","last_peered":"2024-09-17T08:22:46.510727+0000","last_clean":"2024-09-17T08:22:46.510727+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:22:46.510727+0000","last_undegraded":"2024-09-17T08:22:46.510727+0000","last_fullsized":"2024-09-17T08:22:46.510727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:20.860156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:20.860156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076486699999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":276,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510526+0000","last_change":"2024-09-17T08:21:24.016408+0000","last_active":"2024-09-17T08:22:46.510526+0000","last_peered":"2024-09-17T08:22:46.510526+0000","last_clean":"2024-09-17T08:22:46.510526+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:22:46.510526+0000","last_undegraded":"2024-09-17T08:22:46.510526+0000","last_fullsized":"2024-09-17T08:22:46.510526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.016289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.016289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011053393999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":348,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.510563+0000","last_change":"2024-09-17T08:21:27.998120+0000","last_active":"2024-09-17T08:22:46.510563+0000","last_peered":"2024-09-17T08:22:46.510563+0000","last_clean":"2024-09-17T08:22:46.510563+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:22:46.510563+0000","last_undegraded":"2024-09-17T08:22:46.510563+0000","last_fullsized":"2024-09-17T08:22:46.510563+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:21:27.997982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.997982+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010572906999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":288,"reported_epoch":270,"state":"active+clean","last_fresh":"2024-09-17T08:22:46.499048+0000","last_change":"2024-09-17T08:21:21.069425+0000","last_active":"2024-09-17T08:22:46.499048+0000","last_peered":"2024-09-17T08:22:46.499048+0000","last_clean":"2024-09-17T08:22:46.499048+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:22:46.499048+0000","last_undegraded":"2024-09-17T08:22:46.499048+0000","last_fullsized":"2024-09-17T08:22:46.499048+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.069304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.069304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013987202000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":3585998872,"num_objects":54719,"num_object_clones":0,"num_object_copies":164157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54719,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207675,"num_write_kb":8396545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2199,"num_bytes_recovered":144113664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10745753600,"data_stored":10745741384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82612,"ondisk_log_size":82612,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1121,"num_read_kb":878,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":441,"num_read_kb":436,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070625,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2495836,"kb_used_data":2017620,"kb_used_omap":1,"kb_used_meta":478206,"kb_avail":91249316,"statfs":{"total":95995035648,"available":93439299584,"internally_reserved":0,"allocated":2066042880,"data_stored":2065743099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":489683403},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.6970000000000001,"15min":1.2090000000000001},"min":{"1min":0.223,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.2400000000000002,"5min":62.018000000000001,"15min":62.018000000000001},"last":2.391},{"interface":"front","average":{"1min":1.212,"5min":1.7350000000000001,"15min":1.2809999999999999},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":3.4329999999999998,"5min":61.896999999999998,"15min":61.896999999999998},"last":2.7290000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":4.0490000000000004,"5min":2.6019999999999999,"15min":1.3480000000000001},"min":{"1min":0.33000000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":44.917999999999999,"5min":62.067,"15min":62.067},"last":2.4889999999999999},{"interface":"front","average":{"1min":3.7090000000000001,"5min":2.3500000000000001,"15min":1.2270000000000001},"min":{"1min":0.43099999999999999,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":44.996000000000002,"5min":62.194000000000003,"15min":62.194000000000003},"last":2.7589999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:22:25 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.89600000000000002,"15min":0.86699999999999999},"min":{"1min":0.39000000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":2.8050000000000002},{"interface":"front","average":{"1min":1.3169999999999999,"5min":0.70899999999999996,"15min":0.60799999999999998},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":3.492,"5min":1.4710000000000001,"15min":1.1040000000000001},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":44.606000000000002,"5min":44.606000000000002,"15min":44.606000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":4.117,"5min":1.75,"15min":1.2430000000000001},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":44.591000000000001,"5min":44.591000000000001,"15min":44.591000000000001},"last":2.4569999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.80800000000000005,"15min":0.57399999999999995},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.6200000000000001,"5min":5.335,"15min":5.335},"last":2.7839999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68000000000000005,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.26100000000000001,"15min":0.23899999999999999},"max":{"1min":2.7749999999999999,"5min":5.9669999999999996,"15min":5.9669999999999996},"last":0.34699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":0.89000000000000001,"15min":0.68400000000000005},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":8.6400000000000006,"5min":8.6400000000000006,"15min":16.808},"last":2.5099999999999998},{"interface":"front","average":{"1min":1.083,"5min":0.878,"15min":0.66600000000000004},"min":{"1min":0.34200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.5899999999999999,"5min":5.7400000000000002,"15min":11.842000000000001},"last":3.0299999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":0.96299999999999997,"15min":0.68500000000000005},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.26600000000000001},"max":{"1min":6.6509999999999998,"5min":7.2359999999999998,"15min":7.2359999999999998},"last":2.8239999999999998},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.74199999999999999,"15min":0.58699999999999997},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":2.8500000000000001,"5min":5.9400000000000004,"15min":5.9400000000000004},"last":0.36799999999999999}]}]},{"osd":6,"up_from":50,"seq":214748364967,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":939928,"kb_used_data":615948,"kb_used_omap":1,"kb_used_meta":323966,"kb_avail":92805224,"statfs":{"total":95995035648,"available":95032549376,"internally_reserved":0,"allocated":630730752,"data_stored":630446620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":331741639},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":2.0470000000000002,"15min":2.0190000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.2189999999999999,"5min":81.007000000000005,"15min":81.007000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":2.0579999999999998,"15min":2.0710000000000002},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.19900000000000001},"max":{"1min":4.8710000000000004,"5min":80.310000000000002,"15min":80.310000000000002},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":2.516,"15min":1.5369999999999999},"min":{"1min":0.28699999999999998,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":3.7869999999999999,"5min":80.813000000000002,"15min":80.813000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.101,"5min":2.5190000000000001,"15min":1.5820000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5259999999999998,"5min":80.281000000000006,"15min":80.281000000000006},"last":0.41199999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.67400000000000004,"15min":0.64000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.6930000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":57.305999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.57899999999999996,"15min":0.54600000000000004},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":57.289999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":2.294,"15min":1.8420000000000001},"min":{"1min":0.32000000000000001,"5min":0.29899999999999999,"15min":0.28699999999999998},"max":{"1min":3.8250000000000002,"5min":80.427000000000007,"15min":80.427000000000007},"last":0.42799999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":2.125,"15min":1.764},"min":{"1min":0.36899999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.2930000000000001,"5min":80.379999999999995,"15min":80.379999999999995},"last":0.44400000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67900000000000005,"15min":0.56899999999999995},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.215},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":4.6219999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.51800000000000002},"min":{"1min":0.30299999999999999,"5min":0.26800000000000002,"15min":0.25},"max":{"1min":2.4060000000000001,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.80700000000000005,"15min":0.61599999999999999},"min":{"1min":0.33100000000000002,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":5.5659999999999998,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.504},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.57899999999999996},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1869999999999998,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Tue Sep 17 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.79100000000000004,"15min":0.64700000000000002},"min":{"1min":0.317,"5min":0.246,"15min":0.246},"max":{"1min":2.6909999999999998,"5min":4.774,"15min":4.774},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.76100000000000001,"15min":0.59799999999999998},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5590000000000002,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":0.57899999999999996}]}]},{"osd":5,"up_from":41,"seq":176093659307,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2496268,"kb_used_data":2013892,"kb_used_omap":1,"kb_used_meta":482302,"kb_avail":91248884,"statfs":{"total":95995035648,"available":93438857216,"internally_reserved":0,"allocated":2062225408,"data_stored":2061940676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":493877702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.458,"15min":1.357},"min":{"1min":0.21299999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":3.4670000000000001,"5min":28.867000000000001,"15min":63.098999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.482,"15min":1.304},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.9620000000000002,"5min":28.998999999999999,"15min":62.567},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.321,"15min":0.91100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":3.0830000000000002,"5min":17.457000000000001,"15min":17.457000000000001},"last":31.600999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3080000000000001,"15min":0.97299999999999998},"min":{"1min":0.26300000000000001,"5min":0.248,"15min":0.22},"max":{"1min":5.8810000000000002,"5min":17.274000000000001,"15min":17.274000000000001},"last":26.942}]},{"osd":2,"last update":"Tue Sep 17 08:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.77600000000000002,"15min":0.745},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.012,"15min":1.075},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.1200000000000001,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.29299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.1339999999999999,"15min":1.2629999999999999},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":11.291,"5min":11.291,"15min":75.286000000000001},"last":26.513999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.113,"15min":1.2450000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":77.206999999999994},"last":28.190000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":1.117,"15min":0.66300000000000003},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.19},"max":{"1min":1.05,"5min":42.421999999999997,"15min":42.421999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.0740000000000001,"15min":0.67600000000000005},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":2.573,"5min":42.191000000000003,"15min":42.191000000000003},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.57499999999999996},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27700000000000002},"max":{"1min":3.4769999999999999,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.69699999999999995,"15min":0.58499999999999996},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":1.2909999999999999,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.436}]},{"osd":7,"last update":"Tue Sep 17 08:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.91100000000000003,"15min":0.66300000000000003},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":3.3610000000000002,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.80200000000000005,"15min":0.60899999999999999},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":2.5750000000000002,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.66300000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953648,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":989460,"kb_used_data":669580,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":92755692,"statfs":{"total":95995035648,"available":94981828608,"internally_reserved":0,"allocated":685649920,"data_stored":685357601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":327547334},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.91200000000000003},"min":{"1min":0.24099999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":23.786000000000001},"last":4.5229999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.219,"15min":0.219},"max":{"1min":2.2890000000000001,"5min":3.7839999999999998,"15min":23.245000000000001},"last":0.29799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":1.5640000000000001,"15min":1.0860000000000001},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.3789999999999996,"5min":14.164,"15min":23.135000000000002},"last":1.635},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.401,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.246,"15min":0.24099999999999999},"max":{"1min":9.0719999999999992,"5min":9.0719999999999992,"15min":23.434999999999999},"last":1.5}]},{"osd":2,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.65400000000000003,"15min":0.59199999999999997},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.6950000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":0.66700000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.353,"15min":1.7969999999999999},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.226,"5min":10.012,"15min":154.137},"last":3.1400000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.208,"15min":1.6779999999999999},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":5.556,"5min":9.5180000000000007,"15min":153.54400000000001},"last":2.2370000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.55400000000000005},"min":{"1min":0.27600000000000002,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":3.0510000000000002,"5min":6.2329999999999997,"15min":6.2329999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.872,"15min":0.59599999999999997},"min":{"1min":0.27000000000000002,"5min":0.26800000000000002,"15min":0.25900000000000001},"max":{"1min":3.3940000000000001,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":2.4940000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76200000000000001,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":1.9750000000000001,"5min":4.827,"15min":4.827},"last":1.526},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.66900000000000004,"15min":0.58699999999999997},"min":{"1min":0.33700000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82799999999999996,"5min":3.601,"15min":6.0010000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.93600000000000005,"15min":0.65000000000000002},"min":{"1min":0.371,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.9860000000000002,"5min":4.875,"15min":4.875},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.90500000000000003,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.27300000000000002},"max":{"1min":2.4470000000000001,"5min":4.6950000000000003,"15min":4.6950000000000003},"last":1.5960000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117172,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2319648,"kb_used_data":1792920,"kb_used_omap":1,"kb_used_meta":526590,"kb_avail":91425504,"statfs":{"total":95995035648,"available":93619716096,"internally_reserved":0,"allocated":1835950080,"data_stored":1835644950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":539228616},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.958,"5min":1.7230000000000001,"15min":1.3520000000000001},"min":{"1min":0.156,"5min":0.156,"15min":0.156},"max":{"1min":9.7029999999999994,"5min":9.7029999999999994,"15min":9.7029999999999994},"last":0.218},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.5069999999999999,"15min":1.2270000000000001},"min":{"1min":0.254,"5min":0.17799999999999999,"15min":0.16200000000000001},"max":{"1min":4.3049999999999997,"5min":14.118,"15min":14.118},"last":0.26300000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.6259999999999999,"15min":1.2829999999999999},"min":{"1min":0.28899999999999998,"5min":0.254,"15min":0.186},"max":{"1min":6.7039999999999997,"5min":9.5,"15min":9.5},"last":0.32900000000000001},{"interface":"front","average":{"1min":2.2189999999999999,"5min":1.552,"15min":1.2709999999999999},"min":{"1min":0.27700000000000002,"5min":0.192,"15min":0.188},"max":{"1min":5.9210000000000003,"5min":5.9210000000000003,"15min":10.891999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:22:27 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.167,"15min":1.105},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":5.3899999999999997,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.994,"5min":1.712,"15min":1.665},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.2809999999999997,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":0.30099999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:21:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6990000000000001,"5min":1.579,"15min":1.1659999999999999},"min":{"1min":0.377,"5min":0.28100000000000003,"15min":0.252},"max":{"1min":5.5460000000000003,"5min":35.841000000000001,"15min":35.841000000000001},"last":0.378},{"interface":"front","average":{"1min":1.863,"5min":1.629,"15min":1.1439999999999999},"min":{"1min":0.33100000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.5999999999999996,"5min":35.857999999999997,"15min":35.857999999999997},"last":0.35699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":1.3140000000000001,"15min":1.1100000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.629,"15min":1.2330000000000001},"min":{"1min":0.44900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.8520000000000003,"5min":9.4260000000000002,"15min":9.4260000000000002},"last":0.36499999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:22:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.27,"15min":1.087},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.29499999999999998},"max":{"1min":7.6669999999999998,"5min":7.6669999999999998,"15min":10.945},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6870000000000001,"15min":1.302},"min":{"1min":0.39000000000000001,"5min":0.314,"15min":0.311},"max":{"1min":9.6600000000000001,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.39000000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:22:01 2024","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.343,"15min":1.111},"min":{"1min":0.36199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":7.6390000000000002,"5min":7.6390000000000002,"15min":8.4149999999999991},"last":38.936999999999998},{"interface":"front","average":{"1min":1.768,"5min":1.3180000000000001,"15min":1.087},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":5.5780000000000003,"5min":5.5780000000000003,"15min":7.7830000000000004},"last":36.634999999999998}]}]},{"osd":2,"up_from":227,"seq":974957576224,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718172,"kb_used_data":622288,"kb_used_omap":4,"kb_used_meta":95867,"kb_avail":93026980,"statfs":{"total":95995035648,"available":95259627520,"internally_reserved":0,"allocated":637222912,"data_stored":636934487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":98168483},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":1.143,"15min":1.0289999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":7.7450000000000001,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.3939999999999999,"15min":1.4410000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.8300000000000001,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.2889999999999999,"15min":1.25},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":5.8129999999999997,"5min":5.8129999999999997,"15min":5.8129999999999997},"last":0.309},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.3089999999999999,"15min":1.284},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":4.5810000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.72299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.4630000000000001,"15min":1.4379999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.248},{"interface":"front","average":{"1min":1.9910000000000001,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3200000000000003,"5min":5.3200000000000003,"15min":5.3200000000000003},"last":0.27100000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":0.90200000000000002,"15min":0.84999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5870000000000002,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.157,"5min":0.88800000000000001,"15min":0.84299999999999997},"min":{"1min":0.307,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.411,"5min":3.411,"15min":3.411},"last":0.377}]},{"osd":5,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.5009999999999999,"5min":4.915,"15min":4.915},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.194,"5min":0.97499999999999998,"15min":0.93899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.4489999999999998,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.65900000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.77900000000000003,"15min":0.71299999999999997},"min":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.262,"5min":0.95399999999999996,"15min":0.90300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5409999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.036,"15min":0.99399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.93899999999999995,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":5.2690000000000001,"5min":5.2690000000000001,"15min":5.2690000000000001},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2587244,"kb_used_data":2006500,"kb_used_omap":1,"kb_used_meta":580734,"kb_avail":91157908,"statfs":{"total":95995035648,"available":93345697792,"internally_reserved":0,"allocated":2054656000,"data_stored":2054339196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":594672070},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:21:52 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.496,"15min":1.3169999999999999},"min":{"1min":0.19500000000000001,"5min":0.155,"15min":0.155},"max":{"1min":3.302,"5min":8.4100000000000001,"15min":31.585999999999999},"last":9.5730000000000004},{"interface":"front","average":{"1min":3.4100000000000001,"5min":1.913,"15min":1.516},"min":{"1min":0.45000000000000001,"5min":0.19500000000000001,"15min":0.17199999999999999},"max":{"1min":16.812000000000001,"5min":16.812000000000001,"15min":35.807000000000002},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:22:24 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.1439999999999999,"15min":0.91000000000000003},"min":{"1min":0.315,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":22.943000000000001,"5min":22.943000000000001,"15min":22.943000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.55,"5min":1.498,"15min":1.4890000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":5.4560000000000004,"5min":5.4560000000000004,"15min":5.4560000000000004},"last":9.6110000000000007}]},{"osd":3,"last update":"Tue Sep 17 08:22:34 2024","interfaces":[{"interface":"back","average":{"1min":2.161,"5min":1.6140000000000001,"15min":1.2290000000000001},"min":{"1min":0.441,"5min":0.184,"15min":0.184},"max":{"1min":5.1920000000000002,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":4.5679999999999996},{"interface":"front","average":{"1min":3.5449999999999999,"5min":2.2469999999999999,"15min":1.4279999999999999},"min":{"1min":0.41399999999999998,"5min":0.23699999999999999,"15min":0.189},"max":{"1min":22.984999999999999,"5min":22.984999999999999,"15min":22.984999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:21:59 2024","interfaces":[{"interface":"back","average":{"1min":2.3330000000000002,"5min":1.698,"15min":1.143},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":16.884,"5min":16.884,"15min":16.884},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.369,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.2290000000000001,"5min":6.6580000000000004,"15min":10.208},"last":9.6890000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:22:14 2024","interfaces":[{"interface":"back","average":{"1min":1.8380000000000001,"5min":1.411,"15min":1.0389999999999999},"min":{"1min":0.50700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.9740000000000002,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":4.4989999999999997},{"interface":"front","average":{"1min":2.6139999999999999,"5min":1.8069999999999999,"15min":1.2290000000000001},"min":{"1min":0.45500000000000002,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":16.853999999999999,"5min":16.853999999999999,"15min":16.853999999999999},"last":1.1890000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.605,"15min":1.125},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.28199999999999997},"max":{"1min":3.2869999999999999,"5min":6.6020000000000003,"15min":10.253},"last":9.7159999999999993},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.575,"15min":1.153},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.27200000000000002},"max":{"1min":3.4039999999999999,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":4.5369999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:22:09 2024","interfaces":[{"interface":"back","average":{"1min":2.847,"5min":1.8080000000000001,"15min":1.2290000000000001},"min":{"1min":0.57099999999999995,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":16.908999999999999,"5min":16.908999999999999,"15min":16.908999999999999},"last":0.624},{"interface":"front","average":{"1min":1.506,"5min":1.526,"15min":1.125},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":3.2349999999999999,"5min":6.5620000000000003,"15min":10.279},"last":9.7379999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738560,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1288976,"kb_used_data":884616,"kb_used_omap":1,"kb_used_meta":404350,"kb_avail":92456176,"statfs":{"total":95995035648,"available":94675124224,"internally_reserved":0,"allocated":905846784,"data_stored":905557405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":414054854},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:21:51 2024","interfaces":[{"interface":"back","average":{"1min":4.827,"5min":3.198,"15min":1.629},"min":{"1min":0.254,"5min":0.189,"15min":0.189},"max":{"1min":41.247,"5min":61.649000000000001,"15min":61.649000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":5.1120000000000001,"5min":3.2290000000000001,"15min":1.7649999999999999},"min":{"1min":0.253,"5min":0.183,"15min":0.18099999999999999},"max":{"1min":40.436,"5min":61.281999999999996,"15min":61.281999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:22:29 2024","interfaces":[{"interface":"back","average":{"1min":2.379,"5min":1.383,"15min":1.2170000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":39.023000000000003,"5min":39.023000000000003,"15min":39.023000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":3.0350000000000001,"5min":1.3140000000000001,"15min":1.0269999999999999},"min":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":41.003999999999998,"5min":41.003999999999998,"15min":41.003999999999998},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:22:37 2024","interfaces":[{"interface":"back","average":{"1min":1.738,"5min":2.23,"15min":1.847},"min":{"1min":0.20799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.2199999999999998,"5min":39.942999999999998,"15min":61.926000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.8380000000000001,"15min":1.649},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":5.7679999999999998,"5min":38.950000000000003,"15min":61.616},"last":0.73799999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:21:51 2024","interfaces":[{"interface":"back","average":{"1min":2.786,"5min":1.7929999999999999,"15min":1.2030000000000001},"min":{"1min":0.34799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":41.158000000000001,"5min":41.158000000000001,"15min":41.158000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":2.6480000000000001,"5min":1.871,"15min":1.238},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":38.978000000000002,"5min":38.978000000000002,"15min":38.978000000000002},"last":0.623}]},{"osd":5,"last update":"Tue Sep 17 08:22:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.7529999999999999,"15min":1.157},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1559999999999997,"5min":39.058,"15min":39.058},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.998,"15min":1.3109999999999999},"min":{"1min":0.38500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":5.101,"5min":43.485999999999997,"15min":43.485999999999997},"last":0.435}]},{"osd":6,"last update":"Tue Sep 17 08:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.7450000000000001,"15min":1.1879999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.919,"5min":38.917999999999999,"15min":38.917999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.9810000000000001,"15min":1.331},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":5.1239999999999997,"5min":41.189,"15min":41.189},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8849999999999998,"5min":1.9119999999999999,"15min":1.395},"min":{"1min":0.374,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":39.045999999999999,"5min":39.045999999999999,"15min":39.045999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.8879999999999999,"15min":1.3340000000000001},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.307},"max":{"1min":39.082999999999998,"5min":39.082999999999998,"15min":39.082999999999998},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":890044416,"data_stored":890044416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2015236096,"data_stored":2015232024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":633143296,"data_stored":633143296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1803157504,"data_stored":1803157504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":680263680,"data_stored":680263680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2056392704,"data_stored":2056388632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":614137856,"data_stored":614137856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2053378048,"data_stored":2053373976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:23:02.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:23:02.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:02.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:02.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:02.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:02.940 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:02.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:03.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:03.082 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:03.095 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:03.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:03.098 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:03.197 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:03.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.238 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:03.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:03.259 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:23:03.334 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:03.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:03.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.370 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:03.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:03.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:03.370 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:03.371 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:03.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:03.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:03.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:03.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:03.645 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:23:03.653 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:03.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.654 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:03.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:03.776 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:03.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.777 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:03.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:03.912 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:03.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:03.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:04.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:04.047 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:04.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:04.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:04.190 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:04.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:04.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:04.325 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:04.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:04.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:04.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:04 smithi079 ceph-mon[40649]: pgmap v578: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 643 B/s rd, 0 B/s wr, 931 op/s 2024-09-17T08:23:04.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:04.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:04 smithi079 ceph-mon[29565]: pgmap v578: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 643 B/s rd, 0 B/s wr, 931 op/s 2024-09-17T08:23:04.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:23:04.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:04.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.467 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:04.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:04.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:04 smithi173 ceph-mon[33187]: pgmap v578: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 643 B/s rd, 0 B/s wr, 931 op/s 2024-09-17T08:23:04.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:04.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:04.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:04.705 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:04.738 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:04.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:04.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:04.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:04.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:04.880 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:04.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:05.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.016 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:05.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:05.117 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:05.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:05.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:05.285 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:05.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:05.427 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:05.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:05.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:05.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:05.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:05.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:05.713 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:23:05.713 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:23:05.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:05.824 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:05 smithi079 ceph-mon[40649]: pgmap v579: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:23:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:05 smithi079 ceph-mon[29565]: pgmap v579: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:23:05.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:05.936 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:05.957 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:05.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:05.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:05 smithi173 ceph-mon[33187]: pgmap v579: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:23:06.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:06.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:06.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:06.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:06.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:06.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:06.352 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:06.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:06.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:06.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:23:06.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.496 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:06.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:06.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:06.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:06.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:06.763 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:06.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:06.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:06.883 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:23:06.913 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:06.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:06.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:07.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:07.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:07.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:07.152 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:07.196 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:07.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:07.298 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:07.328 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:07.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:07.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:07.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:07.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:07.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:07.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:07.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:07.713 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:07 smithi079 ceph-mon[40649]: pgmap v580: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 902 op/s 2024-09-17T08:23:07.714 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:07 smithi079 ceph-mon[29565]: pgmap v580: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 902 op/s 2024-09-17T08:23:07.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:07 smithi173 ceph-mon[33187]: pgmap v580: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 902 op/s 2024-09-17T08:23:07.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:07.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:07.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:07.892 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:07.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:07.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:07.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:08.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.033 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:08.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:08.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:08.170 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:08.272 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:08.303 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:08.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:08.396 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:08.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:08.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.459 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:08.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:08.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:08.581 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:08 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:23:08.595 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:08.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:08.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:08 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:23:08.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:08 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:23:08.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:08.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:08.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:08.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:08.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:08.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:08.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:08.981 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:23:08.981 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:23:09.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:09.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.066 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:09.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:09.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:09.213 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:09.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:09.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:09.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:09.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:09.491 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:09.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:09.593 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:09.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:09.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:09.654 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:09 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:23:09.654 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:09 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:23:09.654 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:09 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:09.654 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:09 smithi173 ceph-mon[33187]: pgmap v581: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:09.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:23:09.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:09.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:09.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:09.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[40649]: pgmap v581: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:09 smithi079 ceph-mon[29565]: pgmap v581: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:09.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:09.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:09.899 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:10.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:10.043 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:10.059 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:23:10.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:10.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.180 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:10.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:10.314 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:10.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:10.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:10.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:10.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.590 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:10.591 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:23:10.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:10.719 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1300193712' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:10.719 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:10 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:10.722 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1300193712' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:10.722 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:10 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:10.737 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:10.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:10.839 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:10.865 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:10.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:10.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:10.967 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:10.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1300193712' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:10.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:10 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:23:11.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:11.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:11.070 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:23:11.072 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:23:11.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:11.136 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:11.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:11.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:11.259 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:11.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:11.361 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:11.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.398 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:11.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:11.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:11.540 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:11.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:11.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:11.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:11.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:11.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:11.800 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:11.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:23:11.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[40649]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T08:23:11.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[40649]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:11.801 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[40649]: pgmap v583: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:11.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:11.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:23:11.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[29565]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T08:23:11.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[29565]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:11.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:11 smithi079 ceph-mon[29565]: pgmap v583: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:11.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:11.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:11.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:11 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:11.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:11 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:23:11.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:11 smithi173 ceph-mon[33187]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T08:23:11.892 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:11 smithi173 ceph-mon[33187]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:11.892 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:11 smithi173 ceph-mon[33187]: pgmap v583: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:11.920 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:11.955 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:11.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:11.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:12.056 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:12.095 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:12.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.097 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:12.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:12.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:12.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:12.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:12.351 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:23:12.351 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:23:12.378 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:12.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.379 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:12.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:12.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:12.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:12.532 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":582,"stamp":"2024-09-17T08:23:10.584471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2442860846,"num_objects":37463,"num_object_clones":0,"num_object_copies":112389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37463,"num_whiteouts":0,"num_read":1802,"num_read_kb":1648,"num_write":226022,"num_write_kb":8399004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2431,"num_bytes_recovered":145889393,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83872,"ondisk_log_size":83872,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10609164,"kb_used_data":7363532,"kb_used_omap":15,"kb_used_meta":3245552,"kb_avail":739352052,"statfs":{"total":767960285184,"available":757096501248,"internally_reserved":0,"allocated":7540256768,"data_stored":7537806192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":3323446037},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-805896192,"num_objects":-12297,"num_object_clones":0,"num_object_copies":-36891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12297,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12297,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509544"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":273,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624099+0000","last_change":"2024-09-17T08:22:32.714143+0000","last_active":"2024-09-17T08:22:58.624099+0000","last_peered":"2024-09-17T08:22:58.624099+0000","last_clean":"2024-09-17T08:22:58.624099+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:22:58.624099+0000","last_undegraded":"2024-09-17T08:22:58.624099+0000","last_fullsized":"2024-09-17T08:22:58.624099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.714032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.714032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.893234+0000","scrub_duration":0.052748915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":291,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547940+0000","last_change":"2024-09-17T08:22:08.213925+0000","last_active":"2024-09-17T08:22:58.547940+0000","last_peered":"2024-09-17T08:22:58.547940+0000","last_clean":"2024-09-17T08:22:58.547940+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:22:58.547940+0000","last_undegraded":"2024-09-17T08:22:58.547940+0000","last_fullsized":"2024-09-17T08:22:58.547940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:08.213764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:22:08.213764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:08.214149+0000","scrub_duration":0.0019928789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":339,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624100+0000","last_change":"2024-09-17T08:21:10.994605+0000","last_active":"2024-09-17T08:22:58.624100+0000","last_peered":"2024-09-17T08:22:58.624100+0000","last_clean":"2024-09-17T08:22:58.624100+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:22:58.624100+0000","last_undegraded":"2024-09-17T08:22:58.624100+0000","last_fullsized":"2024-09-17T08:22:58.624100+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:21:10.993827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:10.993827+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010019035000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":293,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548373+0000","last_change":"2024-09-17T08:21:26.022703+0000","last_active":"2024-09-17T08:22:58.548373+0000","last_peered":"2024-09-17T08:22:58.548373+0000","last_clean":"2024-09-17T08:22:58.548373+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:22:58.548373+0000","last_undegraded":"2024-09-17T08:22:58.548373+0000","last_fullsized":"2024-09-17T08:22:58.548373+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:26.022562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.022562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081398640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624522+0000","last_change":"2024-09-17T08:22:34.633621+0000","last_active":"2024-09-17T08:22:58.624522+0000","last_peered":"2024-09-17T08:22:58.624522+0000","last_clean":"2024-09-17T08:22:58.624522+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:22:58.624522+0000","last_undegraded":"2024-09-17T08:22:58.624522+0000","last_fullsized":"2024-09-17T08:22:58.624522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:34.633510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:22:34.633510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:55.117384+0000","scrub_duration":0.0028104390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":406,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548424+0000","last_change":"2024-09-17T08:22:12.026146+0000","last_active":"2024-09-17T08:22:58.548424+0000","last_peered":"2024-09-17T08:22:58.548424+0000","last_clean":"2024-09-17T08:22:58.548424+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:22:58.548424+0000","last_undegraded":"2024-09-17T08:22:58.548424+0000","last_fullsized":"2024-09-17T08:22:58.548424+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:12.026000+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.026000+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:37.957272+0000","scrub_duration":0.0077245940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548312+0000","last_change":"2024-09-17T08:22:11.468233+0000","last_active":"2024-09-17T08:22:58.548312+0000","last_peered":"2024-09-17T08:22:58.548312+0000","last_clean":"2024-09-17T08:22:58.548312+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:22:58.548312+0000","last_undegraded":"2024-09-17T08:22:58.548312+0000","last_fullsized":"2024-09-17T08:22:58.548312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:11.468116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.468116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:33.231991+0000","scrub_duration":0.0051835620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":302,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548543+0000","last_change":"2024-09-17T08:22:32.148128+0000","last_active":"2024-09-17T08:22:58.548543+0000","last_peered":"2024-09-17T08:22:58.548543+0000","last_clean":"2024-09-17T08:22:58.548543+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:22:58.548543+0000","last_undegraded":"2024-09-17T08:22:58.548543+0000","last_fullsized":"2024-09-17T08:22:58.548543+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.147832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.147832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.754489+0000","scrub_duration":0.0067988659999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625994+0000","last_change":"2024-09-17T08:22:14.492261+0000","last_active":"2024-09-17T08:22:58.625994+0000","last_peered":"2024-09-17T08:22:58.625994+0000","last_clean":"2024-09-17T08:22:58.625994+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:22:58.625994+0000","last_undegraded":"2024-09-17T08:22:58.625994+0000","last_fullsized":"2024-09-17T08:22:58.625994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.492185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.492185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:43.260230+0000","scrub_duration":0.057522131999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":322,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548598+0000","last_change":"2024-09-17T08:21:34.973710+0000","last_active":"2024-09-17T08:22:58.548598+0000","last_peered":"2024-09-17T08:22:58.548598+0000","last_clean":"2024-09-17T08:22:58.548598+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:22:58.548598+0000","last_undegraded":"2024-09-17T08:22:58.548598+0000","last_fullsized":"2024-09-17T08:22:58.548598+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:21:34.973570+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.973570+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.059354883999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":315,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625037+0000","last_change":"2024-09-17T08:22:22.423974+0000","last_active":"2024-09-17T08:22:58.625037+0000","last_peered":"2024-09-17T08:22:58.625037+0000","last_clean":"2024-09-17T08:22:58.625037+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:22:58.625037+0000","last_undegraded":"2024-09-17T08:22:58.625037+0000","last_fullsized":"2024-09-17T08:22:58.625037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.423862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.807818+0000","scrub_duration":0.0032447660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548163+0000","last_change":"2024-09-17T08:21:24.523252+0000","last_active":"2024-09-17T08:22:58.548163+0000","last_peered":"2024-09-17T08:22:58.548163+0000","last_clean":"2024-09-17T08:22:58.548163+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:22:58.548163+0000","last_undegraded":"2024-09-17T08:22:58.548163+0000","last_fullsized":"2024-09-17T08:22:58.548163+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.523136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.523136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078907060000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":305,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625707+0000","last_change":"2024-09-17T08:21:20.504900+0000","last_active":"2024-09-17T08:22:58.625707+0000","last_peered":"2024-09-17T08:22:58.625707+0000","last_clean":"2024-09-17T08:22:58.625707+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:22:58.625707+0000","last_undegraded":"2024-09-17T08:22:58.625707+0000","last_fullsized":"2024-09-17T08:22:58.625707+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.504720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.504720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018477529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":317,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548688+0000","last_change":"2024-09-17T08:22:36.105717+0000","last_active":"2024-09-17T08:22:58.548688+0000","last_peered":"2024-09-17T08:22:58.548688+0000","last_clean":"2024-09-17T08:22:58.548688+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:22:58.548688+0000","last_undegraded":"2024-09-17T08:22:58.548688+0000","last_fullsized":"2024-09-17T08:22:58.548688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:22:36.105550+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:57.914458+0000","scrub_duration":0.012187885000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":189,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547919+0000","last_change":"2024-09-17T08:22:35.817757+0000","last_active":"2024-09-17T08:22:58.547919+0000","last_peered":"2024-09-17T08:22:58.547919+0000","last_clean":"2024-09-17T08:22:58.547919+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:22:58.547919+0000","last_undegraded":"2024-09-17T08:22:58.547919+0000","last_fullsized":"2024-09-17T08:22:58.547919+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:35.817629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.817629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.187574+0000","scrub_duration":0.0023115000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":302,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547416+0000","last_change":"2024-09-17T08:22:28.020605+0000","last_active":"2024-09-17T08:22:58.547416+0000","last_peered":"2024-09-17T08:22:58.547416+0000","last_clean":"2024-09-17T08:22:58.547416+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:22:58.547416+0000","last_undegraded":"2024-09-17T08:22:58.547416+0000","last_fullsized":"2024-09-17T08:22:58.547416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:28.019265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:28.019265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.020045+0000","scrub_duration":0.0075046389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"274'13898","reported_seq":15527,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.659933+0000","last_change":"2024-09-17T08:22:35.057102+0000","last_active":"2024-09-17T08:23:08.659933+0000","last_peered":"2024-09-17T08:23:08.659933+0000","last_clean":"2024-09-17T08:23:08.659933+0000","last_became_active":"2024-09-17T08:20:32.362328+0000","last_became_peered":"2024-09-17T08:20:32.362328+0000","last_unstale":"2024-09-17T08:23:08.659933+0000","last_undegraded":"2024-09-17T08:23:08.659933+0000","last_fullsized":"2024-09-17T08:23:08.659933+0000","mapping_epoch":230,"log_start":"262'8900","ondisk_log_start":"262'8900","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11650","last_scrub_stamp":"2024-09-17T08:22:35.056760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.056760+0000","objects_scrubbed":4535,"log_size":4998,"log_dups_size":0,"ondisk_log_size":4998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.046356+0000","scrub_duration":1.5582728380000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146669568,"num_objects":2238,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13898,"num_write_kb":516352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":47382528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":412,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549152+0000","last_change":"2024-09-17T08:22:11.061223+0000","last_active":"2024-09-17T08:22:58.549152+0000","last_peered":"2024-09-17T08:22:58.549152+0000","last_clean":"2024-09-17T08:22:58.549152+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:22:58.549152+0000","last_undegraded":"2024-09-17T08:22:58.549152+0000","last_fullsized":"2024-09-17T08:22:58.549152+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:22:11.061047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.061047+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:14.465518+0000","scrub_duration":0.0082017440000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625721+0000","last_change":"2024-09-17T08:21:29.435457+0000","last_active":"2024-09-17T08:22:58.625721+0000","last_peered":"2024-09-17T08:22:58.625721+0000","last_clean":"2024-09-17T08:22:58.625721+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:22:58.625721+0000","last_undegraded":"2024-09-17T08:22:58.625721+0000","last_fullsized":"2024-09-17T08:22:58.625721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:29.435329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.435329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017420199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548691+0000","last_change":"2024-09-17T08:20:16.153180+0000","last_active":"2024-09-17T08:22:58.548691+0000","last_peered":"2024-09-17T08:22:58.548691+0000","last_clean":"2024-09-17T08:22:58.548691+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:22:58.548691+0000","last_undegraded":"2024-09-17T08:22:58.548691+0000","last_fullsized":"2024-09-17T08:22:58.548691+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:16.153101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:16.153101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017057730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":258,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.653232+0000","last_change":"2024-09-17T08:21:15.739147+0000","last_active":"2024-09-17T08:22:58.653232+0000","last_peered":"2024-09-17T08:22:58.653232+0000","last_clean":"2024-09-17T08:22:58.653232+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:22:58.653232+0000","last_undegraded":"2024-09-17T08:22:58.653232+0000","last_fullsized":"2024-09-17T08:22:58.653232+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.739010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.739010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013099520999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"274'13912","reported_seq":14076,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.436506+0000","last_change":"2024-09-17T08:22:37.068629+0000","last_active":"2024-09-17T08:23:08.436506+0000","last_peered":"2024-09-17T08:23:08.436506+0000","last_clean":"2024-09-17T08:23:08.436506+0000","last_became_active":"2024-09-17T08:20:07.223417+0000","last_became_peered":"2024-09-17T08:20:07.223417+0000","last_unstale":"2024-09-17T08:23:08.436506+0000","last_undegraded":"2024-09-17T08:23:08.436506+0000","last_fullsized":"2024-09-17T08:23:08.436506+0000","mapping_epoch":221,"log_start":"262'9000","ondisk_log_start":"262'9000","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11785","last_scrub_stamp":"2024-09-17T08:22:37.068424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.068424+0000","objects_scrubbed":4475,"log_size":4912,"log_dups_size":0,"ondisk_log_size":4912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.205656+0000","scrub_duration":1.4410351079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151781376,"num_objects":2316,"num_object_clones":0,"num_object_copies":6948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13912,"num_write_kb":519296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":318,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548987+0000","last_change":"2024-09-17T08:22:29.100509+0000","last_active":"2024-09-17T08:22:58.548987+0000","last_peered":"2024-09-17T08:22:58.548987+0000","last_clean":"2024-09-17T08:22:58.548987+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:22:58.548987+0000","last_undegraded":"2024-09-17T08:22:58.548987+0000","last_fullsized":"2024-09-17T08:22:58.548987+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:29.100364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.100364+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.213754+0000","scrub_duration":0.023929505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549267+0000","last_change":"2024-09-17T08:22:24.597248+0000","last_active":"2024-09-17T08:22:58.549267+0000","last_peered":"2024-09-17T08:22:58.549267+0000","last_clean":"2024-09-17T08:22:58.549267+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:22:58.549267+0000","last_undegraded":"2024-09-17T08:22:58.549267+0000","last_fullsized":"2024-09-17T08:22:58.549267+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:24.597137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:22:24.597137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.974690+0000","scrub_duration":0.0075741749999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":395,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.078259+0000","last_change":"2024-09-17T08:21:11.769223+0000","last_active":"2024-09-17T08:23:01.078259+0000","last_peered":"2024-09-17T08:23:01.078259+0000","last_clean":"2024-09-17T08:23:01.078259+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:23:01.078259+0000","last_undegraded":"2024-09-17T08:23:01.078259+0000","last_fullsized":"2024-09-17T08:23:01.078259+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:21:11.769092+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:21:11.769092+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066652409999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":296,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549183+0000","last_change":"2024-09-17T08:21:10.972195+0000","last_active":"2024-09-17T08:22:58.549183+0000","last_peered":"2024-09-17T08:22:58.549183+0000","last_clean":"2024-09-17T08:22:58.549183+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:22:58.549183+0000","last_undegraded":"2024-09-17T08:22:58.549183+0000","last_fullsized":"2024-09-17T08:22:58.549183+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:10.972020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:21:10.972020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011239071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"274'14508","reported_seq":16085,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.478489+0000","last_change":"2024-09-17T08:20:32.368171+0000","last_active":"2024-09-17T08:23:08.478489+0000","last_peered":"2024-09-17T08:23:08.478489+0000","last_clean":"2024-09-17T08:23:08.478489+0000","last_became_active":"2024-09-17T08:20:32.365928+0000","last_became_peered":"2024-09-17T08:20:32.365928+0000","last_unstale":"2024-09-17T08:23:08.478489+0000","last_undegraded":"2024-09-17T08:23:08.478489+0000","last_fullsized":"2024-09-17T08:23:08.478489+0000","mapping_epoch":230,"log_start":"262'9600","ondisk_log_start":"262'9600","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":0,"log_size":4908,"log_dups_size":0,"ondisk_log_size":4908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154271744,"num_objects":2354,"num_object_clones":0,"num_object_copies":7062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14508,"num_write_kb":539584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":49152000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":160,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.653017+0000","last_change":"2024-09-17T08:22:27.019007+0000","last_active":"2024-09-17T08:22:58.653017+0000","last_peered":"2024-09-17T08:22:58.653017+0000","last_clean":"2024-09-17T08:22:58.653017+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:22:58.653017+0000","last_undegraded":"2024-09-17T08:22:58.653017+0000","last_fullsized":"2024-09-17T08:22:58.653017+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:27.018869+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.018869+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.142241+0000","scrub_duration":0.0034750470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548231+0000","last_change":"2024-09-17T08:22:05.559395+0000","last_active":"2024-09-17T08:22:58.548231+0000","last_peered":"2024-09-17T08:22:58.548231+0000","last_clean":"2024-09-17T08:22:58.548231+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:22:58.548231+0000","last_undegraded":"2024-09-17T08:22:58.548231+0000","last_fullsized":"2024-09-17T08:22:58.548231+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:05.559268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:05.559268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.693611+0000","scrub_duration":0.010154139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":442,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.076611+0000","last_change":"2024-09-17T08:21:35.884656+0000","last_active":"2024-09-17T08:23:01.076611+0000","last_peered":"2024-09-17T08:23:01.076611+0000","last_clean":"2024-09-17T08:23:01.076611+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:23:01.076611+0000","last_undegraded":"2024-09-17T08:23:01.076611+0000","last_fullsized":"2024-09-17T08:23:01.076611+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:21:35.884516+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086332900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548544+0000","last_change":"2024-09-17T08:22:20.469387+0000","last_active":"2024-09-17T08:22:58.548544+0000","last_peered":"2024-09-17T08:22:58.548544+0000","last_clean":"2024-09-17T08:22:58.548544+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:22:58.548544+0000","last_undegraded":"2024-09-17T08:22:58.548544+0000","last_fullsized":"2024-09-17T08:22:58.548544+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.469252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.469252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:28.979091+0000","scrub_duration":0.0085710660000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"274'14389","reported_seq":14548,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:07.702761+0000","last_change":"2024-09-17T08:22:42.795845+0000","last_active":"2024-09-17T08:23:07.702761+0000","last_peered":"2024-09-17T08:23:07.702761+0000","last_clean":"2024-09-17T08:23:07.702761+0000","last_became_active":"2024-09-17T08:20:07.153440+0000","last_became_peered":"2024-09-17T08:20:07.153440+0000","last_unstale":"2024-09-17T08:23:07.702761+0000","last_undegraded":"2024-09-17T08:23:07.702761+0000","last_fullsized":"2024-09-17T08:23:07.702761+0000","mapping_epoch":221,"log_start":"263'9700","ondisk_log_start":"263'9700","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12590","last_scrub_stamp":"2024-09-17T08:22:42.795634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:42.795634+0000","objects_scrubbed":4222,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.287069+0000","scrub_duration":1.2865534140000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156434432,"num_objects":2387,"num_object_clones":0,"num_object_copies":7161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14389,"num_write_kb":536832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":356,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549263+0000","last_change":"2024-09-17T08:22:14.033870+0000","last_active":"2024-09-17T08:22:58.549263+0000","last_peered":"2024-09-17T08:22:58.549263+0000","last_clean":"2024-09-17T08:22:58.549263+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:22:58.549263+0000","last_undegraded":"2024-09-17T08:22:58.549263+0000","last_fullsized":"2024-09-17T08:22:58.549263+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:22:14.033529+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.033529+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.942917+0000","scrub_duration":0.0038854010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":298,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624620+0000","last_change":"2024-09-17T08:21:23.559857+0000","last_active":"2024-09-17T08:22:58.624620+0000","last_peered":"2024-09-17T08:22:58.624620+0000","last_clean":"2024-09-17T08:22:58.624620+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:22:58.624620+0000","last_undegraded":"2024-09-17T08:22:58.624620+0000","last_fullsized":"2024-09-17T08:22:58.624620+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:23.559763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:23.559763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052019450000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":179,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549238+0000","last_change":"2024-09-17T08:22:30.071610+0000","last_active":"2024-09-17T08:22:58.549238+0000","last_peered":"2024-09-17T08:22:58.549238+0000","last_clean":"2024-09-17T08:22:58.549238+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:22:58.549238+0000","last_undegraded":"2024-09-17T08:22:58.549238+0000","last_fullsized":"2024-09-17T08:22:58.549238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.071480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.161204+0000","scrub_duration":0.0047153050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":262,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624342+0000","last_change":"2024-09-17T08:20:13.845069+0000","last_active":"2024-09-17T08:22:58.624342+0000","last_peered":"2024-09-17T08:22:58.624342+0000","last_clean":"2024-09-17T08:22:58.624342+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:22:58.624342+0000","last_undegraded":"2024-09-17T08:22:58.624342+0000","last_fullsized":"2024-09-17T08:22:58.624342+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:13.844986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:13.844986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001309288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"274'14151","reported_seq":14325,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.485255+0000","last_change":"2024-09-17T08:22:29.304499+0000","last_active":"2024-09-17T08:23:08.485255+0000","last_peered":"2024-09-17T08:23:08.485255+0000","last_clean":"2024-09-17T08:23:08.485255+0000","last_became_active":"2024-09-17T08:20:07.153490+0000","last_became_peered":"2024-09-17T08:20:07.153490+0000","last_unstale":"2024-09-17T08:23:08.485255+0000","last_undegraded":"2024-09-17T08:23:08.485255+0000","last_fullsized":"2024-09-17T08:23:08.485255+0000","mapping_epoch":221,"log_start":"262'9300","ondisk_log_start":"262'9300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11539","last_scrub_stamp":"2024-09-17T08:22:29.304266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.304266+0000","objects_scrubbed":4996,"log_size":4851,"log_dups_size":0,"ondisk_log_size":4851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:40.689344+0000","scrub_duration":1.700727825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152109056,"num_objects":2321,"num_object_clones":0,"num_object_copies":6963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14151,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":320,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548516+0000","last_change":"2024-09-17T08:22:20.997014+0000","last_active":"2024-09-17T08:22:58.548516+0000","last_peered":"2024-09-17T08:22:58.548516+0000","last_clean":"2024-09-17T08:22:58.548516+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:22:58.548516+0000","last_undegraded":"2024-09-17T08:22:58.548516+0000","last_fullsized":"2024-09-17T08:22:58.548516+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:22:20.996746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.996746+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.038376+0000","scrub_duration":0.0061057289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":244,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625214+0000","last_change":"2024-09-17T08:22:20.368530+0000","last_active":"2024-09-17T08:22:58.625214+0000","last_peered":"2024-09-17T08:22:58.625214+0000","last_clean":"2024-09-17T08:22:58.625214+0000","last_became_active":"2024-09-17T08:20:23.487784+0000","last_became_peered":"2024-09-17T08:20:23.487784+0000","last_unstale":"2024-09-17T08:22:58.625214+0000","last_undegraded":"2024-09-17T08:22:58.625214+0000","last_fullsized":"2024-09-17T08:22:58.625214+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.368411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.368411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:27.290703+0000","scrub_duration":0.0016290580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":475,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.992039+0000","last_change":"2024-09-17T08:22:07.339466+0000","last_active":"2024-09-17T08:23:02.992039+0000","last_peered":"2024-09-17T08:23:02.992039+0000","last_clean":"2024-09-17T08:23:02.992039+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:23:02.992039+0000","last_undegraded":"2024-09-17T08:23:02.992039+0000","last_fullsized":"2024-09-17T08:23:02.992039+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:22:07.339335+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:22:07.339335+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.917029+0000","scrub_duration":0.011996602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.626014+0000","last_change":"2024-09-17T08:21:12.324228+0000","last_active":"2024-09-17T08:22:58.626014+0000","last_peered":"2024-09-17T08:22:58.626014+0000","last_clean":"2024-09-17T08:22:58.626014+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:22:58.626014+0000","last_undegraded":"2024-09-17T08:22:58.626014+0000","last_fullsized":"2024-09-17T08:22:58.626014+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:12.323711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:47.964546+0000","last_clean_scrub_stamp":"2024-09-17T08:21:12.323711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019279740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"274'13968","reported_seq":14125,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:07.663093+0000","last_change":"2024-09-17T08:22:39.712172+0000","last_active":"2024-09-17T08:23:07.663093+0000","last_peered":"2024-09-17T08:23:07.663093+0000","last_clean":"2024-09-17T08:23:07.663093+0000","last_became_active":"2024-09-17T08:20:07.346251+0000","last_became_peered":"2024-09-17T08:20:07.346251+0000","last_unstale":"2024-09-17T08:23:07.663093+0000","last_undegraded":"2024-09-17T08:23:07.663093+0000","last_fullsized":"2024-09-17T08:23:07.663093+0000","mapping_epoch":221,"log_start":"262'9300","ondisk_log_start":"262'9300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12088","last_scrub_stamp":"2024-09-17T08:22:39.711942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.711942+0000","objects_scrubbed":4288,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.633790+0000","scrub_duration":1.2802566520000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155254808,"num_objects":2370,"num_object_clones":0,"num_object_copies":7110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13968,"num_write_kb":522753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":328,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.623979+0000","last_change":"2024-09-17T08:21:26.037150+0000","last_active":"2024-09-17T08:22:58.623979+0000","last_peered":"2024-09-17T08:22:58.623979+0000","last_clean":"2024-09-17T08:22:58.623979+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:22:58.623979+0000","last_undegraded":"2024-09-17T08:22:58.623979+0000","last_fullsized":"2024-09-17T08:22:58.623979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:21:26.036963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.036963+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038814650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549200+0000","last_change":"2024-09-17T08:22:15.514138+0000","last_active":"2024-09-17T08:22:58.549200+0000","last_peered":"2024-09-17T08:22:58.549200+0000","last_clean":"2024-09-17T08:22:58.549200+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:22:58.549200+0000","last_undegraded":"2024-09-17T08:22:58.549200+0000","last_fullsized":"2024-09-17T08:22:58.549200+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:15.514031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.514031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.523751+0000","scrub_duration":0.0041206280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":276,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548435+0000","last_change":"2024-09-17T08:21:32.922504+0000","last_active":"2024-09-17T08:22:58.548435+0000","last_peered":"2024-09-17T08:22:58.548435+0000","last_clean":"2024-09-17T08:22:58.548435+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:22:58.548435+0000","last_undegraded":"2024-09-17T08:22:58.548435+0000","last_fullsized":"2024-09-17T08:22:58.548435+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:32.922405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:21:32.922405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015573643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":289,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548639+0000","last_change":"2024-09-17T08:20:31.986066+0000","last_active":"2024-09-17T08:22:58.548639+0000","last_peered":"2024-09-17T08:22:58.548639+0000","last_clean":"2024-09-17T08:22:58.548639+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:22:58.548639+0000","last_undegraded":"2024-09-17T08:22:58.548639+0000","last_fullsized":"2024-09-17T08:22:58.548639+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:31.985972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:31.985972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018779650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"274'17899","reported_seq":18301,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:07.692032+0000","last_change":"2024-09-17T08:22:54.434167+0000","last_active":"2024-09-17T08:23:07.692032+0000","last_peered":"2024-09-17T08:23:07.692032+0000","last_clean":"2024-09-17T08:23:07.692032+0000","last_became_active":"2024-09-17T08:20:07.153814+0000","last_became_peered":"2024-09-17T08:20:07.153814+0000","last_unstale":"2024-09-17T08:23:07.692032+0000","last_undegraded":"2024-09-17T08:23:07.692032+0000","last_fullsized":"2024-09-17T08:23:07.692032+0000","mapping_epoch":221,"log_start":"268'13203","ondisk_log_start":"268'13203","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'16933","last_scrub_stamp":"2024-09-17T08:22:54.433940+0000","last_deep_scrub":"271'16933","last_deep_scrub_stamp":"2024-09-17T08:22:54.433940+0000","last_clean_scrub_stamp":"2024-09-17T08:22:54.433940+0000","objects_scrubbed":3978,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":23,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:02.333157+0000","scrub_duration":22.034200755000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154468352,"num_objects":2357,"num_object_clones":0,"num_object_copies":7071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13921,"num_write_kb":520896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":340,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548018+0000","last_change":"2024-09-17T08:21:26.896828+0000","last_active":"2024-09-17T08:22:58.548018+0000","last_peered":"2024-09-17T08:22:58.548018+0000","last_clean":"2024-09-17T08:22:58.548018+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:22:58.548018+0000","last_undegraded":"2024-09-17T08:22:58.548018+0000","last_fullsized":"2024-09-17T08:22:58.548018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:21:26.896576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.896576+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056806000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":296,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.626073+0000","last_change":"2024-09-17T08:21:27.406621+0000","last_active":"2024-09-17T08:22:58.626073+0000","last_peered":"2024-09-17T08:22:58.626073+0000","last_clean":"2024-09-17T08:22:58.626073+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:22:58.626073+0000","last_undegraded":"2024-09-17T08:22:58.626073+0000","last_fullsized":"2024-09-17T08:22:58.626073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:21:27.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.406373+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043336249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":242,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547436+0000","last_change":"2024-09-17T08:22:27.034592+0000","last_active":"2024-09-17T08:22:58.547436+0000","last_peered":"2024-09-17T08:22:58.547436+0000","last_clean":"2024-09-17T08:22:58.547436+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:22:58.547436+0000","last_undegraded":"2024-09-17T08:22:58.547436+0000","last_fullsized":"2024-09-17T08:22:58.547436+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.034476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.034476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.281308+0000","scrub_duration":0.0053609369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":184,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548390+0000","last_change":"2024-09-17T08:21:13.406948+0000","last_active":"2024-09-17T08:22:58.548390+0000","last_peered":"2024-09-17T08:22:58.548390+0000","last_clean":"2024-09-17T08:22:58.548390+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:22:58.548390+0000","last_undegraded":"2024-09-17T08:22:58.548390+0000","last_fullsized":"2024-09-17T08:22:58.548390+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:13.406818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:13.406818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039848110000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"274'14154","reported_seq":14314,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.637589+0000","last_change":"2024-09-17T08:22:40.874195+0000","last_active":"2024-09-17T08:23:08.637589+0000","last_peered":"2024-09-17T08:23:08.637589+0000","last_clean":"2024-09-17T08:23:08.637589+0000","last_became_active":"2024-09-17T08:20:07.346503+0000","last_became_peered":"2024-09-17T08:20:07.346503+0000","last_unstale":"2024-09-17T08:23:08.637589+0000","last_undegraded":"2024-09-17T08:23:08.637589+0000","last_fullsized":"2024-09-17T08:23:08.637589+0000","mapping_epoch":221,"log_start":"262'9200","ondisk_log_start":"262'9200","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12220","last_scrub_stamp":"2024-09-17T08:22:40.873622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.873622+0000","objects_scrubbed":4349,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.542314+0000","scrub_duration":1.240716989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155451392,"num_objects":2372,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14154,"num_write_kb":528832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":201,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549027+0000","last_change":"2024-09-17T08:21:20.455004+0000","last_active":"2024-09-17T08:22:58.549027+0000","last_peered":"2024-09-17T08:22:58.549027+0000","last_clean":"2024-09-17T08:22:58.549027+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:22:58.549027+0000","last_undegraded":"2024-09-17T08:22:58.549027+0000","last_fullsized":"2024-09-17T08:22:58.549027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:21:20.454861+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.454861+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043903680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548951+0000","last_change":"2024-09-17T08:22:17.515205+0000","last_active":"2024-09-17T08:22:58.548951+0000","last_peered":"2024-09-17T08:22:58.548951+0000","last_clean":"2024-09-17T08:22:58.548951+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:22:58.548951+0000","last_undegraded":"2024-09-17T08:22:58.548951+0000","last_fullsized":"2024-09-17T08:22:58.548951+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:17.515085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.515085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.852597+0000","scrub_duration":0.001995199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":455,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.984112+0000","last_change":"2024-09-17T08:21:29.854449+0000","last_active":"2024-09-17T08:23:02.984112+0000","last_peered":"2024-09-17T08:23:02.984112+0000","last_clean":"2024-09-17T08:23:02.984112+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:23:02.984112+0000","last_undegraded":"2024-09-17T08:23:02.984112+0000","last_fullsized":"2024-09-17T08:23:02.984112+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:21:29.853904+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:19:34.379477+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.853904+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059980889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547296+0000","last_change":"2024-09-17T08:21:21.799057+0000","last_active":"2024-09-17T08:22:58.547296+0000","last_peered":"2024-09-17T08:22:58.547296+0000","last_clean":"2024-09-17T08:22:58.547296+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:22:58.547296+0000","last_undegraded":"2024-09-17T08:22:58.547296+0000","last_fullsized":"2024-09-17T08:22:58.547296+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.798694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.798694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002756157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"274'14210","reported_seq":14374,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.484844+0000","last_change":"2024-09-17T08:22:37.976919+0000","last_active":"2024-09-17T08:23:08.484844+0000","last_peered":"2024-09-17T08:23:08.484844+0000","last_clean":"2024-09-17T08:23:08.484844+0000","last_became_active":"2024-09-17T08:20:07.345130+0000","last_became_peered":"2024-09-17T08:20:07.345130+0000","last_unstale":"2024-09-17T08:23:08.484844+0000","last_undegraded":"2024-09-17T08:23:08.484844+0000","last_fullsized":"2024-09-17T08:23:08.484844+0000","mapping_epoch":221,"log_start":"258'8300","ondisk_log_start":"258'8300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12113","last_scrub_stamp":"2024-09-17T08:22:37.976676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.976676+0000","objects_scrubbed":4399,"log_size":5910,"log_dups_size":0,"ondisk_log_size":5910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.418342+0000","scrub_duration":1.1806302419999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242432,"num_objects":2262,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14210,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":322,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.985420+0000","last_change":"2024-09-17T08:21:14.458315+0000","last_active":"2024-09-17T08:23:02.985420+0000","last_peered":"2024-09-17T08:23:02.985420+0000","last_clean":"2024-09-17T08:23:02.985420+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:23:02.985420+0000","last_undegraded":"2024-09-17T08:23:02.985420+0000","last_fullsized":"2024-09-17T08:23:02.985420+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:21:14.458167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.458167+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024233291000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549101+0000","last_change":"2024-09-17T08:20:22.073899+0000","last_active":"2024-09-17T08:22:58.549101+0000","last_peered":"2024-09-17T08:22:58.549101+0000","last_clean":"2024-09-17T08:22:58.549101+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:22:58.549101+0000","last_undegraded":"2024-09-17T08:22:58.549101+0000","last_fullsized":"2024-09-17T08:22:58.549101+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:22.073341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:20:22.073341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048532590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":305,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548308+0000","last_change":"2024-09-17T08:22:18.168279+0000","last_active":"2024-09-17T08:22:58.548308+0000","last_peered":"2024-09-17T08:22:58.548308+0000","last_clean":"2024-09-17T08:22:58.548308+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:22:58.548308+0000","last_undegraded":"2024-09-17T08:22:58.548308+0000","last_fullsized":"2024-09-17T08:22:58.548308+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.168157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.168157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.314142+0000","scrub_duration":0.0039078109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":274,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.082243+0000","last_change":"2024-09-17T08:21:13.022290+0000","last_active":"2024-09-17T08:23:01.082243+0000","last_peered":"2024-09-17T08:23:01.082243+0000","last_clean":"2024-09-17T08:23:01.082243+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:23:01.082243+0000","last_undegraded":"2024-09-17T08:23:01.082243+0000","last_fullsized":"2024-09-17T08:23:01.082243+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:13.022174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:13.022174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005631545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"274'14006","reported_seq":14180,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.653107+0000","last_change":"2024-09-17T08:22:31.345117+0000","last_active":"2024-09-17T08:23:08.653107+0000","last_peered":"2024-09-17T08:23:08.653107+0000","last_clean":"2024-09-17T08:23:08.653107+0000","last_became_active":"2024-09-17T08:20:07.347116+0000","last_became_peered":"2024-09-17T08:20:07.347116+0000","last_unstale":"2024-09-17T08:23:08.653107+0000","last_undegraded":"2024-09-17T08:23:08.653107+0000","last_fullsized":"2024-09-17T08:23:08.653107+0000","mapping_epoch":221,"log_start":"261'9000","ondisk_log_start":"261'9000","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11550","last_scrub_stamp":"2024-09-17T08:22:31.344820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.344820+0000","objects_scrubbed":4856,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:37.031561+0000","scrub_duration":1.7801603639999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154271744,"num_objects":2354,"num_object_clones":0,"num_object_copies":7062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":187,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548891+0000","last_change":"2024-09-17T08:22:08.570430+0000","last_active":"2024-09-17T08:22:58.548891+0000","last_peered":"2024-09-17T08:22:58.548891+0000","last_clean":"2024-09-17T08:22:58.548891+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:22:58.548891+0000","last_undegraded":"2024-09-17T08:22:58.548891+0000","last_fullsized":"2024-09-17T08:22:58.548891+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:22:08.569507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:08.569507+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:24.727398+0000","scrub_duration":0.010760135000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":645,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:07.025634+0000","last_change":"2024-09-17T08:21:21.456425+0000","last_active":"2024-09-17T08:23:07.025634+0000","last_peered":"2024-09-17T08:23:07.025634+0000","last_clean":"2024-09-17T08:23:07.025634+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:23:07.025634+0000","last_undegraded":"2024-09-17T08:23:07.025634+0000","last_fullsized":"2024-09-17T08:23:07.025634+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:21:21.455503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.455503+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015087046,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":456,"num_read_kb":451,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652559+0000","last_change":"2024-09-17T08:21:21.794874+0000","last_active":"2024-09-17T08:22:58.652559+0000","last_peered":"2024-09-17T08:22:58.652559+0000","last_clean":"2024-09-17T08:22:58.652559+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:22:58.652559+0000","last_undegraded":"2024-09-17T08:22:58.652559+0000","last_fullsized":"2024-09-17T08:22:58.652559+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.794753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.794753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020168569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":272,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.657359+0000","last_change":"2024-09-17T08:22:09.870585+0000","last_active":"2024-09-17T08:22:58.657359+0000","last_peered":"2024-09-17T08:22:58.657359+0000","last_clean":"2024-09-17T08:22:58.657359+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:22:58.657359+0000","last_undegraded":"2024-09-17T08:22:58.657359+0000","last_fullsized":"2024-09-17T08:22:58.657359+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:09.870413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.870413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.195536+0000","scrub_duration":0.0021174589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":296,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625630+0000","last_change":"2024-09-17T08:22:40.481597+0000","last_active":"2024-09-17T08:22:58.625630+0000","last_peered":"2024-09-17T08:22:58.625630+0000","last_clean":"2024-09-17T08:22:58.625630+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:22:58.625630+0000","last_undegraded":"2024-09-17T08:22:58.625630+0000","last_fullsized":"2024-09-17T08:22:58.625630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.481384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.481384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.087943+0000","scrub_duration":0.0030329860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"274'13967","reported_seq":14117,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.477669+0000","last_change":"2024-09-17T08:22:43.087899+0000","last_active":"2024-09-17T08:23:08.477669+0000","last_peered":"2024-09-17T08:23:08.477669+0000","last_clean":"2024-09-17T08:23:08.477669+0000","last_became_active":"2024-09-17T08:20:07.346820+0000","last_became_peered":"2024-09-17T08:20:07.346820+0000","last_unstale":"2024-09-17T08:23:08.477669+0000","last_undegraded":"2024-09-17T08:23:08.477669+0000","last_fullsized":"2024-09-17T08:23:08.477669+0000","mapping_epoch":221,"log_start":"257'8000","ondisk_log_start":"257'8000","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12231","last_scrub_stamp":"2024-09-17T08:22:43.087628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:43.087628+0000","objects_scrubbed":4033,"log_size":5967,"log_dups_size":0,"ondisk_log_size":5967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:44.864552+0000","scrub_duration":1.249210038,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147783680,"num_objects":2255,"num_object_clones":0,"num_object_copies":6765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":396,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548128+0000","last_change":"2024-09-17T08:20:23.574560+0000","last_active":"2024-09-17T08:22:58.548128+0000","last_peered":"2024-09-17T08:22:58.548128+0000","last_clean":"2024-09-17T08:22:58.548128+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:22:58.548128+0000","last_undegraded":"2024-09-17T08:22:58.548128+0000","last_fullsized":"2024-09-17T08:22:58.548128+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:20:23.574471+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.574471+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":298,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547760+0000","last_change":"2024-09-17T08:20:14.498127+0000","last_active":"2024-09-17T08:22:58.547760+0000","last_peered":"2024-09-17T08:22:58.547760+0000","last_clean":"2024-09-17T08:22:58.547760+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:22:58.547760+0000","last_undegraded":"2024-09-17T08:22:58.547760+0000","last_fullsized":"2024-09-17T08:22:58.547760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:20:14.498043+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:20:14.498043+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017037160000000001,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":309,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624686+0000","last_change":"2024-09-17T08:21:33.434991+0000","last_active":"2024-09-17T08:22:58.624686+0000","last_peered":"2024-09-17T08:22:58.624686+0000","last_clean":"2024-09-17T08:22:58.624686+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:22:58.624686+0000","last_undegraded":"2024-09-17T08:22:58.624686+0000","last_fullsized":"2024-09-17T08:22:58.624686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.434904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.434904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037511929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548978+0000","last_change":"2024-09-17T08:20:23.088911+0000","last_active":"2024-09-17T08:22:58.548978+0000","last_peered":"2024-09-17T08:22:58.548978+0000","last_clean":"2024-09-17T08:22:58.548978+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:22:58.548978+0000","last_undegraded":"2024-09-17T08:22:58.548978+0000","last_fullsized":"2024-09-17T08:22:58.548978+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:23.088809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.088809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015125119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548770+0000","last_change":"2024-09-17T08:21:34.401415+0000","last_active":"2024-09-17T08:22:58.548770+0000","last_peered":"2024-09-17T08:22:58.548770+0000","last_clean":"2024-09-17T08:22:58.548770+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:22:58.548770+0000","last_undegraded":"2024-09-17T08:22:58.548770+0000","last_fullsized":"2024-09-17T08:22:58.548770+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:34.401099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.401099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010133837999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"274'14093","reported_seq":14265,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.412801+0000","last_change":"2024-09-17T08:22:32.253807+0000","last_active":"2024-09-17T08:23:08.412801+0000","last_peered":"2024-09-17T08:23:08.412801+0000","last_clean":"2024-09-17T08:23:08.412801+0000","last_became_active":"2024-09-17T08:20:07.345502+0000","last_became_peered":"2024-09-17T08:20:07.345502+0000","last_unstale":"2024-09-17T08:23:08.412801+0000","last_undegraded":"2024-09-17T08:23:08.412801+0000","last_fullsized":"2024-09-17T08:23:08.412801+0000","mapping_epoch":221,"log_start":"262'9200","ondisk_log_start":"262'9200","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11656","last_scrub_stamp":"2024-09-17T08:22:32.253689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.253689+0000","objects_scrubbed":4756,"log_size":4893,"log_dups_size":0,"ondisk_log_size":4893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.659626+0000","scrub_duration":1.6774553399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147259392,"num_objects":2247,"num_object_clones":0,"num_object_copies":6741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14093,"num_write_kb":522880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549068+0000","last_change":"2024-09-17T08:20:20.963771+0000","last_active":"2024-09-17T08:22:58.549068+0000","last_peered":"2024-09-17T08:22:58.549068+0000","last_clean":"2024-09-17T08:22:58.549068+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:22:58.549068+0000","last_undegraded":"2024-09-17T08:22:58.549068+0000","last_fullsized":"2024-09-17T08:22:58.549068+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:20:20.963676+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:20:20.963676+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042143700000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":287,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624297+0000","last_change":"2024-09-17T08:21:34.970246+0000","last_active":"2024-09-17T08:22:58.624297+0000","last_peered":"2024-09-17T08:22:58.624297+0000","last_clean":"2024-09-17T08:22:58.624297+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:22:58.624297+0000","last_undegraded":"2024-09-17T08:22:58.624297+0000","last_fullsized":"2024-09-17T08:22:58.624297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:21:34.969971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.969971+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041400559999999996,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":162,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652880+0000","last_change":"2024-09-17T08:22:14.962129+0000","last_active":"2024-09-17T08:22:58.652880+0000","last_peered":"2024-09-17T08:22:58.652880+0000","last_clean":"2024-09-17T08:22:58.652880+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:22:58.652880+0000","last_undegraded":"2024-09-17T08:22:58.652880+0000","last_fullsized":"2024-09-17T08:22:58.652880+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.962019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.962019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.750253+0000","scrub_duration":0.0046754020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":552,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624849+0000","last_change":"2024-09-17T08:22:13.427742+0000","last_active":"2024-09-17T08:22:58.624849+0000","last_peered":"2024-09-17T08:22:58.624849+0000","last_clean":"2024-09-17T08:22:58.624849+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:22:58.624849+0000","last_undegraded":"2024-09-17T08:22:58.624849+0000","last_fullsized":"2024-09-17T08:22:58.624849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:22:13.427622+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_clean_scrub_stamp":"2024-09-17T08:22:13.427622+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:14.950844+0000","scrub_duration":0.0033528830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":291,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625481+0000","last_change":"2024-09-17T08:21:15.369223+0000","last_active":"2024-09-17T08:22:58.625481+0000","last_peered":"2024-09-17T08:22:58.625481+0000","last_clean":"2024-09-17T08:22:58.625481+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:22:58.625481+0000","last_undegraded":"2024-09-17T08:22:58.625481+0000","last_fullsized":"2024-09-17T08:22:58.625481+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.368937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.368937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021146210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":178,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624499+0000","last_change":"2024-09-17T08:21:30.014753+0000","last_active":"2024-09-17T08:22:58.624499+0000","last_peered":"2024-09-17T08:22:58.624499+0000","last_clean":"2024-09-17T08:22:58.624499+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:22:58.624499+0000","last_undegraded":"2024-09-17T08:22:58.624499+0000","last_fullsized":"2024-09-17T08:22:58.624499+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:30.013651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.013651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012423214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.7","version":"274'14191","reported_seq":14386,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:07.706499+0000","last_change":"2024-09-17T08:22:17.960994+0000","last_active":"2024-09-17T08:23:07.706499+0000","last_peered":"2024-09-17T08:23:07.706499+0000","last_clean":"2024-09-17T08:23:07.706499+0000","last_became_active":"2024-09-17T08:20:07.345480+0000","last_became_peered":"2024-09-17T08:20:07.345480+0000","last_unstale":"2024-09-17T08:23:07.706499+0000","last_undegraded":"2024-09-17T08:23:07.706499+0000","last_fullsized":"2024-09-17T08:23:07.706499+0000","mapping_epoch":221,"log_start":"263'9500","ondisk_log_start":"263'9500","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'10727","last_scrub_stamp":"2024-09-17T08:22:17.960812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.960812+0000","objects_scrubbed":5910,"log_size":4691,"log_dups_size":0,"ondisk_log_size":4691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.367005+0000","scrub_duration":1.5674201079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156565504,"num_objects":2389,"num_object_clones":0,"num_object_copies":7167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14191,"num_write_kb":530560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":211,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624372+0000","last_change":"2024-09-17T08:22:25.349111+0000","last_active":"2024-09-17T08:22:58.624372+0000","last_peered":"2024-09-17T08:22:58.624372+0000","last_clean":"2024-09-17T08:22:58.624372+0000","last_became_active":"2024-09-17T08:20:02.496306+0000","last_became_peered":"2024-09-17T08:20:02.496306+0000","last_unstale":"2024-09-17T08:22:58.624372+0000","last_undegraded":"2024-09-17T08:22:58.624372+0000","last_fullsized":"2024-09-17T08:22:58.624372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:22:25.348962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.348962+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.315979+0000","scrub_duration":0.010428712999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":253,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652997+0000","last_change":"2024-09-17T08:21:14.706611+0000","last_active":"2024-09-17T08:22:58.652997+0000","last_peered":"2024-09-17T08:22:58.652997+0000","last_clean":"2024-09-17T08:22:58.652997+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:22:58.652997+0000","last_undegraded":"2024-09-17T08:22:58.652997+0000","last_fullsized":"2024-09-17T08:22:58.652997+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:14.706481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.706481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045730429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":259,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652594+0000","last_change":"2024-09-17T08:21:09.680879+0000","last_active":"2024-09-17T08:22:58.652594+0000","last_peered":"2024-09-17T08:22:58.652594+0000","last_clean":"2024-09-17T08:22:58.652594+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:22:58.652594+0000","last_undegraded":"2024-09-17T08:22:58.652594+0000","last_fullsized":"2024-09-17T08:22:58.652594+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:09.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:21:09.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001955245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":289,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.079320+0000","last_change":"2024-09-17T08:21:18.452335+0000","last_active":"2024-09-17T08:23:01.079320+0000","last_peered":"2024-09-17T08:23:01.079320+0000","last_clean":"2024-09-17T08:23:01.079320+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:23:01.079320+0000","last_undegraded":"2024-09-17T08:23:01.079320+0000","last_fullsized":"2024-09-17T08:23:01.079320+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.452221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.452221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042305550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"274'13669","reported_seq":15240,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:06.255330+0000","last_change":"2024-09-17T08:22:43.617938+0000","last_active":"2024-09-17T08:23:06.255330+0000","last_peered":"2024-09-17T08:23:06.255330+0000","last_clean":"2024-09-17T08:23:06.255330+0000","last_became_active":"2024-09-17T08:20:33.313140+0000","last_became_peered":"2024-09-17T08:20:33.313140+0000","last_unstale":"2024-09-17T08:23:06.255330+0000","last_undegraded":"2024-09-17T08:23:06.255330+0000","last_fullsized":"2024-09-17T08:23:06.255330+0000","mapping_epoch":231,"log_start":"251'6800","ondisk_log_start":"251'6800","created":221,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":2949,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161021952,"num_objects":2457,"num_object_clones":0,"num_object_copies":7371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13669,"num_write_kb":516032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":726,"num_bytes_recovered":47579136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":351,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547815+0000","last_change":"2024-09-17T08:22:29.010241+0000","last_active":"2024-09-17T08:22:58.547815+0000","last_peered":"2024-09-17T08:22:58.547815+0000","last_clean":"2024-09-17T08:22:58.547815+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:22:58.547815+0000","last_undegraded":"2024-09-17T08:22:58.547815+0000","last_fullsized":"2024-09-17T08:22:58.547815+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:22:29.010088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.010088+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.705148+0000","scrub_duration":0.0044362949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":180,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548387+0000","last_change":"2024-09-17T08:22:40.621682+0000","last_active":"2024-09-17T08:22:58.548387+0000","last_peered":"2024-09-17T08:22:58.548387+0000","last_clean":"2024-09-17T08:22:58.548387+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:22:58.548387+0000","last_undegraded":"2024-09-17T08:22:58.548387+0000","last_fullsized":"2024-09-17T08:22:58.548387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.621579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.621579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.471410+0000","scrub_duration":0.0048029379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":273,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.658441+0000","last_change":"2024-09-17T08:22:30.062596+0000","last_active":"2024-09-17T08:22:58.658441+0000","last_peered":"2024-09-17T08:22:58.658441+0000","last_clean":"2024-09-17T08:22:58.658441+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:22:58.658441+0000","last_undegraded":"2024-09-17T08:22:58.658441+0000","last_fullsized":"2024-09-17T08:22:58.658441+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.062098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.062098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.782037+0000","scrub_duration":0.0090326120000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":289,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.088514+0000","last_change":"2024-09-17T08:20:24.572671+0000","last_active":"2024-09-17T08:23:01.088514+0000","last_peered":"2024-09-17T08:23:01.088514+0000","last_clean":"2024-09-17T08:23:01.088514+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:23:01.088514+0000","last_undegraded":"2024-09-17T08:23:01.088514+0000","last_fullsized":"2024-09-17T08:23:01.088514+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:20:24.572550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027118630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"274'14046","reported_seq":14219,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.502843+0000","last_change":"2024-09-17T08:22:31.507556+0000","last_active":"2024-09-17T08:23:08.502843+0000","last_peered":"2024-09-17T08:23:08.502843+0000","last_clean":"2024-09-17T08:23:08.502843+0000","last_became_active":"2024-09-17T08:20:07.346402+0000","last_became_peered":"2024-09-17T08:20:07.346402+0000","last_unstale":"2024-09-17T08:23:08.502843+0000","last_undegraded":"2024-09-17T08:23:08.502843+0000","last_fullsized":"2024-09-17T08:23:08.502843+0000","mapping_epoch":221,"log_start":"257'8100","ondisk_log_start":"257'8100","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11557","last_scrub_stamp":"2024-09-17T08:22:31.507376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.507376+0000","objects_scrubbed":4853,"log_size":5946,"log_dups_size":0,"ondisk_log_size":5946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.972003+0000","scrub_duration":1.513714271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151650304,"num_objects":2314,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14046,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":344,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624504+0000","last_change":"2024-09-17T08:22:29.458182+0000","last_active":"2024-09-17T08:22:58.624504+0000","last_peered":"2024-09-17T08:22:58.624504+0000","last_clean":"2024-09-17T08:22:58.624504+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:22:58.624504+0000","last_undegraded":"2024-09-17T08:22:58.624504+0000","last_fullsized":"2024-09-17T08:22:58.624504+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:29.458054+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.458054+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:35.090765+0000","scrub_duration":0.0040122279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":274,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.659226+0000","last_change":"2024-09-17T08:22:10.889503+0000","last_active":"2024-09-17T08:22:58.659226+0000","last_peered":"2024-09-17T08:22:58.659226+0000","last_clean":"2024-09-17T08:22:58.659226+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:22:58.659226+0000","last_undegraded":"2024-09-17T08:22:58.659226+0000","last_fullsized":"2024-09-17T08:22:58.659226+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:10.889256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:10.889256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:32.589523+0000","scrub_duration":0.006339183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":454,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.702372+0000","last_change":"2024-09-17T08:22:33.388252+0000","last_active":"2024-09-17T08:23:02.702372+0000","last_peered":"2024-09-17T08:23:02.702372+0000","last_clean":"2024-09-17T08:23:02.702372+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:23:02.702372+0000","last_undegraded":"2024-09-17T08:23:02.702372+0000","last_fullsized":"2024-09-17T08:23:02.702372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.388161+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.477432+0000","scrub_duration":0.002025004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":91,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624814+0000","last_change":"2024-09-17T08:22:06.237985+0000","last_active":"2024-09-17T08:22:58.624814+0000","last_peered":"2024-09-17T08:22:58.624814+0000","last_clean":"2024-09-17T08:22:58.624814+0000","last_became_active":"2024-09-17T08:20:02.504850+0000","last_became_peered":"2024-09-17T08:20:02.504850+0000","last_unstale":"2024-09-17T08:22:58.624814+0000","last_undegraded":"2024-09-17T08:22:58.624814+0000","last_fullsized":"2024-09-17T08:22:58.624814+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:06.237871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.237871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:17.845181+0000","scrub_duration":0.001444615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"274'14044","reported_seq":14107,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:08.622333+0000","last_change":"2024-09-17T08:20:07.154512+0000","last_active":"2024-09-17T08:23:08.622333+0000","last_peered":"2024-09-17T08:23:08.622333+0000","last_clean":"2024-09-17T08:23:08.622333+0000","last_became_active":"2024-09-17T08:20:07.154215+0000","last_became_peered":"2024-09-17T08:20:07.154215+0000","last_unstale":"2024-09-17T08:23:08.622333+0000","last_undegraded":"2024-09-17T08:23:08.622333+0000","last_fullsized":"2024-09-17T08:23:08.622333+0000","mapping_epoch":221,"log_start":"262'9100","ondisk_log_start":"262'9100","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:20:06.139182+0000","objects_scrubbed":0,"log_size":4944,"log_dups_size":0,"ondisk_log_size":4944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149028864,"num_objects":2274,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14044,"num_write_kb":522176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":363,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.074085+0000","last_change":"2024-09-17T08:22:39.204171+0000","last_active":"2024-09-17T08:23:01.074085+0000","last_peered":"2024-09-17T08:23:01.074085+0000","last_clean":"2024-09-17T08:23:01.074085+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:23:01.074085+0000","last_undegraded":"2024-09-17T08:23:01.074085+0000","last_fullsized":"2024-09-17T08:23:01.074085+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:22:39.204047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.204047+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:03.816885+0000","scrub_duration":0.061163012000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":297,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549372+0000","last_change":"2024-09-17T08:22:15.001277+0000","last_active":"2024-09-17T08:22:58.549372+0000","last_peered":"2024-09-17T08:22:58.549372+0000","last_clean":"2024-09-17T08:22:58.549372+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:22:58.549372+0000","last_undegraded":"2024-09-17T08:22:58.549372+0000","last_fullsized":"2024-09-17T08:22:58.549372+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:22:15.001149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.001149+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:39.794479+0000","scrub_duration":0.0054858570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":278,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.664704+0000","last_change":"2024-09-17T08:21:17.762815+0000","last_active":"2024-09-17T08:22:58.664704+0000","last_peered":"2024-09-17T08:22:58.664704+0000","last_clean":"2024-09-17T08:22:58.664704+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:22:58.664704+0000","last_undegraded":"2024-09-17T08:22:58.664704+0000","last_fullsized":"2024-09-17T08:22:58.664704+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:17.762302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:21:17.762302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054597830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":93,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652516+0000","last_change":"2024-09-17T08:21:32.565481+0000","last_active":"2024-09-17T08:22:58.652516+0000","last_peered":"2024-09-17T08:22:58.652516+0000","last_clean":"2024-09-17T08:22:58.652516+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:22:58.652516+0000","last_undegraded":"2024-09-17T08:22:58.652516+0000","last_fullsized":"2024-09-17T08:22:58.652516+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:32.565367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:32.565367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022451039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":306,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548049+0000","last_change":"2024-09-17T08:20:12.524600+0000","last_active":"2024-09-17T08:22:58.548049+0000","last_peered":"2024-09-17T08:22:58.548049+0000","last_clean":"2024-09-17T08:22:58.548049+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:22:58.548049+0000","last_undegraded":"2024-09-17T08:22:58.548049+0000","last_fullsized":"2024-09-17T08:22:58.548049+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:20:12.524489+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:20:12.524489+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001784494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":292,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.664838+0000","last_change":"2024-09-17T08:22:21.972879+0000","last_active":"2024-09-17T08:22:58.664838+0000","last_peered":"2024-09-17T08:22:58.664838+0000","last_clean":"2024-09-17T08:22:58.664838+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:22:58.664838+0000","last_undegraded":"2024-09-17T08:22:58.664838+0000","last_fullsized":"2024-09-17T08:22:58.664838+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:21.972582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.972582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.784884+0000","scrub_duration":0.0084090889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":282,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624573+0000","last_change":"2024-09-17T08:20:19.890506+0000","last_active":"2024-09-17T08:22:58.624573+0000","last_peered":"2024-09-17T08:22:58.624573+0000","last_clean":"2024-09-17T08:22:58.624573+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:22:58.624573+0000","last_undegraded":"2024-09-17T08:22:58.624573+0000","last_fullsized":"2024-09-17T08:22:58.624573+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:19.890356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:19.890356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012482635000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":265,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652677+0000","last_change":"2024-09-17T08:21:30.595600+0000","last_active":"2024-09-17T08:22:58.652677+0000","last_peered":"2024-09-17T08:22:58.652677+0000","last_clean":"2024-09-17T08:22:58.652677+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:22:58.652677+0000","last_undegraded":"2024-09-17T08:22:58.652677+0000","last_fullsized":"2024-09-17T08:22:58.652677+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.595456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002658952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":170,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624567+0000","last_change":"2024-09-17T08:22:27.392439+0000","last_active":"2024-09-17T08:22:58.624567+0000","last_peered":"2024-09-17T08:22:58.624567+0000","last_clean":"2024-09-17T08:22:58.624567+0000","last_became_active":"2024-09-17T08:20:02.510430+0000","last_became_peered":"2024-09-17T08:20:02.510430+0000","last_unstale":"2024-09-17T08:22:58.624567+0000","last_undegraded":"2024-09-17T08:22:58.624567+0000","last_fullsized":"2024-09-17T08:22:58.624567+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.392168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.392168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:41.379368+0000","scrub_duration":0.0041184029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":329,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624686+0000","last_change":"2024-09-17T08:21:30.956509+0000","last_active":"2024-09-17T08:22:58.624686+0000","last_peered":"2024-09-17T08:22:58.624686+0000","last_clean":"2024-09-17T08:22:58.624686+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:22:58.624686+0000","last_undegraded":"2024-09-17T08:22:58.624686+0000","last_fullsized":"2024-09-17T08:22:58.624686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:21:30.956385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.956385+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034003470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":196,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624301+0000","last_change":"2024-09-17T08:22:22.621469+0000","last_active":"2024-09-17T08:22:58.624301+0000","last_peered":"2024-09-17T08:22:58.624301+0000","last_clean":"2024-09-17T08:22:58.624301+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:22:58.624301+0000","last_undegraded":"2024-09-17T08:22:58.624301+0000","last_fullsized":"2024-09-17T08:22:58.624301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.621350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:41.649184+0000","scrub_duration":0.011642807999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":284,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625555+0000","last_change":"2024-09-17T08:20:12.398221+0000","last_active":"2024-09-17T08:22:58.625555+0000","last_peered":"2024-09-17T08:22:58.625555+0000","last_clean":"2024-09-17T08:22:58.625555+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:22:58.625555+0000","last_undegraded":"2024-09-17T08:22:58.625555+0000","last_fullsized":"2024-09-17T08:22:58.625555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:12.398134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:12.398134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012127189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":349,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624637+0000","last_change":"2024-09-17T08:22:12.398758+0000","last_active":"2024-09-17T08:22:58.624637+0000","last_peered":"2024-09-17T08:22:58.624637+0000","last_clean":"2024-09-17T08:22:58.624637+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:22:58.624637+0000","last_undegraded":"2024-09-17T08:22:58.624637+0000","last_fullsized":"2024-09-17T08:22:58.624637+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:12.398295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.398295+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:13.689428+0000","scrub_duration":0.011555360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":225,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624144+0000","last_change":"2024-09-17T08:20:31.890892+0000","last_active":"2024-09-17T08:22:58.624144+0000","last_peered":"2024-09-17T08:22:58.624144+0000","last_clean":"2024-09-17T08:22:58.624144+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:22:58.624144+0000","last_undegraded":"2024-09-17T08:22:58.624144+0000","last_fullsized":"2024-09-17T08:22:58.624144+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:20:31.890809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001129559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":267,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.985862+0000","last_change":"2024-09-17T08:21:29.598725+0000","last_active":"2024-09-17T08:23:02.985862+0000","last_peered":"2024-09-17T08:23:02.985862+0000","last_clean":"2024-09-17T08:23:02.985862+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:23:02.985862+0000","last_undegraded":"2024-09-17T08:23:02.985862+0000","last_fullsized":"2024-09-17T08:23:02.985862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:21:29.598601+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.598601+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021201536999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":284,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547757+0000","last_change":"2024-09-17T08:22:25.070949+0000","last_active":"2024-09-17T08:22:58.547757+0000","last_peered":"2024-09-17T08:22:58.547757+0000","last_clean":"2024-09-17T08:22:58.547757+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:22:58.547757+0000","last_undegraded":"2024-09-17T08:22:58.547757+0000","last_fullsized":"2024-09-17T08:22:58.547757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:25.070821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.070821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:38.414437+0000","scrub_duration":0.0076690669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":231,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625919+0000","last_change":"2024-09-17T08:22:09.487846+0000","last_active":"2024-09-17T08:22:58.625919+0000","last_peered":"2024-09-17T08:22:58.625919+0000","last_clean":"2024-09-17T08:22:58.625919+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:22:58.625919+0000","last_undegraded":"2024-09-17T08:22:58.625919+0000","last_fullsized":"2024-09-17T08:22:58.625919+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:09.487639+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.487639+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.378775+0000","scrub_duration":0.0094462299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547983+0000","last_change":"2024-09-17T08:22:26.034397+0000","last_active":"2024-09-17T08:22:58.547983+0000","last_peered":"2024-09-17T08:22:58.547983+0000","last_clean":"2024-09-17T08:22:58.547983+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:22:58.547983+0000","last_undegraded":"2024-09-17T08:22:58.547983+0000","last_fullsized":"2024-09-17T08:22:58.547983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:22:26.033887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.236921+0000","scrub_duration":0.011752558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":449,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:02.987391+0000","last_change":"2024-09-17T08:20:23.856731+0000","last_active":"2024-09-17T08:23:02.987391+0000","last_peered":"2024-09-17T08:23:02.987391+0000","last_clean":"2024-09-17T08:23:02.987391+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:23:02.987391+0000","last_undegraded":"2024-09-17T08:23:02.987391+0000","last_fullsized":"2024-09-17T08:23:02.987391+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:20:23.856627+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:20:23.856627+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001634627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":278,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549247+0000","last_change":"2024-09-17T08:21:15.924915+0000","last_active":"2024-09-17T08:22:58.549247+0000","last_peered":"2024-09-17T08:22:58.549247+0000","last_clean":"2024-09-17T08:22:58.549247+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:22:58.549247+0000","last_undegraded":"2024-09-17T08:22:58.549247+0000","last_fullsized":"2024-09-17T08:22:58.549247+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.924766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.924766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057564210000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":201,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549051+0000","last_change":"2024-09-17T08:21:12.364191+0000","last_active":"2024-09-17T08:22:58.549051+0000","last_peered":"2024-09-17T08:22:58.549051+0000","last_clean":"2024-09-17T08:22:58.549051+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:22:58.549051+0000","last_undegraded":"2024-09-17T08:22:58.549051+0000","last_fullsized":"2024-09-17T08:22:58.549051+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:21:12.364059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:12.364059+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076608910000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":147,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625391+0000","last_change":"2024-09-17T08:22:12.321397+0000","last_active":"2024-09-17T08:22:58.625391+0000","last_peered":"2024-09-17T08:22:58.625391+0000","last_clean":"2024-09-17T08:22:58.625391+0000","last_became_active":"2024-09-17T08:20:02.492161+0000","last_became_peered":"2024-09-17T08:20:02.492161+0000","last_unstale":"2024-09-17T08:22:58.625391+0000","last_undegraded":"2024-09-17T08:22:58.625391+0000","last_fullsized":"2024-09-17T08:22:58.625391+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:12.321279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.321279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:26.989484+0000","scrub_duration":0.0030511240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":293,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624190+0000","last_change":"2024-09-17T08:22:16.495330+0000","last_active":"2024-09-17T08:22:58.624190+0000","last_peered":"2024-09-17T08:22:58.624190+0000","last_clean":"2024-09-17T08:22:58.624190+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:22:58.624190+0000","last_undegraded":"2024-09-17T08:22:58.624190+0000","last_fullsized":"2024-09-17T08:22:58.624190+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:16.495149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:16.495149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:25.181375+0000","scrub_duration":0.001489045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":302,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547218+0000","last_change":"2024-09-17T08:22:14.202171+0000","last_active":"2024-09-17T08:22:58.547218+0000","last_peered":"2024-09-17T08:22:58.547218+0000","last_clean":"2024-09-17T08:22:58.547218+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:22:58.547218+0000","last_undegraded":"2024-09-17T08:22:58.547218+0000","last_fullsized":"2024-09-17T08:22:58.547218+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.201855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.201855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:29.297433+0000","scrub_duration":0.018711583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":351,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548707+0000","last_change":"2024-09-17T08:22:43.519295+0000","last_active":"2024-09-17T08:22:58.548707+0000","last_peered":"2024-09-17T08:22:58.548707+0000","last_clean":"2024-09-17T08:22:58.548707+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:22:58.548707+0000","last_undegraded":"2024-09-17T08:22:58.548707+0000","last_fullsized":"2024-09-17T08:22:58.548707+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:20:18.048698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:20:18.048698+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019777420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":169,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548882+0000","last_change":"2024-09-17T08:22:18.499171+0000","last_active":"2024-09-17T08:22:58.548882+0000","last_peered":"2024-09-17T08:22:58.548882+0000","last_clean":"2024-09-17T08:22:58.548882+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:22:58.548882+0000","last_undegraded":"2024-09-17T08:22:58.548882+0000","last_fullsized":"2024-09-17T08:22:58.548882+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.499009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.499009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:34.753804+0000","scrub_duration":0.020240600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":158,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549187+0000","last_change":"2024-09-17T08:20:13.128701+0000","last_active":"2024-09-17T08:22:58.549187+0000","last_peered":"2024-09-17T08:22:58.549187+0000","last_clean":"2024-09-17T08:22:58.549187+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:22:58.549187+0000","last_undegraded":"2024-09-17T08:22:58.549187+0000","last_fullsized":"2024-09-17T08:22:58.549187+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:13.128622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:13.128622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013289720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":273,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.623903+0000","last_change":"2024-09-17T08:21:33.941971+0000","last_active":"2024-09-17T08:22:58.623903+0000","last_peered":"2024-09-17T08:22:58.623903+0000","last_clean":"2024-09-17T08:22:58.623903+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:22:58.623903+0000","last_undegraded":"2024-09-17T08:22:58.623903+0000","last_fullsized":"2024-09-17T08:22:58.623903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.941822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.941822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:03.413596+0000","scrub_duration":0.0057366700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":334,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.623957+0000","last_change":"2024-09-17T08:22:06.283965+0000","last_active":"2024-09-17T08:22:58.623957+0000","last_peered":"2024-09-17T08:22:58.623957+0000","last_clean":"2024-09-17T08:22:58.623957+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:22:58.623957+0000","last_undegraded":"2024-09-17T08:22:58.623957+0000","last_fullsized":"2024-09-17T08:22:58.623957+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:22:06.283841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.283841+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:12.974698+0000","scrub_duration":0.0044434549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":152,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.657453+0000","last_change":"2024-09-17T08:20:24.347336+0000","last_active":"2024-09-17T08:22:58.657453+0000","last_peered":"2024-09-17T08:22:58.657453+0000","last_clean":"2024-09-17T08:22:58.657453+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:22:58.657453+0000","last_undegraded":"2024-09-17T08:22:58.657453+0000","last_fullsized":"2024-09-17T08:22:58.657453+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:24.347238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:20:24.347238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013570769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":143,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624280+0000","last_change":"2024-09-17T08:21:29.113802+0000","last_active":"2024-09-17T08:22:58.624280+0000","last_peered":"2024-09-17T08:22:58.624280+0000","last_clean":"2024-09-17T08:22:58.624280+0000","last_became_active":"2024-09-17T08:20:02.510678+0000","last_became_peered":"2024-09-17T08:20:02.510678+0000","last_unstale":"2024-09-17T08:22:58.624280+0000","last_undegraded":"2024-09-17T08:22:58.624280+0000","last_fullsized":"2024-09-17T08:22:58.624280+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.113668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00142106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":288,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625013+0000","last_change":"2024-09-17T08:21:14.392541+0000","last_active":"2024-09-17T08:22:58.625013+0000","last_peered":"2024-09-17T08:22:58.625013+0000","last_clean":"2024-09-17T08:22:58.625013+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:22:58.625013+0000","last_undegraded":"2024-09-17T08:22:58.625013+0000","last_fullsized":"2024-09-17T08:22:58.625013+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:14.392384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:04.412068+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.392384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021806350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":308,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624396+0000","last_change":"2024-09-17T08:22:31.627432+0000","last_active":"2024-09-17T08:22:58.624396+0000","last_peered":"2024-09-17T08:22:58.624396+0000","last_clean":"2024-09-17T08:22:58.624396+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:22:58.624396+0000","last_undegraded":"2024-09-17T08:22:58.624396+0000","last_fullsized":"2024-09-17T08:22:58.624396+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:31.627297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.627297+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:49.425907+0000","scrub_duration":0.0041108009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":183,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625146+0000","last_change":"2024-09-17T08:22:21.330021+0000","last_active":"2024-09-17T08:22:58.625146+0000","last_peered":"2024-09-17T08:22:58.625146+0000","last_clean":"2024-09-17T08:22:58.625146+0000","last_became_active":"2024-09-17T08:20:23.487930+0000","last_became_peered":"2024-09-17T08:20:23.487930+0000","last_unstale":"2024-09-17T08:22:58.625146+0000","last_undegraded":"2024-09-17T08:22:58.625146+0000","last_fullsized":"2024-09-17T08:22:58.625146+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:22:21.329888+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.329888+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:21.638203+0000","scrub_duration":0.0021372819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624305+0000","last_change":"2024-09-17T08:21:31.458346+0000","last_active":"2024-09-17T08:22:58.624305+0000","last_peered":"2024-09-17T08:22:58.624305+0000","last_clean":"2024-09-17T08:22:58.624305+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:22:58.624305+0000","last_undegraded":"2024-09-17T08:22:58.624305+0000","last_fullsized":"2024-09-17T08:22:58.624305+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.458200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.458200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020112709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":273,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624158+0000","last_change":"2024-09-17T08:22:39.655694+0000","last_active":"2024-09-17T08:22:58.624158+0000","last_peered":"2024-09-17T08:22:58.624158+0000","last_clean":"2024-09-17T08:22:58.624158+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:22:58.624158+0000","last_undegraded":"2024-09-17T08:22:58.624158+0000","last_fullsized":"2024-09-17T08:22:58.624158+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:39.655581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.655581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.268378+0000","scrub_duration":0.0042238659999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625411+0000","last_change":"2024-09-17T08:21:19.480328+0000","last_active":"2024-09-17T08:22:58.625411+0000","last_peered":"2024-09-17T08:22:58.625411+0000","last_clean":"2024-09-17T08:22:58.625411+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:22:58.625411+0000","last_undegraded":"2024-09-17T08:22:58.625411+0000","last_fullsized":"2024-09-17T08:22:58.625411+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:21:19.480200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:19.480200+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035346700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":179,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548452+0000","last_change":"2024-09-17T08:20:19.999322+0000","last_active":"2024-09-17T08:22:58.548452+0000","last_peered":"2024-09-17T08:22:58.548452+0000","last_clean":"2024-09-17T08:22:58.548452+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:22:58.548452+0000","last_undegraded":"2024-09-17T08:22:58.548452+0000","last_fullsized":"2024-09-17T08:22:58.548452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:19.999232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:20:19.999232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001724136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":319,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547975+0000","last_change":"2024-09-17T08:22:20.129709+0000","last_active":"2024-09-17T08:22:58.547975+0000","last_peered":"2024-09-17T08:22:58.547975+0000","last_clean":"2024-09-17T08:22:58.547975+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:22:58.547975+0000","last_undegraded":"2024-09-17T08:22:58.547975+0000","last_fullsized":"2024-09-17T08:22:58.547975+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:20.129556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.129556+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:40.752500+0000","scrub_duration":0.010294539,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":366,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.075106+0000","last_change":"2024-09-17T08:20:10.960103+0000","last_active":"2024-09-17T08:23:01.075106+0000","last_peered":"2024-09-17T08:23:01.075106+0000","last_clean":"2024-09-17T08:23:01.075106+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:23:01.075106+0000","last_undegraded":"2024-09-17T08:23:01.075106+0000","last_fullsized":"2024-09-17T08:23:01.075106+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:20:10.959993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:20:10.959993+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014768190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":169,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.083632+0000","last_change":"2024-09-17T08:20:15.122143+0000","last_active":"2024-09-17T08:23:01.083632+0000","last_peered":"2024-09-17T08:23:01.083632+0000","last_clean":"2024-09-17T08:23:01.083632+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:23:01.083632+0000","last_undegraded":"2024-09-17T08:23:01.083632+0000","last_fullsized":"2024-09-17T08:23:01.083632+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:15.122049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:15.122049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016647579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624391+0000","last_change":"2024-09-17T08:21:31.941666+0000","last_active":"2024-09-17T08:22:58.624391+0000","last_peered":"2024-09-17T08:22:58.624391+0000","last_clean":"2024-09-17T08:22:58.624391+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:22:58.624391+0000","last_undegraded":"2024-09-17T08:22:58.624391+0000","last_fullsized":"2024-09-17T08:22:58.624391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.941534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066151480000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":176,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.625065+0000","last_change":"2024-09-17T08:22:39.435438+0000","last_active":"2024-09-17T08:22:58.625065+0000","last_peered":"2024-09-17T08:22:58.625065+0000","last_clean":"2024-09-17T08:22:58.625065+0000","last_became_active":"2024-09-17T08:20:02.495177+0000","last_became_peered":"2024-09-17T08:20:02.495177+0000","last_unstale":"2024-09-17T08:22:58.625065+0000","last_undegraded":"2024-09-17T08:22:58.625065+0000","last_fullsized":"2024-09-17T08:22:58.625065+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:39.435291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.435291+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:42.016755+0000","scrub_duration":0.005519843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":304,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.547902+0000","last_change":"2024-09-17T08:21:19.815826+0000","last_active":"2024-09-17T08:22:58.547902+0000","last_peered":"2024-09-17T08:22:58.547902+0000","last_clean":"2024-09-17T08:22:58.547902+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:22:58.547902+0000","last_undegraded":"2024-09-17T08:22:58.547902+0000","last_fullsized":"2024-09-17T08:22:58.547902+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:19.815475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:21:19.815475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061512909999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":274,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.549199+0000","last_change":"2024-09-17T08:22:33.121286+0000","last_active":"2024-09-17T08:22:58.549199+0000","last_peered":"2024-09-17T08:22:58.549199+0000","last_clean":"2024-09-17T08:22:58.549199+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:22:58.549199+0000","last_undegraded":"2024-09-17T08:22:58.549199+0000","last_fullsized":"2024-09-17T08:22:58.549199+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:33.121209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.121209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.285318+0000","scrub_duration":0.0012454370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":231,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548713+0000","last_change":"2024-09-17T08:21:25.000057+0000","last_active":"2024-09-17T08:22:58.548713+0000","last_peered":"2024-09-17T08:22:58.548713+0000","last_clean":"2024-09-17T08:22:58.548713+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:22:58.548713+0000","last_undegraded":"2024-09-17T08:22:58.548713+0000","last_fullsized":"2024-09-17T08:22:58.548713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.999967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038102280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":396,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.652751+0000","last_change":"2024-09-17T08:22:20.983964+0000","last_active":"2024-09-17T08:22:58.652751+0000","last_peered":"2024-09-17T08:22:58.652751+0000","last_clean":"2024-09-17T08:22:58.652751+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:22:58.652751+0000","last_undegraded":"2024-09-17T08:22:58.652751+0000","last_fullsized":"2024-09-17T08:22:58.652751+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:20.983823+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.983823+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.519128+0000","scrub_duration":0.010983009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":169,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-17T08:23:01.083414+0000","last_change":"2024-09-17T08:22:22.507255+0000","last_active":"2024-09-17T08:23:01.083414+0000","last_peered":"2024-09-17T08:23:01.083414+0000","last_clean":"2024-09-17T08:23:01.083414+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:23:01.083414+0000","last_undegraded":"2024-09-17T08:23:01.083414+0000","last_fullsized":"2024-09-17T08:23:01.083414+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.507142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.507142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:52.303965+0000","scrub_duration":0.0043685180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":263,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624572+0000","last_change":"2024-09-17T08:20:20.860711+0000","last_active":"2024-09-17T08:22:58.624572+0000","last_peered":"2024-09-17T08:22:58.624572+0000","last_clean":"2024-09-17T08:22:58.624572+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:22:58.624572+0000","last_undegraded":"2024-09-17T08:22:58.624572+0000","last_fullsized":"2024-09-17T08:22:58.624572+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:20:20.860156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:20:20.860156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076486699999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624346+0000","last_change":"2024-09-17T08:21:24.016408+0000","last_active":"2024-09-17T08:22:58.624346+0000","last_peered":"2024-09-17T08:22:58.624346+0000","last_clean":"2024-09-17T08:22:58.624346+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:22:58.624346+0000","last_undegraded":"2024-09-17T08:22:58.624346+0000","last_fullsized":"2024-09-17T08:22:58.624346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.016289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.016289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011053393999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":351,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.624302+0000","last_change":"2024-09-17T08:21:27.998120+0000","last_active":"2024-09-17T08:22:58.624302+0000","last_peered":"2024-09-17T08:22:58.624302+0000","last_clean":"2024-09-17T08:22:58.624302+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:22:58.624302+0000","last_undegraded":"2024-09-17T08:22:58.624302+0000","last_fullsized":"2024-09-17T08:22:58.624302+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:21:27.997982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.997982+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010572906999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":291,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-17T08:22:58.548972+0000","last_change":"2024-09-17T08:21:21.069425+0000","last_active":"2024-09-17T08:22:58.548972+0000","last_peered":"2024-09-17T08:22:58.548972+0000","last_clean":"2024-09-17T08:22:58.548972+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:22:58.548972+0000","last_undegraded":"2024-09-17T08:22:58.548972+0000","last_fullsized":"2024-09-17T08:22:58.548972+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.069304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.069304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013987202000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":2442264600,"num_objects":37267,"num_object_clones":0,"num_object_copies":111801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225127,"num_write_kb":8396545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2199,"num_bytes_recovered":144113664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7394766848,"data_stored":7394754632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82902,"ondisk_log_size":82902,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1121,"num_read_kb":878,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":456,"num_read_kb":451,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070628,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1884416,"kb_used_data":1384056,"kb_used_omap":1,"kb_used_meta":500350,"kb_avail":91860736,"statfs":{"total":95995035648,"available":94065393664,"internally_reserved":0,"allocated":1417273344,"data_stored":1416961312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":512358859},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2010000000000001,"5min":1.927,"15min":1.319},"min":{"1min":0.28000000000000003,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":13.446,"5min":62.018000000000001,"15min":62.018000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":2.2559999999999998,"5min":1.948,"15min":1.395},"min":{"1min":0.313,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":13.417,"5min":61.896999999999998,"15min":61.896999999999998},"last":2.4670000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":2.6789999999999998,"15min":1.446},"min":{"1min":0.23899999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":14.731,"5min":62.067,"15min":62.067},"last":0.432},{"interface":"front","average":{"1min":3.2639999999999998,"5min":2.722,"15min":1.4079999999999999},"min":{"1min":0.32300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":14.401999999999999,"5min":62.194000000000003,"15min":62.194000000000003},"last":2.9660000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:22:25 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.89600000000000002,"15min":0.86699999999999999},"min":{"1min":0.39000000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":3.0659999999999998},{"interface":"front","average":{"1min":1.3169999999999999,"5min":0.70899999999999996,"15min":0.60799999999999998},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.8809999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":3.1040000000000001,"5min":1.9370000000000001,"15min":1.2749999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":17.356000000000002,"5min":44.606000000000002,"15min":44.606000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":3.1909999999999998,"5min":2.1200000000000001,"15min":1.4159999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":17.315999999999999,"5min":44.591000000000001,"15min":44.591000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":0.88300000000000001,"15min":0.64000000000000001},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":4.391,"5min":5.335,"15min":5.335},"last":2.915},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.56699999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":3.798,"5min":5.9669999999999996,"15min":5.9669999999999996},"last":0.27200000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":0.94799999999999995,"15min":0.74399999999999999},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.8450000000000002,"5min":8.6400000000000006,"15min":16.808},"last":2.2250000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":0.91100000000000003,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":11.842000000000001},"last":2.9409999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.01,"15min":0.74099999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.26600000000000001},"max":{"1min":4.4640000000000004,"5min":7.2359999999999998,"15min":7.2359999999999998},"last":2.9870000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.77500000000000002,"15min":0.61699999999999999},"min":{"1min":0.35099999999999998,"5min":0.246,"15min":0.246},"max":{"1min":3.3849999999999998,"5min":5.9400000000000004,"15min":5.9400000000000004},"last":0.32400000000000001}]}]},{"osd":6,"up_from":50,"seq":214748364969,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790572,"kb_used_data":470692,"kb_used_omap":1,"kb_used_meta":319870,"kb_avail":92954580,"statfs":{"total":95995035648,"available":95185489920,"internally_reserved":0,"allocated":481988608,"data_stored":481696292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":327547335},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":2.0470000000000002,"15min":2.0190000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.2189999999999999,"5min":81.007000000000005,"15min":81.007000000000005},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":2.0579999999999998,"15min":2.0710000000000002},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.19900000000000001},"max":{"1min":4.8710000000000004,"5min":80.310000000000002,"15min":80.310000000000002},"last":2.1190000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":2.516,"15min":1.5369999999999999},"min":{"1min":0.28699999999999998,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":3.7869999999999999,"5min":80.813000000000002,"15min":80.813000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.101,"5min":2.5190000000000001,"15min":1.5820000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5259999999999998,"5min":80.281000000000006,"15min":80.281000000000006},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Sep 17 08:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.67400000000000004,"15min":0.64000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.6930000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.57899999999999996,"15min":0.54600000000000004},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.309}]},{"osd":3,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":2.294,"15min":1.8420000000000001},"min":{"1min":0.32000000000000001,"5min":0.29899999999999999,"15min":0.28699999999999998},"max":{"1min":3.8250000000000002,"5min":80.427000000000007,"15min":80.427000000000007},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.1699999999999999,"5min":2.125,"15min":1.764},"min":{"1min":0.36899999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.2930000000000001,"5min":80.379999999999995,"15min":80.379999999999995},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67900000000000005,"15min":0.56899999999999995},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.215},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":4.6219999999999999},"last":2.1509999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.51800000000000002},"min":{"1min":0.30299999999999999,"5min":0.26800000000000002,"15min":0.25},"max":{"1min":2.4060000000000001,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.80700000000000005,"15min":0.61599999999999999},"min":{"1min":0.33100000000000002,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":5.5659999999999998,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":2.0800000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.57899999999999996},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1869999999999998,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.79100000000000004,"15min":0.64700000000000002},"min":{"1min":0.317,"5min":0.246,"15min":0.246},"max":{"1min":2.6909999999999998,"5min":4.774,"15min":4.774},"last":2.1850000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.76100000000000001,"15min":0.59799999999999998},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5590000000000002,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":0.63300000000000001}]}]},{"osd":5,"up_from":41,"seq":176093659310,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1866480,"kb_used_data":1380072,"kb_used_omap":1,"kb_used_meta":486398,"kb_avail":91878672,"statfs":{"total":95995035648,"available":94083760128,"internally_reserved":0,"allocated":1413193728,"data_stored":1412896745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":498072006},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.458,"15min":1.357},"min":{"1min":0.21299999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":3.4670000000000001,"5min":28.867000000000001,"15min":63.098999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.482,"15min":1.304},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.9620000000000002,"5min":28.998999999999999,"15min":62.567},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.321,"15min":0.91100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":3.0830000000000002,"5min":17.457000000000001,"15min":17.457000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.3080000000000001,"15min":0.97299999999999998},"min":{"1min":0.26300000000000001,"5min":0.248,"15min":0.22},"max":{"1min":5.8810000000000002,"5min":17.274000000000001,"15min":17.274000000000001},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.77600000000000002,"15min":0.745},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":1.012,"15min":1.075},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.1200000000000001,"5min":5.0279999999999996,"15min":5.0279999999999996},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.1339999999999999,"15min":1.2629999999999999},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":11.291,"5min":11.291,"15min":75.286000000000001},"last":67.676000000000002},{"interface":"front","average":{"1min":1.639,"5min":1.113,"15min":1.2450000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":77.206999999999994},"last":67.691999999999993}]},{"osd":4,"last update":"Tue Sep 17 08:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":1.117,"15min":0.66300000000000003},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.19},"max":{"1min":1.05,"5min":42.421999999999997,"15min":42.421999999999997},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.0740000000000001,"15min":0.67600000000000005},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":2.573,"5min":42.191000000000003,"15min":42.191000000000003},"last":0.29899999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.57499999999999996},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27700000000000002},"max":{"1min":3.4769999999999999,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.69699999999999995,"15min":0.58499999999999996},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":1.2909999999999999,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.39200000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":1.349,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.79000000000000004,"15min":0.64900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":3.0059999999999998,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.374}]}]},{"osd":4,"up_from":32,"seq":137438953651,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":795832,"kb_used_data":462000,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":92949320,"statfs":{"total":95995035648,"available":95180103680,"internally_reserved":0,"allocated":473088000,"data_stored":472783430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":341834182},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.91200000000000003},"min":{"1min":0.24099999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":23.786000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.219,"15min":0.219},"max":{"1min":2.2890000000000001,"5min":3.7839999999999998,"15min":23.245000000000001},"last":0.433}]},{"osd":1,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":1.5640000000000001,"15min":1.0860000000000001},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.3789999999999996,"5min":14.164,"15min":23.135000000000002},"last":1.944},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.401,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.246,"15min":0.24099999999999999},"max":{"1min":9.0719999999999992,"5min":9.0719999999999992,"15min":23.434999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.65400000000000003,"15min":0.59199999999999997},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.6950000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":2.8069999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.353,"15min":1.7969999999999999},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.226,"5min":10.012,"15min":154.137},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.208,"15min":1.6779999999999999},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":5.556,"5min":9.5180000000000007,"15min":153.54400000000001},"last":0.35499999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80000000000000004,"15min":0.55400000000000005},"min":{"1min":0.27600000000000002,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":3.0510000000000002,"5min":6.2329999999999997,"15min":6.2329999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.872,"15min":0.59599999999999997},"min":{"1min":0.27000000000000002,"5min":0.26800000000000002,"15min":0.25900000000000001},"max":{"1min":3.3940000000000001,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.39500000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76200000000000001,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":1.9750000000000001,"5min":4.827,"15min":4.827},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.66900000000000004,"15min":0.58699999999999997},"min":{"1min":0.33700000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82799999999999996,"5min":3.601,"15min":6.0010000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":0.878,"15min":0.68700000000000006},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.0259999999999998,"5min":4.875,"15min":4.875},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.875,"15min":0.65400000000000003},"min":{"1min":0.30399999999999999,"5min":0.28199999999999997,"15min":0.27300000000000002},"max":{"1min":2.7949999999999999,"5min":4.6950000000000003,"15min":4.6950000000000003},"last":0.45300000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117175,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1744516,"kb_used_data":1217916,"kb_used_omap":1,"kb_used_meta":526590,"kb_avail":92000636,"statfs":{"total":95995035648,"available":94208651264,"internally_reserved":0,"allocated":1247145984,"data_stored":1246828603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":539228616},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.958,"5min":1.7230000000000001,"15min":1.3520000000000001},"min":{"1min":0.156,"5min":0.156,"15min":0.156},"max":{"1min":9.7029999999999994,"5min":9.7029999999999994,"15min":9.7029999999999994},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.5069999999999999,"15min":1.2270000000000001},"min":{"1min":0.254,"5min":0.17799999999999999,"15min":0.16200000000000001},"max":{"1min":4.3049999999999997,"5min":14.118,"15min":14.118},"last":1.7689999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.6259999999999999,"15min":1.2829999999999999},"min":{"1min":0.28899999999999998,"5min":0.254,"15min":0.186},"max":{"1min":6.7039999999999997,"5min":9.5,"15min":9.5},"last":1.7949999999999999},{"interface":"front","average":{"1min":2.2189999999999999,"5min":1.552,"15min":1.2709999999999999},"min":{"1min":0.27700000000000002,"5min":0.192,"15min":0.188},"max":{"1min":5.9210000000000003,"5min":5.9210000000000003,"15min":10.891999999999999},"last":2.0329999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:22:27 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.167,"15min":1.105},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":5.3899999999999997,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.994,"5min":1.712,"15min":1.665},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":6.2809999999999997,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":0.72499999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.6339999999999999,"15min":1.214},"min":{"1min":0.28499999999999998,"5min":0.28100000000000003,"15min":0.252},"max":{"1min":4.3339999999999996,"5min":35.841000000000001,"15min":35.841000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.5820000000000001,"15min":1.165},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.9359999999999999,"5min":35.857999999999997,"15min":35.857999999999997},"last":0.30599999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":1.3140000000000001,"15min":1.1100000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.629,"15min":1.2330000000000001},"min":{"1min":0.44900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.8520000000000003,"5min":9.4260000000000002,"15min":9.4260000000000002},"last":0.78500000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:22:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.27,"15min":1.087},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.29499999999999998},"max":{"1min":7.6669999999999998,"5min":7.6669999999999998,"15min":10.945},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6870000000000001,"15min":1.302},"min":{"1min":0.39000000000000001,"5min":0.314,"15min":0.311},"max":{"1min":9.6600000000000001,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.755}]},{"osd":7,"last update":"Tue Sep 17 08:23:02 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":1.5780000000000001,"15min":1.236},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.316},"max":{"1min":38.936999999999998,"5min":38.936999999999998,"15min":38.936999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":2.802,"5min":1.6399999999999999,"15min":1.2350000000000001},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":36.634999999999998,"5min":36.634999999999998,"15min":36.634999999999998},"last":0.96399999999999997}]}]},{"osd":2,"up_from":227,"seq":974957576226,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":589360,"kb_used_data":483624,"kb_used_omap":4,"kb_used_meta":105723,"kb_avail":93155792,"statfs":{"total":95995035648,"available":95391531008,"internally_reserved":0,"allocated":495230976,"data_stored":494934367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":108261027},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.8240000000000001,"5min":1.143,"15min":1.0289999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":7.7450000000000001,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":0.373},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.3939999999999999,"15min":1.4410000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.8300000000000001,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.2889999999999999,"15min":1.25},"min":{"1min":0.23499999999999999,"5min":0.224,"15min":0.224},"max":{"1min":5.8129999999999997,"5min":5.8129999999999997,"15min":5.8129999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.3089999999999999,"15min":1.284},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":4.5810000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.31}]},{"osd":3,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.4630000000000001,"15min":1.4379999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.9910000000000001,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3200000000000003,"5min":5.3200000000000003,"15min":5.3200000000000003},"last":0.85199999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":0.90200000000000002,"15min":0.84999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5870000000000002,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.157,"5min":0.88800000000000001,"15min":0.84299999999999997},"min":{"1min":0.307,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.411,"5min":3.411,"15min":3.411},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1,"15min":0.95999999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.5009999999999999,"5min":4.915,"15min":4.915},"last":1.24},{"interface":"front","average":{"1min":1.194,"5min":0.97499999999999998,"15min":0.93899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.4489999999999998,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":1.4079999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.77900000000000003,"15min":0.71299999999999997},"min":{"1min":0.36399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.262,"5min":0.95399999999999996,"15min":0.90300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5409999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.437}]},{"osd":7,"last update":"Tue Sep 17 08:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.036,"15min":0.99399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.93899999999999995,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":5.2690000000000001,"5min":5.2690000000000001,"15min":5.2690000000000001},"last":0.81399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1934608,"kb_used_data":1366152,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":91810544,"statfs":{"total":95995035648,"available":94013997056,"internally_reserved":0,"allocated":1398939648,"data_stored":1398610593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":582089158},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.508,"15min":1.407},"min":{"1min":0.21199999999999999,"5min":0.155,"15min":0.155},"max":{"1min":9.5730000000000004,"5min":9.5730000000000004,"15min":31.585999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.7450000000000001,"15min":1.5860000000000001},"min":{"1min":0.248,"5min":0.19500000000000001,"15min":0.17199999999999999},"max":{"1min":5.3129999999999997,"5min":16.812000000000001,"15min":35.807000000000002},"last":0.26900000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:22:24 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.1439999999999999,"15min":0.91000000000000003},"min":{"1min":0.315,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":22.943000000000001,"5min":22.943000000000001,"15min":22.943000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.498,"15min":1.4890000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":5.4560000000000004,"5min":5.4560000000000004,"15min":5.4560000000000004},"last":1.4450000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:22:34 2024","interfaces":[{"interface":"back","average":{"1min":2.161,"5min":1.6140000000000001,"15min":1.2290000000000001},"min":{"1min":0.441,"5min":0.184,"15min":0.184},"max":{"1min":5.1920000000000002,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":1.708},{"interface":"front","average":{"1min":3.5449999999999999,"5min":2.2469999999999999,"15min":1.4279999999999999},"min":{"1min":0.41399999999999998,"5min":0.23699999999999999,"15min":0.189},"max":{"1min":22.984999999999999,"5min":22.984999999999999,"15min":22.984999999999999},"last":4.7969999999999997}]},{"osd":4,"last update":"Tue Sep 17 08:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.4610000000000001,"15min":1.1759999999999999},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":3.2919999999999998,"5min":16.884,"15min":16.884},"last":4.7679999999999998},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.409,"15min":1.075},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.25700000000000001},"max":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":10.208},"last":1.327}]},{"osd":5,"last update":"Tue Sep 17 08:22:14 2024","interfaces":[{"interface":"back","average":{"1min":1.8380000000000001,"5min":1.411,"15min":1.0389999999999999},"min":{"1min":0.50700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.9740000000000002,"5min":7.9139999999999997,"15min":7.9139999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":2.6139999999999999,"5min":1.8069999999999999,"15min":1.2290000000000001},"min":{"1min":0.45500000000000002,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":16.853999999999999,"5min":16.853999999999999,"15min":16.853999999999999},"last":4.7450000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:22:34 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.605,"15min":1.125},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.28199999999999997},"max":{"1min":3.2869999999999999,"5min":6.6020000000000003,"15min":10.253},"last":1.357},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.575,"15min":1.153},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.27200000000000002},"max":{"1min":3.4039999999999999,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:22:09 2024","interfaces":[{"interface":"back","average":{"1min":2.847,"5min":1.8080000000000001,"15min":1.2290000000000001},"min":{"1min":0.57099999999999995,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":16.908999999999999,"5min":16.908999999999999,"15min":16.908999999999999},"last":4.71},{"interface":"front","average":{"1min":1.506,"5min":1.526,"15min":1.125},"min":{"1min":0.53900000000000003,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":3.2349999999999999,"5min":6.5620000000000003,"15min":10.279},"last":1.411}]}]},{"osd":0,"up_from":8,"seq":34359738563,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1003380,"kb_used_data":599020,"kb_used_omap":1,"kb_used_meta":404350,"kb_avail":92741772,"statfs":{"total":95995035648,"available":94967574528,"internally_reserved":0,"allocated":613396480,"data_stored":613094850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":414054854},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":2.3239999999999998,"15min":1.6950000000000001},"min":{"1min":0.20399999999999999,"5min":0.189,"15min":0.189},"max":{"1min":4.9459999999999997,"5min":43.493000000000002,"15min":61.649000000000001},"last":0.217},{"interface":"front","average":{"1min":1.538,"5min":2.4460000000000002,"15min":1.845},"min":{"1min":0.247,"5min":0.20200000000000001,"15min":0.18099999999999999},"max":{"1min":5.3840000000000003,"5min":40.436,"15min":61.281999999999996},"last":1.9510000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:22:29 2024","interfaces":[{"interface":"back","average":{"1min":2.379,"5min":1.383,"15min":1.2170000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":39.023000000000003,"5min":39.023000000000003,"15min":39.023000000000003},"last":0.93600000000000005},{"interface":"front","average":{"1min":3.0350000000000001,"5min":1.3140000000000001,"15min":1.0269999999999999},"min":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":41.003999999999998,"5min":41.003999999999998,"15min":41.003999999999998},"last":1.0409999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:22:37 2024","interfaces":[{"interface":"back","average":{"1min":1.738,"5min":2.23,"15min":1.847},"min":{"1min":0.20799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.2199999999999998,"5min":39.942999999999998,"15min":61.926000000000002},"last":1.171},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.8380000000000001,"15min":1.649},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":5.7679999999999998,"5min":38.950000000000003,"15min":61.616},"last":0.47299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.401,"15min":1.236},"min":{"1min":0.32200000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.0800000000000001,"5min":41.158000000000001,"15min":41.158000000000001},"last":1.077},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.4430000000000001,"15min":1.2529999999999999},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.9670000000000001,"5min":38.978000000000002,"15min":38.978000000000002},"last":0.96799999999999997}]},{"osd":5,"last update":"Tue Sep 17 08:22:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.7529999999999999,"15min":1.157},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1559999999999997,"5min":39.058,"15min":39.058},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.998,"15min":1.3109999999999999},"min":{"1min":0.38500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":5.101,"5min":43.485999999999997,"15min":43.485999999999997},"last":1.115}]},{"osd":6,"last update":"Tue Sep 17 08:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.7450000000000001,"15min":1.1879999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.919,"5min":38.917999999999999,"15min":38.917999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.9810000000000001,"15min":1.331},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":5.1239999999999997,"5min":41.189,"15min":41.189},"last":1.145}]},{"osd":7,"last update":"Tue Sep 17 08:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.5269999999999999,"15min":1.4279999999999999},"min":{"1min":0.308,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.9319999999999999,"5min":39.045999999999999,"15min":39.045999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.4930000000000001,"15min":1.3700000000000001},"min":{"1min":0.40200000000000002,"5min":0.34399999999999997,"15min":0.307},"max":{"1min":3.7570000000000001,"5min":39.082999999999998,"15min":39.082999999999998},"last":2.3580000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":594608128,"data_stored":594608128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1360072704,"data_stored":1360068632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":486735872,"data_stored":486735872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1215102976,"data_stored":1215102976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":466681856,"data_stored":466681856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1404440576,"data_stored":1404436504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":464715776,"data_stored":464715776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1402408960,"data_stored":1402404888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:23:12.540 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-17T08:23:12.540 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:23:12.540 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:23:12.541 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-17T08:23:12.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 0 2024-09-17T08:23:12.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:12.648 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:12.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[40649]: 9.2 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[29565]: 9.2 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:23:12.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:12 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:23:12.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:12.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:12.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:12.930 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:12.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:12.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:23:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:23:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-mon[33187]: 9.2 scrub starts 2024-09-17T08:23:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:23:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:23:13.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:13.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:23:13.074 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:13.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:13.172 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:23:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:23:13.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:13.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:13.221 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:23:13.221 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:23:13.250 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:23:13.250 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:23:13.273 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:13.283 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:23:13.283 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:23:13.283 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:13.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:13.314 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:23:13.315 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:23:13.347 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:23:13.347 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:23:13.361 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:13.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:13.379 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:23:13.379 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:23:13.412 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:23:13.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:23:13.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:13.475 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:12 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:13.492 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:13.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:13.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:13.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:13.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:13.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:13.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:13.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:13.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:23:13.877 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[40649]: 9.2 scrub ok 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/185802366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[40649]: pgmap v584: 148 pgs: 148 active+clean; 2.2 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[29565]: 9.2 scrub ok 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/185802366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:13.878 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:13 smithi079 ceph-mon[29565]: pgmap v584: 148 pgs: 148 active+clean; 2.2 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:13.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:13.907 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:13.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:13 smithi173 ceph-mon[33187]: 9.2 scrub ok 2024-09-17T08:23:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/185802366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:13 smithi173 ceph-mon[33187]: pgmap v584: 148 pgs: 148 active+clean; 2.2 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:23:14.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:14.063 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:14.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:14.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:14.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:14.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:14.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:14.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:14.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.464 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:14.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:14.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:14.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:14.746 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:14.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:23:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:23:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:14 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:23:14.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:14.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:14.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:23:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:14 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:23:14.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:15.012 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:15.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:15.115 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:15.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:15.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:15.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:15.286 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:15.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:15.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:15.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:15.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:15.559 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:15.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:15.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:15.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:15.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:15.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:15.826 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:15.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:15.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:15.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:15.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T08:23:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:15 smithi173 ceph-mon[33187]: pgmap v586: 148 pgs: 148 active+clean; 1.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-17T08:23:16.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:16.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:16.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[40649]: pgmap v586: 148 pgs: 148 active+clean; 1.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:15 smithi079 ceph-mon[29565]: pgmap v586: 148 pgs: 148 active+clean; 1.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-17T08:23:16.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:16.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:16.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:16.299 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:16.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:16.370 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:16.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:16.472 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:16.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.503 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:16.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:16.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:16.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.649 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:16.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:16.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:23:16.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:23:16.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[29565]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T08:23:16.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[40649]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:23:16.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:16 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:23:16.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:16.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.796 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:16.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:16.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:16.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:16.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:16.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:16 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:23:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:16 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:23:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:16 smithi173 ceph-mon[33187]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T08:23:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:16 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:23:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:16 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:23:17.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:17.063 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:17.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:17.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:17.191 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:17.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:17.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:17.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:17.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:17.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:17.463 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:17.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:17.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:17.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:17.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:17.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:17.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:17.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:17.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:23:17.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:17.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:17.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:17.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2914422870' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:17 smithi173 ceph-mon[33187]: pgmap v588: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 741 B/s rd, 0 B/s wr, 944 op/s 2024-09-17T08:23:17.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:18.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:18.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:18.058 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2914422870' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:18.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[40649]: pgmap v588: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 741 B/s rd, 0 B/s wr, 944 op/s 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2914422870' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-17T08:23:18.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:17 smithi079 ceph-mon[29565]: pgmap v588: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 741 B/s rd, 0 B/s wr, 944 op/s 2024-09-17T08:23:18.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:18.167 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:18.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.169 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:18.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:18.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:18.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:18.409 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:18.439 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:18.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:18.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:18.576 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:18.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:18.678 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:18.714 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:18.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:18.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:18.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.838 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:18.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:18.940 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:18.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:18.970 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:18.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: 9.4 scrub starts 2024-09-17T08:23:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:23:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:23:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: 9.4 scrub ok 2024-09-17T08:23:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:23:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:18 smithi173 ceph-mon[33187]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T08:23:19.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:19.092 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:19.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: 9.4 scrub starts 2024-09-17T08:23:19.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:23:19.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: 9.4 scrub ok 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[40649]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: 9.4 scrub starts 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: 9.4 scrub ok 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-17T08:23:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:18 smithi079 ceph-mon[29565]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T08:23:19.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:19.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:19.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:19.319 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:19.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:19.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:19.463 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:19.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:19.599 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:19.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:19.700 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:19.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.724 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:19.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:19.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.857 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:19.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:23:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:19 smithi173 ceph-mon[33187]: pgmap v591: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 632 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:23:19.988 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:19.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:19.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:20.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:23:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:23:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T08:23:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:23:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:23:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[29565]: pgmap v591: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 632 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:23:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:19 smithi079 ceph-mon[40649]: pgmap v591: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 632 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-17T08:23:20.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:20.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:20.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:20.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.265 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:20.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:20.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:20.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:20.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:20.485 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:20.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:20.572 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:20.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:20.674 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:20.704 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:20.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:20.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:20.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.846 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:20.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:20.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:20 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:23:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:20 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:23:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:20 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:20.981 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:20.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:20.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:21.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:21.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:23:21.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:23:21.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:21.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:23:21.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:23:21.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:20 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:21.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:21.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:21.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:21.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:21.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:21.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:21.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:21.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.385 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:21.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:21.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:21.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:21.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:21.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:21.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:21.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.790 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:21.791 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:21.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:21.917 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:21.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:21.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:21 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:23:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:21 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:23:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:21 smithi173 ceph-mon[33187]: pgmap v592: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:23:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:21 smithi173 ceph-mon[33187]: 9.6 scrub starts 2024-09-17T08:23:22.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:22.057 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:22.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:22.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[29565]: pgmap v592: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[29565]: 9.6 scrub starts 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[40649]: pgmap v592: 148 pgs: 3 peering, 145 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:23:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:21 smithi079 ceph-mon[40649]: 9.6 scrub starts 2024-09-17T08:23:22.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:22.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:22.195 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:22.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:22.342 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:22.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:22.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:22.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:22.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:22.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:22.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:22.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:22.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:22.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:22.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:22.852 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:22.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:22.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:22.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:22.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:23.022 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:23.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:23.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:23.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:23.258 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:23.286 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:23.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:23.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:23.420 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:23.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:23.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:23.559 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:23.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[40649]: 9.6 scrub ok 2024-09-17T08:23:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:23:23.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:23:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[29565]: 9.6 scrub ok 2024-09-17T08:23:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:23:23.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:23 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:23:23.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:23.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:23 smithi173 ceph-mon[33187]: 9.6 scrub ok 2024-09-17T08:23:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:23 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:23:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:23 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:23:23.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:23.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:23.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:23.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:23.975 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:23.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:23.976 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:24.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:24.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:24.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:24.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:24.251 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:24.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:24.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:24.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:24.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:24.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: pgmap v593: 148 pgs: 2 peering, 146 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 391 B/s rd, 0 B/s wr, 986 op/s 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:23:24.580 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: pgmap v593: 148 pgs: 2 peering, 146 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 391 B/s rd, 0 B/s wr, 986 op/s 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:23:24.581 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:24 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:23:24.592 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:24.604 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:24.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:24.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:23:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:23:24.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: pgmap v593: 148 pgs: 2 peering, 146 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 391 B/s rd, 0 B/s wr, 986 op/s 2024-09-17T08:23:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:23:24.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:24 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:23:24.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:24.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:24.829 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:24.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:24.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:24.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:24.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:24.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:24.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:25.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:25.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:25.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:25.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:25.268 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:25.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:25.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:25.384 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:23:25.384 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:23:25.384 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-17T08:23:25.385 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:23:25.402 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:25.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:25.453 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:23:25.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:25.533 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:25.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:25.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:25.673 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:25.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:23:25.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:25.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:23:25.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:23:25.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:23:25.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:23:25.807 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:25.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.809 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:25.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:25.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:25.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:25.943 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:25.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:26.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:26.074 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:26.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:26.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:26.211 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:26.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:26.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:26.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:26.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:26.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:26.491 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:26.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:26.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:26.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:23:26.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:23:26.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.12 deep-scrub starts 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.12 deep-scrub ok 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: pgmap v595: 148 pgs: 148 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:23:26.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[29565]: Health check update: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.12 deep-scrub starts 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.12 deep-scrub ok 2024-09-17T08:23:26.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: pgmap v595: 148 pgs: 148 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:23:26.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:23:26.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:23:26.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:26 smithi079 ceph-mon[40649]: Health check update: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:26.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.627 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:26.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/991639783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:23:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.12 deep-scrub starts 2024-09-17T08:23:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.12 deep-scrub ok 2024-09-17T08:23:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: pgmap v595: 148 pgs: 148 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-17T08:23:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:23:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:23:26.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:26 smithi173 ceph-mon[33187]: Health check update: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:23:26.728 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:26.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.762 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:26.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:26.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:26.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:26.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:26.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:27.002 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:27.037 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.038 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:27.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:27.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:27.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:27.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:27.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:27.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:27.343 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:23:27.344 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:23:27.344 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T08:23:27.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:23:27.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:27.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:27.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:27.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:27.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:27.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:27.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:27.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-17T08:23:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:23:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:23:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:23:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:23:27.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:27 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:23:27.730 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:27.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:27.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-17T08:23:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:23:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:23:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:23:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:23:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:23:27.870 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:23:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:27 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:23:27.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:27.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:27.973 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:28.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:28.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:28.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.143 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:28.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:28.244 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:28.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:28.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:28.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:28.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:28.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:28.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:28.547 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:28.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:28.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:23:28.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:28.708 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:23:28.708 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:23:28.708 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[29565]: pgmap v596: 148 pgs: 148 active+clean; 1.2 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 927 B/s rd, 0 B/s wr, 827 op/s 2024-09-17T08:23:28.708 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[40649]: pgmap v596: 148 pgs: 148 active+clean; 1.2 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 927 B/s rd, 0 B/s wr, 827 op/s 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:23:28.709 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:28 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:23:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:28 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:23:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:28 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:23:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:28 smithi173 ceph-mon[33187]: pgmap v596: 148 pgs: 148 active+clean; 1.2 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 927 B/s rd, 0 B/s wr, 827 op/s 2024-09-17T08:23:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:28 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:23:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:28 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:23:28.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:28.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:28.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:23:28.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:28.899 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:28.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:28.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:29.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:29.057 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:29.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:29.159 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:29.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:29.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:29.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:29.296 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:23:29.296 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:23:29.314 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:29.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:29.334 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:23:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:23:29.373 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:23:29.373 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:23:29.407 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:29.413 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:29.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:23:29.415 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:23:29.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:29.421 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:29.440 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:29.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:29.449 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:23:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:23:29.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:23:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:23:29.512 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:23:29.512 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:23:29.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:29.543 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:23:29.543 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:23:29.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.582 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:29.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:29.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:29.719 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:29.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:23:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:23:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/493237050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:29.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:29.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:29.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:23:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/493237050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:23:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/493237050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:29.961 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:29.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:29.990 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:29.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:30.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:30.137 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:30.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:30.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.274 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:30.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:30.418 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:30.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[29565]: 5.3 deep-scrub starts 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[29565]: 5.3 deep-scrub ok 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[29565]: pgmap v597: 148 pgs: 148 active+clean; 825 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:23:30.535 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[40649]: 5.3 deep-scrub starts 2024-09-17T08:23:30.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[40649]: 5.3 deep-scrub ok 2024-09-17T08:23:30.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[40649]: pgmap v597: 148 pgs: 148 active+clean; 825 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:30.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:23:30.536 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:30 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:23:30.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:30.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:30.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:30.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:30 smithi173 ceph-mon[33187]: 5.3 deep-scrub starts 2024-09-17T08:23:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:30 smithi173 ceph-mon[33187]: 5.3 deep-scrub ok 2024-09-17T08:23:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:30 smithi173 ceph-mon[33187]: pgmap v597: 148 pgs: 148 active+clean; 825 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:30 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:23:30.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:30 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:23:30.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:30.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:30.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:30.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:30.965 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:30.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:30.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:31.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:31.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.106 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:31.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:31.207 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:31.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:31.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:31.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:31.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:31.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:31.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:31.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.522 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:31.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:31.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:31.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:31.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:31.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:31 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:23:31.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:31 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:23:31.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:31.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:31.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:31 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:23:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:31 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:23:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:31 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:23:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:31 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:23:31.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:31.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:31.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:31.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:32.017 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:32.046 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:32.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.047 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:32.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:32.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.182 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:32.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:32.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:32.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:32.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:32.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:32.471 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:32.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:32.572 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:32.608 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:32.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[29565]: pgmap v598: 148 pgs: 148 active+clean; 801 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:32.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[40649]: pgmap v598: 148 pgs: 148 active+clean; 801 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:32.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:32.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:32.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:32.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:32.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:32.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:32.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:32 smithi173 ceph-mon[33187]: pgmap v598: 148 pgs: 148 active+clean; 801 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:23:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:32 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:23:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:32 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:23:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:32.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:33.029 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:33.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:33.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:33.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:33.174 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:33.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:33.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:33.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:33.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:33.412 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:33.439 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:33.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:33.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:33.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:33.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.563 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:33.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:33.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:33.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.699 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:33.699 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:33.800 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:33.835 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:33.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:23:33.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 6.6 deep-scrub starts 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[40649]: 6.6 deep-scrub ok 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 6.6 deep-scrub starts 2024-09-17T08:23:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:33 smithi079 ceph-mon[29565]: 6.6 deep-scrub ok 2024-09-17T08:23:33.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:33.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:23:33.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:33.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:33.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:23:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:23:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:23:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:23:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 6.6 deep-scrub starts 2024-09-17T08:23:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:33 smithi173 ceph-mon[33187]: 6.6 deep-scrub ok 2024-09-17T08:23:34.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:34.122 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:34.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:34.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:34.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.271 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:34.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:34.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:34.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:34.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:34.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:34.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:34.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:34.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.662 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:34.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:34.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:34.799 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:34.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:23:34.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: pgmap v599: 148 pgs: 148 active+clean; 700 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: pgmap v599: 148 pgs: 148 active+clean; 700 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:23:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:34 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:23:34.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:34.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:34.941 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:34.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: pgmap v599: 148 pgs: 148 active+clean; 700 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:23:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:34 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:23:35.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:35.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:35.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:35.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:35.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:35.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:35.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:35.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.351 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:35.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:35.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:35.486 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:35.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:35.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:35.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:35 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:23:35.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:35 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:23:35.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:35 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:23:35.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:35 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:23:35.623 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:35.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:35.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:35.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:35.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:35.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:35.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:35.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:35 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:23:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:35 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:23:35.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:36.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:36.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:36.137 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:36.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.168 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:36.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:36.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:36.304 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:36.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:36.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:36.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.449 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:36.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:23:36.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:36.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:36.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:36.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 6.3 deep-scrub starts 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 6.3 deep-scrub ok 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: pgmap v600: 148 pgs: 148 active+clean; 424 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 6.3 deep-scrub starts 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 6.3 deep-scrub ok 2024-09-17T08:23:36.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: pgmap v600: 148 pgs: 148 active+clean; 424 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:36 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:23:36.682 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:36.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:36.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:36.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:36.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:36.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:36.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 6.3 deep-scrub starts 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 6.3 deep-scrub ok 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: pgmap v600: 148 pgs: 148 active+clean; 424 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:23:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:36 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:23:36.983 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:36.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:36.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:37.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:37.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:37.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:37.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:37.259 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:37.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:37.361 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:37.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:37.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:37.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:37.527 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:37.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:37.558 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 281, 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:23:36.496929+0000", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:20:22.465407+0000", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:15:28.332713+0000", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5801984, 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-17T08:23:37.559 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 19, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 9, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:23:37.560 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.561 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.562 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.563 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.564 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.565 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6, 2024-09-17T08:23:37.566 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.5, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:23:37.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.570 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.571 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:23:37.572 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:23:37.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:23:37.574 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:23:37.575 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.576 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.577 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9900000095367432, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9900000095367432, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1699999570846558, 2024-09-17T08:23:37.578 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:23:37.580 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.582 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:23:37.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.8799999952316284, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.8799999952316284, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.5, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.5, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:23:37.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:23:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.588 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.589 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:23:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.592 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.595 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.596 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 9, 2024-09-17T08:23:37.597 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_2", 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:20:05.575722+0000", 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:23:37.598 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "225", 2024-09-17T08:23:37.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:23:37.600 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:23:37.601 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:23:37.602 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:23:37.603 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.604 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:23:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.609 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.610 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.614 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 18, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 204, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 227, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 205, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/359954782", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/359954782", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/359954782", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/359954782", 2024-09-17T08:23:37.619 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:23:37.620 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.623 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:23:37.624 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.625 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.626 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.627 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.628 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 205, 2024-09-17T08:23:37.631 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:23:37.633 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:23:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:23:37.636 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 227, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:23:37.641 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:23:37.642 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 278, 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.645 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:23:37.647 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:23:15.468485+0000", 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.648 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:22:25.269155+0000", 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:22:46.185021+0000", 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 204 2024-09-17T08:23:37.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:23:17.041572+0000", 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:23:37.650 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:22:42.725258+0000", 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:23:13.819167+0000", 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:22:54.538053+0000", 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:23:01.017101+0000", 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.653 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:23:37.654 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.655 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "6.b", 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:23:37.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 6 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.657 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:23:37.658 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:23:37.659 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:37.660 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:23:37.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:37.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:37.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:37.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:37.818 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:37.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:23:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1297951219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T08:23:37.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1297951219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:23:37.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:37.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:37.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:37.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:23:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:23:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/787072420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:23:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T08:23:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1297951219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:23:38.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:38.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:38.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:38.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:38.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:38.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:38.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:38.378 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:38.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:38.480 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:38.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:38.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:38.564 INFO:tasks.thrashosds.thrasher:j is {'epoch': 281, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:23:36.496929+0000', 'last_up_change': '2024-09-17T08:20:22.465407+0000', 'last_in_change': '2024-09-17T08:15:28.332713+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 1.1699999570846558, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-17T08:20:05.575722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 278, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 278, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 18, 'last_clean_end': 204, 'up_from': 227, 'up_thru': 278, 'down_at': 205, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 359954782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 359954782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 359954782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 359954782}]}, 'public_addr': '172.21.15.79:6819/359954782', 'cluster_addr': '172.21.15.79:6821/359954782', 'heartbeat_back_addr': '172.21.15.79:6825/359954782', 'heartbeat_front_addr': '172.21.15.79:6823/359954782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 278, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 278, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 205, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 227, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 278, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:23:15.468485+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:22:25.269155+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:22:46.185021+0000', 'dead_epoch': 204}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:23:17.041572+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:22:42.725258+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:23:13.819167+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:22:54.538053+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:23:01.017101+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 2}, {'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:23:38.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:23:38.617 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:38.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:38.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:38.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:38.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:38.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:38.810 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Removed 131196 objects 2024-09-17T08:23:38.810 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Clean up completed and total clean up time :110.587 2024-09-17T08:23:38.815 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:38 smithi079 ceph-mon[29565]: pgmap v602: 148 pgs: 148 active+clean; 399 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:38.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:38 smithi079 ceph-mon[40649]: pgmap v602: 148 pgs: 148 active+clean; 399 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:38.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:38.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:38.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:38.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:38 smithi173 ceph-mon[33187]: pgmap v602: 148 pgs: 148 active+clean; 399 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:23:39.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:39.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:39.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:39.086 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:23:39.086 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:23:39.093 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:39.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:39.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:39.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:39.288 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:39.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:39.308 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:39.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:39.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:23:39.412 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:39.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.413 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:39.492 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-17T08:23:39.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-17T08:23:39.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:39.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:39.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:39.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:39.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:39.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:39.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:39.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:39.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:39.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:39.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:39.981 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:39.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:40.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:40.103 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:40.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:40.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:40.227 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:40.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:40.329 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:40.355 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:40.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:40.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:40.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:40.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:40.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:23:40.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:40.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:40.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:40.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:40.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:40.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:40.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:40.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:40.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:40 smithi079 ceph-mon[40649]: pgmap v603: 148 pgs: 148 active+clean; 44 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:23:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:40 smithi079 ceph-mon[29565]: pgmap v603: 148 pgs: 148 active+clean; 44 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:23:40.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:40.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:40 smithi173 ceph-mon[33187]: pgmap v603: 148 pgs: 148 active+clean; 44 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:23:40.983 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:40.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:40.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:41.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:41.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:41.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:41.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:41.235 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:41.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:41.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:41.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:41.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:41.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:41.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:41.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:41.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:41.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:41.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:41.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:41.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:41.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:41.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:41.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[40649]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T08:23:41.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[40649]: pgmap v605: 148 pgs: 148 active+clean; 32 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:23:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[29565]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T08:23:41.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:41 smithi079 ceph-mon[29565]: pgmap v605: 148 pgs: 148 active+clean; 32 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:23:41.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:41.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:41.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:41 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:41 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:41 smithi173 ceph-mon[33187]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T08:23:41.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:41 smithi173 ceph-mon[33187]: pgmap v605: 148 pgs: 148 active+clean; 32 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:23:41.974 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:41.999 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:42.101 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:42.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.127 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:42.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:42.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:42.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:42.377 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:42.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:42.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.503 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:42.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:42.629 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:42.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.630 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:42.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:42.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.754 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:42.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:42.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:42.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:42.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:42.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:43.005 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:43.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:43.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:43.128 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:43.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:43.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:43.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:43.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:43.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:43.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.382 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:43.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:43.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:43.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:43.507 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:43.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.508 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:43.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:43.632 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:43.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:43.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:43.756 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:43.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:43.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:43 smithi079 ceph-mon[40649]: pgmap v606: 148 pgs: 148 active+clean; 10 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s 2024-09-17T08:23:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:43 smithi079 ceph-mon[29565]: pgmap v606: 148 pgs: 148 active+clean; 10 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s 2024-09-17T08:23:43.882 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:43.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:43.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:43 smithi173 ceph-mon[33187]: pgmap v606: 148 pgs: 148 active+clean; 10 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 828 op/s 2024-09-17T08:23:43.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:44.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.007 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:44.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:44.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:44.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:44.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:44.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:44.257 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:44.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:44.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:44.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:44.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:44.485 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:44.510 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:44.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:44.612 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:44.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:44.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:44.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:44.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:44.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:44.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:44.863 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:44.864 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:44.889 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:44.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:44.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:44.991 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:45.016 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:45.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:45.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:45.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:45.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:45.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:45.269 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:45.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:45.371 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:45.396 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:45.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:45.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:45.522 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:45.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:45.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:45.645 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:45.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:45.720 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:45 smithi079 ceph-mon[29565]: pgmap v607: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 710 op/s 2024-09-17T08:23:45.721 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:45 smithi079 ceph-mon[40649]: pgmap v607: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 710 op/s 2024-09-17T08:23:45.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:45.771 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:45.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:45.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:45.894 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:45.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:45.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:45 smithi173 ceph-mon[33187]: pgmap v607: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 710 op/s 2024-09-17T08:23:45.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:46.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:46.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:46.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:46.144 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:46.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:46.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:46.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:23:46.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:23:46.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:23:46.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:23:46.293 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:23:46.293 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:23:46.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:23:46.309 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:23:46.326 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:23:46.326 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:23:46.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:46.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:23:46.469 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:23:46.499 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:46.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:46.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:46.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:46.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:46.773 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:46.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3238817678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:46 smithi079 ceph-mon[40649]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3238817678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:46 smithi079 ceph-mon[29565]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:46.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:46.901 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:46.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:46.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3238817678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:23:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:46 smithi173 ceph-mon[33187]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:23:47.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:47.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.025 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:47.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:47.148 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:47.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:47.275 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:47.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:47.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:47.398 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:47.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:47.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:47.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:47.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:47.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:47.771 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:47.854 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-09-17T08:23:45.502607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1835,"num_read_kb":1681,"num_write":263289,"num_write_kb":8399004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2431,"num_bytes_recovered":145889393,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83639,"ondisk_log_size":83639,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":3405796,"kb_used_data":7556,"kb_used_omap":15,"kb_used_meta":3398128,"kb_avail":746555420,"statfs":{"total":767960285184,"available":764472750080,"internally_reserved":0,"allocated":7737344,"data_stored":5341514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":3479683861},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-418250776,"num_objects":-6383,"num_object_clones":0,"num_object_copies":-19149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6383,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6383,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.995203"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":281,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515846+0000","last_change":"2024-09-17T08:22:32.714143+0000","last_active":"2024-09-17T08:23:36.515846+0000","last_peered":"2024-09-17T08:23:36.515846+0000","last_clean":"2024-09-17T08:23:36.515846+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:23:36.515846+0000","last_undegraded":"2024-09-17T08:23:36.515846+0000","last_fullsized":"2024-09-17T08:23:36.515846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.714032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.714032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.893234+0000","scrub_duration":0.052748915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.527021+0000","last_change":"2024-09-17T08:23:33.814550+0000","last_active":"2024-09-17T08:23:36.527021+0000","last_peered":"2024-09-17T08:23:36.527021+0000","last_clean":"2024-09-17T08:23:36.527021+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:23:36.527021+0000","last_undegraded":"2024-09-17T08:23:36.527021+0000","last_fullsized":"2024-09-17T08:23:36.527021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:33.814411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:23:33.814411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:46.810297+0000","scrub_duration":0.0040554900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":358,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512818+0000","last_change":"2024-09-17T08:23:28.822917+0000","last_active":"2024-09-17T08:23:36.512818+0000","last_peered":"2024-09-17T08:23:36.512818+0000","last_clean":"2024-09-17T08:23:36.512818+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:23:36.512818+0000","last_undegraded":"2024-09-17T08:23:36.512818+0000","last_fullsized":"2024-09-17T08:23:36.512818+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:23:28.822775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.822775+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:58.191478+0000","scrub_duration":0.0081422939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":312,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523914+0000","last_change":"2024-09-17T08:23:33.566799+0000","last_active":"2024-09-17T08:23:36.523914+0000","last_peered":"2024-09-17T08:23:36.523914+0000","last_clean":"2024-09-17T08:23:36.523914+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:23:36.523914+0000","last_undegraded":"2024-09-17T08:23:36.523914+0000","last_fullsized":"2024-09-17T08:23:36.523914+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:33.566719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:33.566719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:50.057068+0000","scrub_duration":0.016610028999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":288,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513114+0000","last_change":"2024-09-17T08:22:34.633621+0000","last_active":"2024-09-17T08:23:36.513114+0000","last_peered":"2024-09-17T08:23:36.513114+0000","last_clean":"2024-09-17T08:23:36.513114+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:23:36.513114+0000","last_undegraded":"2024-09-17T08:23:36.513114+0000","last_fullsized":"2024-09-17T08:23:36.513114+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:34.633510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:22:34.633510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:55.117384+0000","scrub_duration":0.0028104390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":414,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523916+0000","last_change":"2024-09-17T08:22:12.026146+0000","last_active":"2024-09-17T08:23:36.523916+0000","last_peered":"2024-09-17T08:23:36.523916+0000","last_clean":"2024-09-17T08:23:36.523916+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:23:36.523916+0000","last_undegraded":"2024-09-17T08:23:36.523916+0000","last_fullsized":"2024-09-17T08:23:36.523916+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:12.026000+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.026000+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077245940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510443+0000","last_change":"2024-09-17T08:22:11.468233+0000","last_active":"2024-09-17T08:23:36.510443+0000","last_peered":"2024-09-17T08:23:36.510443+0000","last_clean":"2024-09-17T08:23:36.510443+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:23:36.510443+0000","last_undegraded":"2024-09-17T08:23:36.510443+0000","last_fullsized":"2024-09-17T08:23:36.510443+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:11.468116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.468116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051835620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524525+0000","last_change":"2024-09-17T08:22:32.148128+0000","last_active":"2024-09-17T08:23:36.524525+0000","last_peered":"2024-09-17T08:23:36.524525+0000","last_clean":"2024-09-17T08:23:36.524525+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:23:36.524525+0000","last_undegraded":"2024-09-17T08:23:36.524525+0000","last_fullsized":"2024-09-17T08:23:36.524525+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:32.147832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:48.900525+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.147832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.754489+0000","scrub_duration":0.0067988659999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":298,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.520761+0000","last_change":"2024-09-17T08:22:14.492261+0000","last_active":"2024-09-17T08:23:36.520761+0000","last_peered":"2024-09-17T08:23:36.520761+0000","last_clean":"2024-09-17T08:23:36.520761+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:23:36.520761+0000","last_undegraded":"2024-09-17T08:23:36.520761+0000","last_fullsized":"2024-09-17T08:23:36.520761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.492185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.492185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:43.260230+0000","scrub_duration":0.057522131999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":330,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524699+0000","last_change":"2024-09-17T08:21:34.973710+0000","last_active":"2024-09-17T08:23:36.524699+0000","last_peered":"2024-09-17T08:23:36.524699+0000","last_clean":"2024-09-17T08:23:36.524699+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:23:36.524699+0000","last_undegraded":"2024-09-17T08:23:36.524699+0000","last_fullsized":"2024-09-17T08:23:36.524699+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:21:34.973570+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.973570+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.059354883999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":323,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515780+0000","last_change":"2024-09-17T08:22:22.423974+0000","last_active":"2024-09-17T08:23:36.515780+0000","last_peered":"2024-09-17T08:23:36.515780+0000","last_clean":"2024-09-17T08:23:36.515780+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:23:36.515780+0000","last_undegraded":"2024-09-17T08:23:36.515780+0000","last_fullsized":"2024-09-17T08:23:36.515780+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.423862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032447660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510314+0000","last_change":"2024-09-17T08:21:24.523252+0000","last_active":"2024-09-17T08:23:36.510314+0000","last_peered":"2024-09-17T08:23:36.510314+0000","last_clean":"2024-09-17T08:23:36.510314+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:23:36.510314+0000","last_undegraded":"2024-09-17T08:23:36.510314+0000","last_fullsized":"2024-09-17T08:23:36.510314+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:24.523136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:24.523136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078907060000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":313,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516980+0000","last_change":"2024-09-17T08:21:20.504900+0000","last_active":"2024-09-17T08:23:36.516980+0000","last_peered":"2024-09-17T08:23:36.516980+0000","last_clean":"2024-09-17T08:23:36.516980+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:23:36.516980+0000","last_undegraded":"2024-09-17T08:23:36.516980+0000","last_fullsized":"2024-09-17T08:23:36.516980+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:20.504720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.504720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018477529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":325,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524964+0000","last_change":"2024-09-17T08:22:36.105717+0000","last_active":"2024-09-17T08:23:36.524964+0000","last_peered":"2024-09-17T08:23:36.524964+0000","last_clean":"2024-09-17T08:23:36.524964+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:23:36.524964+0000","last_undegraded":"2024-09-17T08:23:36.524964+0000","last_fullsized":"2024-09-17T08:23:36.524964+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:22:36.105550+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:57.914458+0000","scrub_duration":0.012187885000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":197,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526523+0000","last_change":"2024-09-17T08:22:35.817757+0000","last_active":"2024-09-17T08:23:36.526523+0000","last_peered":"2024-09-17T08:23:36.526523+0000","last_clean":"2024-09-17T08:23:36.526523+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:23:36.526523+0000","last_undegraded":"2024-09-17T08:23:36.526523+0000","last_fullsized":"2024-09-17T08:23:36.526523+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:35.817629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.817629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.187574+0000","scrub_duration":0.0023115000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524237+0000","last_change":"2024-09-17T08:22:28.020605+0000","last_active":"2024-09-17T08:23:36.524237+0000","last_peered":"2024-09-17T08:23:36.524237+0000","last_clean":"2024-09-17T08:23:36.524237+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:23:36.524237+0000","last_undegraded":"2024-09-17T08:23:36.524237+0000","last_fullsized":"2024-09-17T08:23:36.524237+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:28.019265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:28.019265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.020045+0000","scrub_duration":0.0075046389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"281'16136","reported_seq":17773,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.758339+0000","last_change":"2024-09-17T08:22:35.057102+0000","last_active":"2024-09-17T08:23:38.758339+0000","last_peered":"2024-09-17T08:23:38.758339+0000","last_clean":"2024-09-17T08:23:38.758339+0000","last_became_active":"2024-09-17T08:20:32.362328+0000","last_became_peered":"2024-09-17T08:20:32.362328+0000","last_unstale":"2024-09-17T08:23:38.758339+0000","last_undegraded":"2024-09-17T08:23:38.758339+0000","last_fullsized":"2024-09-17T08:23:38.758339+0000","mapping_epoch":230,"log_start":"267'11200","ondisk_log_start":"267'11200","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11650","last_scrub_stamp":"2024-09-17T08:22:35.056760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:35.056760+0000","objects_scrubbed":4535,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:01.046356+0000","scrub_duration":1.5582728380000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16136,"num_write_kb":516352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":47382528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":421,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525507+0000","last_change":"2024-09-17T08:22:11.061223+0000","last_active":"2024-09-17T08:23:36.525507+0000","last_peered":"2024-09-17T08:23:36.525507+0000","last_clean":"2024-09-17T08:23:36.525507+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:23:36.525507+0000","last_undegraded":"2024-09-17T08:23:36.525507+0000","last_fullsized":"2024-09-17T08:23:36.525507+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:22:11.061047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:11.061047+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082017440000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515516+0000","last_change":"2024-09-17T08:23:28.485482+0000","last_active":"2024-09-17T08:23:36.515516+0000","last_peered":"2024-09-17T08:23:36.515516+0000","last_clean":"2024-09-17T08:23:36.515516+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:23:36.515516+0000","last_undegraded":"2024-09-17T08:23:36.515516+0000","last_fullsized":"2024-09-17T08:23:36.515516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:28.485377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.485377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:33.274791+0000","scrub_duration":0.0013475239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514127+0000","last_change":"2024-09-17T08:23:24.040027+0000","last_active":"2024-09-17T08:23:36.514127+0000","last_peered":"2024-09-17T08:23:36.514127+0000","last_clean":"2024-09-17T08:23:36.514127+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:23:36.514127+0000","last_undegraded":"2024-09-17T08:23:36.514127+0000","last_fullsized":"2024-09-17T08:23:36.514127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:24.039776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:24.039776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:31.104728+0000","scrub_duration":0.0052197119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":277,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523107+0000","last_change":"2024-09-17T08:23:26.554740+0000","last_active":"2024-09-17T08:23:36.523107+0000","last_peered":"2024-09-17T08:23:36.523107+0000","last_clean":"2024-09-17T08:23:36.523107+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:23:36.523107+0000","last_undegraded":"2024-09-17T08:23:36.523107+0000","last_fullsized":"2024-09-17T08:23:36.523107+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:26.554615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:26.554615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:49.994851+0000","scrub_duration":0.0040208750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"281'16228","reported_seq":16400,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.791051+0000","last_change":"2024-09-17T08:22:37.068629+0000","last_active":"2024-09-17T08:23:38.791051+0000","last_peered":"2024-09-17T08:23:38.791051+0000","last_clean":"2024-09-17T08:23:38.791051+0000","last_became_active":"2024-09-17T08:20:07.223417+0000","last_became_peered":"2024-09-17T08:20:07.223417+0000","last_unstale":"2024-09-17T08:23:38.791051+0000","last_undegraded":"2024-09-17T08:23:38.791051+0000","last_fullsized":"2024-09-17T08:23:38.791051+0000","mapping_epoch":221,"log_start":"267'11300","ondisk_log_start":"267'11300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11785","last_scrub_stamp":"2024-09-17T08:22:37.068424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.068424+0000","objects_scrubbed":4475,"log_size":4928,"log_dups_size":0,"ondisk_log_size":4928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.205656+0000","scrub_duration":1.4410351079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16228,"num_write_kb":519296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":327,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525328+0000","last_change":"2024-09-17T08:22:29.100509+0000","last_active":"2024-09-17T08:23:36.525328+0000","last_peered":"2024-09-17T08:23:36.525328+0000","last_clean":"2024-09-17T08:23:36.525328+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:23:36.525328+0000","last_undegraded":"2024-09-17T08:23:36.525328+0000","last_fullsized":"2024-09-17T08:23:36.525328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:29.100364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.100364+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.213754+0000","scrub_duration":0.023929505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.511829+0000","last_change":"2024-09-17T08:22:24.597248+0000","last_active":"2024-09-17T08:23:36.511829+0000","last_peered":"2024-09-17T08:23:36.511829+0000","last_clean":"2024-09-17T08:23:36.511829+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:23:36.511829+0000","last_undegraded":"2024-09-17T08:23:36.511829+0000","last_fullsized":"2024-09-17T08:23:36.511829+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:24.597137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:22:24.597137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.974690+0000","scrub_duration":0.0075741749999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":421,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:37.993665+0000","last_change":"2024-09-17T08:23:24.604045+0000","last_active":"2024-09-17T08:23:37.993665+0000","last_peered":"2024-09-17T08:23:37.993665+0000","last_clean":"2024-09-17T08:23:37.993665+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:23:37.993665+0000","last_undegraded":"2024-09-17T08:23:37.993665+0000","last_fullsized":"2024-09-17T08:23:37.993665+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:23:24.603891+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:23:24.603891+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:34.799076+0000","scrub_duration":0.011281249,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":316,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526696+0000","last_change":"2024-09-17T08:23:25.473468+0000","last_active":"2024-09-17T08:23:36.526696+0000","last_peered":"2024-09-17T08:23:36.526696+0000","last_clean":"2024-09-17T08:23:36.526696+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:23:36.526696+0000","last_undegraded":"2024-09-17T08:23:36.526696+0000","last_fullsized":"2024-09-17T08:23:36.526696+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.473322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.473322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:51.297297+0000","scrub_duration":0.005682973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"281'16862","reported_seq":18500,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.804103+0000","last_change":"2024-09-17T08:23:12.383063+0000","last_active":"2024-09-17T08:23:38.804103+0000","last_peered":"2024-09-17T08:23:38.804103+0000","last_clean":"2024-09-17T08:23:38.804103+0000","last_became_active":"2024-09-17T08:20:32.365928+0000","last_became_peered":"2024-09-17T08:20:32.365928+0000","last_unstale":"2024-09-17T08:23:38.804103+0000","last_undegraded":"2024-09-17T08:23:38.804103+0000","last_fullsized":"2024-09-17T08:23:38.804103+0000","mapping_epoch":230,"log_start":"267'12000","ondisk_log_start":"267'12000","created":221,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'14827","last_scrub_stamp":"2024-09-17T08:23:12.382359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:23:12.382359+0000","objects_scrubbed":2062,"log_size":4862,"log_dups_size":0,"ondisk_log_size":4862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:29.843696+0000","scrub_duration":0.65965772499999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":539584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":49152000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":168,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522893+0000","last_change":"2024-09-17T08:22:27.019007+0000","last_active":"2024-09-17T08:23:36.522893+0000","last_peered":"2024-09-17T08:23:36.522893+0000","last_clean":"2024-09-17T08:23:36.522893+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:23:36.522893+0000","last_undegraded":"2024-09-17T08:23:36.522893+0000","last_fullsized":"2024-09-17T08:23:36.522893+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:27.018869+0000","last_deep_scrub":"105'22","last_deep_scrub_stamp":"2024-09-17T08:12:51.952706+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.018869+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034750470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510381+0000","last_change":"2024-09-17T08:22:05.559395+0000","last_active":"2024-09-17T08:23:36.510381+0000","last_peered":"2024-09-17T08:23:36.510381+0000","last_clean":"2024-09-17T08:23:36.510381+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:23:36.510381+0000","last_undegraded":"2024-09-17T08:23:36.510381+0000","last_fullsized":"2024-09-17T08:23:36.510381+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:05.559268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:05.559268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010154139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":458,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:37.995591+0000","last_change":"2024-09-17T08:21:35.884656+0000","last_active":"2024-09-17T08:23:37.995591+0000","last_peered":"2024-09-17T08:23:37.995591+0000","last_clean":"2024-09-17T08:23:37.995591+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:23:37.995591+0000","last_undegraded":"2024-09-17T08:23:37.995591+0000","last_fullsized":"2024-09-17T08:23:37.995591+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:21:35.884516+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086332900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514489+0000","last_change":"2024-09-17T08:22:20.469387+0000","last_active":"2024-09-17T08:23:36.514489+0000","last_peered":"2024-09-17T08:23:36.514489+0000","last_clean":"2024-09-17T08:23:36.514489+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:23:36.514489+0000","last_undegraded":"2024-09-17T08:23:36.514489+0000","last_fullsized":"2024-09-17T08:23:36.514489+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.469252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.469252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085710660000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"281'16776","reported_seq":16943,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.787985+0000","last_change":"2024-09-17T08:22:42.795845+0000","last_active":"2024-09-17T08:23:38.787985+0000","last_peered":"2024-09-17T08:23:38.787985+0000","last_clean":"2024-09-17T08:23:38.787985+0000","last_became_active":"2024-09-17T08:20:07.153440+0000","last_became_peered":"2024-09-17T08:20:07.153440+0000","last_unstale":"2024-09-17T08:23:38.787985+0000","last_undegraded":"2024-09-17T08:23:38.787985+0000","last_fullsized":"2024-09-17T08:23:38.787985+0000","mapping_epoch":221,"log_start":"267'12100","ondisk_log_start":"267'12100","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12590","last_scrub_stamp":"2024-09-17T08:22:42.795634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:42.795634+0000","objects_scrubbed":4222,"log_size":4676,"log_dups_size":0,"ondisk_log_size":4676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.287069+0000","scrub_duration":1.2865534140000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16776,"num_write_kb":536832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":365,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526847+0000","last_change":"2024-09-17T08:22:14.033870+0000","last_active":"2024-09-17T08:23:36.526847+0000","last_peered":"2024-09-17T08:23:36.526847+0000","last_clean":"2024-09-17T08:23:36.526847+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:23:36.526847+0000","last_undegraded":"2024-09-17T08:23:36.526847+0000","last_fullsized":"2024-09-17T08:23:36.526847+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:22:14.033529+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.033529+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038854010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":306,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515370+0000","last_change":"2024-09-17T08:21:23.559857+0000","last_active":"2024-09-17T08:23:36.515370+0000","last_peered":"2024-09-17T08:23:36.515370+0000","last_clean":"2024-09-17T08:23:36.515370+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:23:36.515370+0000","last_undegraded":"2024-09-17T08:23:36.515370+0000","last_fullsized":"2024-09-17T08:23:36.515370+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:23.559763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:23.559763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052019450000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":187,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525682+0000","last_change":"2024-09-17T08:22:30.071610+0000","last_active":"2024-09-17T08:23:36.525682+0000","last_peered":"2024-09-17T08:23:36.525682+0000","last_clean":"2024-09-17T08:23:36.525682+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:23:36.525682+0000","last_undegraded":"2024-09-17T08:23:36.525682+0000","last_fullsized":"2024-09-17T08:23:36.525682+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:30.071480+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.071480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047153050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":282,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513074+0000","last_change":"2024-09-17T08:23:18.845122+0000","last_active":"2024-09-17T08:23:36.513074+0000","last_peered":"2024-09-17T08:23:36.513074+0000","last_clean":"2024-09-17T08:23:36.513074+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:23:36.513074+0000","last_undegraded":"2024-09-17T08:23:36.513074+0000","last_fullsized":"2024-09-17T08:23:36.513074+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:18.845030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:18.845030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:42.610373+0000","scrub_duration":0.002003562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"281'16472","reported_seq":16654,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.774638+0000","last_change":"2024-09-17T08:22:29.304499+0000","last_active":"2024-09-17T08:23:38.774638+0000","last_peered":"2024-09-17T08:23:38.774638+0000","last_clean":"2024-09-17T08:23:38.774638+0000","last_became_active":"2024-09-17T08:20:07.153490+0000","last_became_peered":"2024-09-17T08:20:07.153490+0000","last_unstale":"2024-09-17T08:23:38.774638+0000","last_undegraded":"2024-09-17T08:23:38.774638+0000","last_fullsized":"2024-09-17T08:23:38.774638+0000","mapping_epoch":221,"log_start":"267'11600","ondisk_log_start":"267'11600","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11539","last_scrub_stamp":"2024-09-17T08:22:29.304266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.304266+0000","objects_scrubbed":4996,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1.700727825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":328,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524304+0000","last_change":"2024-09-17T08:22:20.997014+0000","last_active":"2024-09-17T08:23:36.524304+0000","last_peered":"2024-09-17T08:23:36.524304+0000","last_clean":"2024-09-17T08:23:36.524304+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:23:36.524304+0000","last_undegraded":"2024-09-17T08:23:36.524304+0000","last_fullsized":"2024-09-17T08:23:36.524304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:22:20.996746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.996746+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.038376+0000","scrub_duration":0.0061057289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516745+0000","last_change":"2024-09-17T08:23:18.669616+0000","last_active":"2024-09-17T08:23:36.516745+0000","last_peered":"2024-09-17T08:23:36.516745+0000","last_clean":"2024-09-17T08:23:36.516745+0000","last_became_active":"2024-09-17T08:23:18.669266+0000","last_became_peered":"2024-09-17T08:23:18.669266+0000","last_unstale":"2024-09-17T08:23:36.516745+0000","last_undegraded":"2024-09-17T08:23:36.516745+0000","last_fullsized":"2024-09-17T08:23:36.516745+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:20.368411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.793906+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.368411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039563619999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":490,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.942725+0000","last_change":"2024-09-17T08:22:07.339466+0000","last_active":"2024-09-17T08:23:40.942725+0000","last_peered":"2024-09-17T08:23:40.942725+0000","last_clean":"2024-09-17T08:23:40.942725+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:23:40.942725+0000","last_undegraded":"2024-09-17T08:23:40.942725+0000","last_fullsized":"2024-09-17T08:23:40.942725+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:22:07.339335+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:22:07.339335+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011996602,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":309,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515265+0000","last_change":"2024-09-17T08:23:35.496064+0000","last_active":"2024-09-17T08:23:36.515265+0000","last_peered":"2024-09-17T08:23:36.515265+0000","last_clean":"2024-09-17T08:23:36.515265+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:23:36.515265+0000","last_undegraded":"2024-09-17T08:23:36.515265+0000","last_fullsized":"2024-09-17T08:23:36.515265+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:23:35.495779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:55.567597+0000","scrub_duration":0.0094404139999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"281'16338","reported_seq":16505,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.814031+0000","last_change":"2024-09-17T08:22:39.712172+0000","last_active":"2024-09-17T08:23:38.814031+0000","last_peered":"2024-09-17T08:23:38.814031+0000","last_clean":"2024-09-17T08:23:38.814031+0000","last_became_active":"2024-09-17T08:20:07.346251+0000","last_became_peered":"2024-09-17T08:20:07.346251+0000","last_unstale":"2024-09-17T08:23:38.814031+0000","last_undegraded":"2024-09-17T08:23:38.814031+0000","last_fullsized":"2024-09-17T08:23:38.814031+0000","mapping_epoch":221,"log_start":"267'11700","ondisk_log_start":"267'11700","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12088","last_scrub_stamp":"2024-09-17T08:22:39.711942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.711942+0000","objects_scrubbed":4288,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.633790+0000","scrub_duration":1.2802566520000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16338,"num_write_kb":522753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":336,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512753+0000","last_change":"2024-09-17T08:21:26.037150+0000","last_active":"2024-09-17T08:23:36.512753+0000","last_peered":"2024-09-17T08:23:36.512753+0000","last_clean":"2024-09-17T08:23:36.512753+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:23:36.512753+0000","last_undegraded":"2024-09-17T08:23:36.512753+0000","last_fullsized":"2024-09-17T08:23:36.512753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:21:26.036963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:26.036963+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038814650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515040+0000","last_change":"2024-09-17T08:22:15.514138+0000","last_active":"2024-09-17T08:23:36.515040+0000","last_peered":"2024-09-17T08:23:36.515040+0000","last_clean":"2024-09-17T08:23:36.515040+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:23:36.515040+0000","last_undegraded":"2024-09-17T08:23:36.515040+0000","last_fullsized":"2024-09-17T08:23:36.515040+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:15.514031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.514031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041206280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":295,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524171+0000","last_change":"2024-09-17T08:23:32.530792+0000","last_active":"2024-09-17T08:23:36.524171+0000","last_peered":"2024-09-17T08:23:36.524171+0000","last_clean":"2024-09-17T08:23:36.524171+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:23:36.524171+0000","last_undegraded":"2024-09-17T08:23:36.524171+0000","last_fullsized":"2024-09-17T08:23:36.524171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:32.530685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:23:32.530685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:44.832818+0000","scrub_duration":0.0016463910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":308,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524888+0000","last_change":"2024-09-17T08:23:24.428983+0000","last_active":"2024-09-17T08:23:36.524888+0000","last_peered":"2024-09-17T08:23:36.524888+0000","last_clean":"2024-09-17T08:23:36.524888+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:23:36.524888+0000","last_undegraded":"2024-09-17T08:23:36.524888+0000","last_fullsized":"2024-09-17T08:23:36.524888+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:24.428849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:24.428849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:26.101445+0000","scrub_duration":0.0050146410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"281'20256","reported_seq":20666,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.800530+0000","last_change":"2024-09-17T08:22:54.434167+0000","last_active":"2024-09-17T08:23:38.800530+0000","last_peered":"2024-09-17T08:23:38.800530+0000","last_clean":"2024-09-17T08:23:38.800530+0000","last_became_active":"2024-09-17T08:20:07.153814+0000","last_became_peered":"2024-09-17T08:20:07.153814+0000","last_unstale":"2024-09-17T08:23:38.800530+0000","last_undegraded":"2024-09-17T08:23:38.800530+0000","last_fullsized":"2024-09-17T08:23:38.800530+0000","mapping_epoch":221,"log_start":"271'15603","ondisk_log_start":"271'15603","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'16933","last_scrub_stamp":"2024-09-17T08:22:54.433940+0000","last_deep_scrub":"271'16933","last_deep_scrub_stamp":"2024-09-17T08:22:54.433940+0000","last_clean_scrub_stamp":"2024-09-17T08:22:54.433940+0000","objects_scrubbed":3978,"log_size":4653,"log_dups_size":0,"ondisk_log_size":4653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":23,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:02.333157+0000","scrub_duration":22.034200755000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":520896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":359,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526438+0000","last_change":"2024-09-17T08:23:32.863437+0000","last_active":"2024-09-17T08:23:36.526438+0000","last_peered":"2024-09-17T08:23:36.526438+0000","last_clean":"2024-09-17T08:23:36.526438+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:23:36.526438+0000","last_undegraded":"2024-09-17T08:23:36.526438+0000","last_fullsized":"2024-09-17T08:23:36.526438+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:23:32.862414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:32.862414+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:37.642249+0000","scrub_duration":0.016919605000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":304,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.520845+0000","last_change":"2024-09-17T08:21:27.406621+0000","last_active":"2024-09-17T08:23:36.520845+0000","last_peered":"2024-09-17T08:23:36.520845+0000","last_clean":"2024-09-17T08:23:36.520845+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:23:36.520845+0000","last_undegraded":"2024-09-17T08:23:36.520845+0000","last_fullsized":"2024-09-17T08:23:36.520845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:21:27.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.406373+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043336249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":251,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523809+0000","last_change":"2024-09-17T08:22:27.034592+0000","last_active":"2024-09-17T08:23:36.523809+0000","last_peered":"2024-09-17T08:23:36.523809+0000","last_clean":"2024-09-17T08:23:36.523809+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:23:36.523809+0000","last_undegraded":"2024-09-17T08:23:36.523809+0000","last_fullsized":"2024-09-17T08:23:36.523809+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.034476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.034476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053609369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":192,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510508+0000","last_change":"2024-09-17T08:21:13.406948+0000","last_active":"2024-09-17T08:23:36.510508+0000","last_peered":"2024-09-17T08:23:36.510508+0000","last_clean":"2024-09-17T08:23:36.510508+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:23:36.510508+0000","last_undegraded":"2024-09-17T08:23:36.510508+0000","last_fullsized":"2024-09-17T08:23:36.510508+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:13.406818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:13.406818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039848110000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"281'16526","reported_seq":16694,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.802461+0000","last_change":"2024-09-17T08:22:40.874195+0000","last_active":"2024-09-17T08:23:38.802461+0000","last_peered":"2024-09-17T08:23:38.802461+0000","last_clean":"2024-09-17T08:23:38.802461+0000","last_became_active":"2024-09-17T08:20:07.346503+0000","last_became_peered":"2024-09-17T08:20:07.346503+0000","last_unstale":"2024-09-17T08:23:38.802461+0000","last_undegraded":"2024-09-17T08:23:38.802461+0000","last_fullsized":"2024-09-17T08:23:38.802461+0000","mapping_epoch":221,"log_start":"267'11600","ondisk_log_start":"267'11600","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12220","last_scrub_stamp":"2024-09-17T08:22:40.873622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.873622+0000","objects_scrubbed":4349,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.542314+0000","scrub_duration":1.240716989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16526,"num_write_kb":528832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":209,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.511752+0000","last_change":"2024-09-17T08:21:20.455004+0000","last_active":"2024-09-17T08:23:36.511752+0000","last_peered":"2024-09-17T08:23:36.511752+0000","last_clean":"2024-09-17T08:23:36.511752+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:23:36.511752+0000","last_undegraded":"2024-09-17T08:23:36.511752+0000","last_fullsized":"2024-09-17T08:23:36.511752+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:21:20.454861+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:21:20.454861+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043903680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514435+0000","last_change":"2024-09-17T08:22:17.515205+0000","last_active":"2024-09-17T08:23:36.514435+0000","last_peered":"2024-09-17T08:23:36.514435+0000","last_clean":"2024-09-17T08:23:36.514435+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:23:36.514435+0000","last_undegraded":"2024-09-17T08:23:36.514435+0000","last_fullsized":"2024-09-17T08:23:36.514435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:17.515085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.515085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001995199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":481,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.793576+0000","last_change":"2024-09-17T08:23:28.852946+0000","last_active":"2024-09-17T08:23:40.793576+0000","last_peered":"2024-09-17T08:23:40.793576+0000","last_clean":"2024-09-17T08:23:40.793576+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:23:40.793576+0000","last_undegraded":"2024-09-17T08:23:40.793576+0000","last_fullsized":"2024-09-17T08:23:40.793576+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.852455+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:41.112497+0000","scrub_duration":0.0028496810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":309,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524116+0000","last_change":"2024-09-17T08:23:22.948251+0000","last_active":"2024-09-17T08:23:36.524116+0000","last_peered":"2024-09-17T08:23:36.524116+0000","last_clean":"2024-09-17T08:23:36.524116+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:23:36.524116+0000","last_undegraded":"2024-09-17T08:23:36.524116+0000","last_fullsized":"2024-09-17T08:23:36.524116+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:22.948133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:22.948133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:31.685002+0000","scrub_duration":0.0057884590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"281'16472","reported_seq":16644,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.732888+0000","last_change":"2024-09-17T08:22:37.976919+0000","last_active":"2024-09-17T08:23:38.732888+0000","last_peered":"2024-09-17T08:23:38.732888+0000","last_clean":"2024-09-17T08:23:38.732888+0000","last_became_active":"2024-09-17T08:20:07.345130+0000","last_became_peered":"2024-09-17T08:20:07.345130+0000","last_unstale":"2024-09-17T08:23:38.732888+0000","last_undegraded":"2024-09-17T08:23:38.732888+0000","last_fullsized":"2024-09-17T08:23:38.732888+0000","mapping_epoch":221,"log_start":"265'10500","ondisk_log_start":"265'10500","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12113","last_scrub_stamp":"2024-09-17T08:22:37.976676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:37.976676+0000","objects_scrubbed":4399,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:45.418342+0000","scrub_duration":1.1806302419999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":527104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":337,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.533724+0000","last_change":"2024-09-17T08:21:14.458315+0000","last_active":"2024-09-17T08:23:40.533724+0000","last_peered":"2024-09-17T08:23:40.533724+0000","last_clean":"2024-09-17T08:23:40.533724+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:23:40.533724+0000","last_undegraded":"2024-09-17T08:23:40.533724+0000","last_fullsized":"2024-09-17T08:23:40.533724+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:21:14.458167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.458167+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024233291000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510460+0000","last_change":"2024-09-17T08:23:28.072844+0000","last_active":"2024-09-17T08:23:36.510460+0000","last_peered":"2024-09-17T08:23:36.510460+0000","last_clean":"2024-09-17T08:23:36.510460+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:23:36.510460+0000","last_undegraded":"2024-09-17T08:23:36.510460+0000","last_fullsized":"2024-09-17T08:23:36.510460+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:28.072608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.072608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:45.936290+0000","scrub_duration":0.0079140149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":313,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526591+0000","last_change":"2024-09-17T08:22:18.168279+0000","last_active":"2024-09-17T08:23:36.526591+0000","last_peered":"2024-09-17T08:23:36.526591+0000","last_clean":"2024-09-17T08:23:36.526591+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:23:36.526591+0000","last_undegraded":"2024-09-17T08:23:36.526591+0000","last_fullsized":"2024-09-17T08:23:36.526591+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.168157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.168157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039078109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":293,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513167+0000","last_change":"2024-09-17T08:23:32.936724+0000","last_active":"2024-09-17T08:23:36.513167+0000","last_peered":"2024-09-17T08:23:36.513167+0000","last_clean":"2024-09-17T08:23:36.513167+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:23:36.513167+0000","last_undegraded":"2024-09-17T08:23:36.513167+0000","last_fullsized":"2024-09-17T08:23:36.513167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:23:32.936595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:34.127965+0000","scrub_duration":0.0020145649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"281'16360","reported_seq":16542,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.789724+0000","last_change":"2024-09-17T08:22:31.345117+0000","last_active":"2024-09-17T08:23:38.789724+0000","last_peered":"2024-09-17T08:23:38.789724+0000","last_clean":"2024-09-17T08:23:38.789724+0000","last_became_active":"2024-09-17T08:20:07.347116+0000","last_became_peered":"2024-09-17T08:20:07.347116+0000","last_unstale":"2024-09-17T08:23:38.789724+0000","last_undegraded":"2024-09-17T08:23:38.789724+0000","last_fullsized":"2024-09-17T08:23:38.789724+0000","mapping_epoch":221,"log_start":"267'11400","ondisk_log_start":"267'11400","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11550","last_scrub_stamp":"2024-09-17T08:22:31.344820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.344820+0000","objects_scrubbed":4856,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1.7801603639999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":195,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513990+0000","last_change":"2024-09-17T08:22:08.570430+0000","last_active":"2024-09-17T08:23:36.513990+0000","last_peered":"2024-09-17T08:23:36.513990+0000","last_clean":"2024-09-17T08:23:36.513990+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:23:36.513990+0000","last_undegraded":"2024-09-17T08:23:36.513990+0000","last_fullsized":"2024-09-17T08:23:36.513990+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:22:08.569507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:08.569507+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010760135000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":689,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:42.185934+0000","last_change":"2024-09-17T08:21:21.456425+0000","last_active":"2024-09-17T08:23:42.185934+0000","last_peered":"2024-09-17T08:23:42.185934+0000","last_clean":"2024-09-17T08:23:42.185934+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:23:42.185934+0000","last_undegraded":"2024-09-17T08:23:42.185934+0000","last_fullsized":"2024-09-17T08:23:42.185934+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:21:21.455503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.455503+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015087046,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":491,"num_read_kb":486,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":272,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.521841+0000","last_change":"2024-09-17T08:21:21.794874+0000","last_active":"2024-09-17T08:23:36.521841+0000","last_peered":"2024-09-17T08:23:36.521841+0000","last_clean":"2024-09-17T08:23:36.521841+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:23:36.521841+0000","last_undegraded":"2024-09-17T08:23:36.521841+0000","last_fullsized":"2024-09-17T08:23:36.521841+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:21.794753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:21:21.794753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020168569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":280,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522152+0000","last_change":"2024-09-17T08:22:09.870585+0000","last_active":"2024-09-17T08:23:36.522152+0000","last_peered":"2024-09-17T08:23:36.522152+0000","last_clean":"2024-09-17T08:23:36.522152+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:23:36.522152+0000","last_undegraded":"2024-09-17T08:23:36.522152+0000","last_fullsized":"2024-09-17T08:23:36.522152+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:09.870413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.870413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021174589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":304,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.517098+0000","last_change":"2024-09-17T08:22:40.481597+0000","last_active":"2024-09-17T08:23:36.517098+0000","last_peered":"2024-09-17T08:23:36.517098+0000","last_clean":"2024-09-17T08:23:36.517098+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:23:36.517098+0000","last_undegraded":"2024-09-17T08:23:36.517098+0000","last_fullsized":"2024-09-17T08:23:36.517098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.481384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.481384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.087943+0000","scrub_duration":0.0030329860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"281'16222","reported_seq":16384,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.801276+0000","last_change":"2024-09-17T08:22:43.087899+0000","last_active":"2024-09-17T08:23:38.801276+0000","last_peered":"2024-09-17T08:23:38.801276+0000","last_clean":"2024-09-17T08:23:38.801276+0000","last_became_active":"2024-09-17T08:20:07.346820+0000","last_became_peered":"2024-09-17T08:20:07.346820+0000","last_unstale":"2024-09-17T08:23:38.801276+0000","last_undegraded":"2024-09-17T08:23:38.801276+0000","last_fullsized":"2024-09-17T08:23:38.801276+0000","mapping_epoch":221,"log_start":"265'10300","ondisk_log_start":"265'10300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'12231","last_scrub_stamp":"2024-09-17T08:22:43.087628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:43.087628+0000","objects_scrubbed":4033,"log_size":5922,"log_dups_size":0,"ondisk_log_size":5922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:44.864552+0000","scrub_duration":1.249210038,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":519104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":415,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526577+0000","last_change":"2024-09-17T08:23:16.938787+0000","last_active":"2024-09-17T08:23:36.526577+0000","last_peered":"2024-09-17T08:23:36.526577+0000","last_clean":"2024-09-17T08:23:36.526577+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:23:36.526577+0000","last_undegraded":"2024-09-17T08:23:36.526577+0000","last_fullsized":"2024-09-17T08:23:36.526577+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:23:16.938654+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:23:16.938654+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:34.179589+0000","scrub_duration":0.0055767519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":317,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524216+0000","last_change":"2024-09-17T08:23:13.874360+0000","last_active":"2024-09-17T08:23:36.524216+0000","last_peered":"2024-09-17T08:23:36.524216+0000","last_clean":"2024-09-17T08:23:36.524216+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:23:36.524216+0000","last_undegraded":"2024-09-17T08:23:36.524216+0000","last_fullsized":"2024-09-17T08:23:36.524216+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:23:13.873868+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:23:13.873868+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:14.936885+0000","scrub_duration":0.003755623,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":317,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515436+0000","last_change":"2024-09-17T08:21:33.434991+0000","last_active":"2024-09-17T08:23:36.515436+0000","last_peered":"2024-09-17T08:23:36.515436+0000","last_clean":"2024-09-17T08:23:36.515436+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:23:36.515436+0000","last_undegraded":"2024-09-17T08:23:36.515436+0000","last_fullsized":"2024-09-17T08:23:36.515436+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.434904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.434904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037511929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514059+0000","last_change":"2024-09-17T08:23:25.014558+0000","last_active":"2024-09-17T08:23:36.514059+0000","last_peered":"2024-09-17T08:23:36.514059+0000","last_clean":"2024-09-17T08:23:36.514059+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:23:36.514059+0000","last_undegraded":"2024-09-17T08:23:36.514059+0000","last_fullsized":"2024-09-17T08:23:36.514059+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.014448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.014448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:45.687648+0000","scrub_duration":0.015526198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514199+0000","last_change":"2024-09-17T08:21:34.401415+0000","last_active":"2024-09-17T08:23:36.514199+0000","last_peered":"2024-09-17T08:23:36.514199+0000","last_clean":"2024-09-17T08:23:36.514199+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:23:36.514199+0000","last_undegraded":"2024-09-17T08:23:36.514199+0000","last_fullsized":"2024-09-17T08:23:36.514199+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:34.401099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.401099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010133837999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"281'16340","reported_seq":16520,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.802202+0000","last_change":"2024-09-17T08:22:32.253807+0000","last_active":"2024-09-17T08:23:38.802202+0000","last_peered":"2024-09-17T08:23:38.802202+0000","last_clean":"2024-09-17T08:23:38.802202+0000","last_became_active":"2024-09-17T08:20:07.345502+0000","last_became_peered":"2024-09-17T08:20:07.345502+0000","last_unstale":"2024-09-17T08:23:38.802202+0000","last_undegraded":"2024-09-17T08:23:38.802202+0000","last_fullsized":"2024-09-17T08:23:38.802202+0000","mapping_epoch":221,"log_start":"267'11500","ondisk_log_start":"267'11500","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11656","last_scrub_stamp":"2024-09-17T08:22:32.253689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:32.253689+0000","objects_scrubbed":4756,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1.6774553399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16340,"num_write_kb":522880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":299,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526006+0000","last_change":"2024-09-17T08:23:20.405772+0000","last_active":"2024-09-17T08:23:36.526006+0000","last_peered":"2024-09-17T08:23:36.526006+0000","last_clean":"2024-09-17T08:23:36.526006+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:23:36.526006+0000","last_undegraded":"2024-09-17T08:23:36.526006+0000","last_fullsized":"2024-09-17T08:23:36.526006+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:23:20.405620+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:23:20.405620+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:48.167616+0000","scrub_duration":0.0051583360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":295,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514140+0000","last_change":"2024-09-17T08:21:34.970246+0000","last_active":"2024-09-17T08:23:36.514140+0000","last_peered":"2024-09-17T08:23:36.514140+0000","last_clean":"2024-09-17T08:23:36.514140+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:23:36.514140+0000","last_undegraded":"2024-09-17T08:23:36.514140+0000","last_fullsized":"2024-09-17T08:23:36.514140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:21:34.969971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:34.969971+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041400559999999996,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522139+0000","last_change":"2024-09-17T08:22:14.962129+0000","last_active":"2024-09-17T08:23:36.522139+0000","last_peered":"2024-09-17T08:23:36.522139+0000","last_clean":"2024-09-17T08:23:36.522139+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:23:36.522139+0000","last_undegraded":"2024-09-17T08:23:36.522139+0000","last_fullsized":"2024-09-17T08:23:36.522139+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.962019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.962019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046754020000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":560,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515576+0000","last_change":"2024-09-17T08:22:13.427742+0000","last_active":"2024-09-17T08:23:36.515576+0000","last_peered":"2024-09-17T08:23:36.515576+0000","last_clean":"2024-09-17T08:23:36.515576+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:23:36.515576+0000","last_undegraded":"2024-09-17T08:23:36.515576+0000","last_fullsized":"2024-09-17T08:23:36.515576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:22:13.427622+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:17:47.209749+0000","last_clean_scrub_stamp":"2024-09-17T08:22:13.427622+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033528830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":299,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516801+0000","last_change":"2024-09-17T08:21:15.369223+0000","last_active":"2024-09-17T08:23:36.516801+0000","last_peered":"2024-09-17T08:23:36.516801+0000","last_clean":"2024-09-17T08:23:36.516801+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:23:36.516801+0000","last_undegraded":"2024-09-17T08:23:36.516801+0000","last_fullsized":"2024-09-17T08:23:36.516801+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:15.368937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:37.370527+0000","last_clean_scrub_stamp":"2024-09-17T08:21:15.368937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021146210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":198,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.943255+0000","last_change":"2024-09-17T08:23:36.011912+0000","last_active":"2024-09-17T08:23:40.943255+0000","last_peered":"2024-09-17T08:23:40.943255+0000","last_clean":"2024-09-17T08:23:40.943255+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:23:40.943255+0000","last_undegraded":"2024-09-17T08:23:40.943255+0000","last_fullsized":"2024-09-17T08:23:40.943255+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:36.011749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:36.011749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:46.453195+0000","scrub_duration":0.017262231999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.7","version":"281'16580","reported_seq":16784,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.782651+0000","last_change":"2024-09-17T08:22:17.960994+0000","last_active":"2024-09-17T08:23:38.782651+0000","last_peered":"2024-09-17T08:23:38.782651+0000","last_clean":"2024-09-17T08:23:38.782651+0000","last_became_active":"2024-09-17T08:20:07.345480+0000","last_became_peered":"2024-09-17T08:20:07.345480+0000","last_unstale":"2024-09-17T08:23:38.782651+0000","last_undegraded":"2024-09-17T08:23:38.782651+0000","last_fullsized":"2024-09-17T08:23:38.782651+0000","mapping_epoch":221,"log_start":"267'11900","ondisk_log_start":"267'11900","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'10727","last_scrub_stamp":"2024-09-17T08:22:17.960812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:17.960812+0000","objects_scrubbed":5910,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1.5674201079999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16580,"num_write_kb":530560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":136,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510533+0000","last_change":"2024-09-17T08:23:18.671333+0000","last_active":"2024-09-17T08:23:36.510533+0000","last_peered":"2024-09-17T08:23:36.510533+0000","last_clean":"2024-09-17T08:23:36.510533+0000","last_became_active":"2024-09-17T08:23:18.670977+0000","last_became_peered":"2024-09-17T08:23:18.670977+0000","last_unstale":"2024-09-17T08:23:36.510533+0000","last_undegraded":"2024-09-17T08:23:36.510533+0000","last_fullsized":"2024-09-17T08:23:36.510533+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:22:25.348962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.348962+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.099284+0000","scrub_duration":0.0021471250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":261,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522560+0000","last_change":"2024-09-17T08:21:14.706611+0000","last_active":"2024-09-17T08:23:36.522560+0000","last_peered":"2024-09-17T08:23:36.522560+0000","last_clean":"2024-09-17T08:23:36.522560+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:23:36.522560+0000","last_undegraded":"2024-09-17T08:23:36.522560+0000","last_fullsized":"2024-09-17T08:23:36.522560+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:14.706481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:21:14.706481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045730429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":278,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522073+0000","last_change":"2024-09-17T08:23:35.663713+0000","last_active":"2024-09-17T08:23:36.522073+0000","last_peered":"2024-09-17T08:23:36.522073+0000","last_clean":"2024-09-17T08:23:36.522073+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:23:36.522073+0000","last_undegraded":"2024-09-17T08:23:36.522073+0000","last_fullsized":"2024-09-17T08:23:36.522073+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:35.663585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:23:35.663585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:38.526350+0000","scrub_duration":0.0039607879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":297,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515502+0000","last_change":"2024-09-17T08:21:18.452335+0000","last_active":"2024-09-17T08:23:36.515502+0000","last_peered":"2024-09-17T08:23:36.515502+0000","last_clean":"2024-09-17T08:23:36.515502+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:23:36.515502+0000","last_undegraded":"2024-09-17T08:23:36.515502+0000","last_fullsized":"2024-09-17T08:23:36.515502+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:18.452221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:18.452221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042305550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"281'16126","reported_seq":17744,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.810807+0000","last_change":"2024-09-17T08:23:22.030483+0000","last_active":"2024-09-17T08:23:38.810807+0000","last_peered":"2024-09-17T08:23:38.810807+0000","last_clean":"2024-09-17T08:23:38.810807+0000","last_became_active":"2024-09-17T08:20:33.313140+0000","last_became_peered":"2024-09-17T08:20:33.313140+0000","last_unstale":"2024-09-17T08:23:38.810807+0000","last_undegraded":"2024-09-17T08:23:38.810807+0000","last_fullsized":"2024-09-17T08:23:38.810807+0000","mapping_epoch":231,"log_start":"263'9300","ondisk_log_start":"263'9300","created":221,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"279'14759","last_scrub_stamp":"2024-09-17T08:23:22.030303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:23:22.030303+0000","objects_scrubbed":1382,"log_size":6826,"log_dups_size":0,"ondisk_log_size":6826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:28.104368+0000","scrub_duration":0.47411159400000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16126,"num_write_kb":516032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":726,"num_bytes_recovered":47579136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":360,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526346+0000","last_change":"2024-09-17T08:22:29.010241+0000","last_active":"2024-09-17T08:23:36.526346+0000","last_peered":"2024-09-17T08:23:36.526346+0000","last_clean":"2024-09-17T08:23:36.526346+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:23:36.526346+0000","last_undegraded":"2024-09-17T08:23:36.526346+0000","last_fullsized":"2024-09-17T08:23:36.526346+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:22:29.010088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.010088+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.705148+0000","scrub_duration":0.0044362949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514349+0000","last_change":"2024-09-17T08:22:40.621682+0000","last_active":"2024-09-17T08:23:36.514349+0000","last_peered":"2024-09-17T08:23:36.514349+0000","last_clean":"2024-09-17T08:23:36.514349+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:23:36.514349+0000","last_undegraded":"2024-09-17T08:23:36.514349+0000","last_fullsized":"2024-09-17T08:23:36.514349+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.621579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.621579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:05.471410+0000","scrub_duration":0.0048029379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":281,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525462+0000","last_change":"2024-09-17T08:22:30.062596+0000","last_active":"2024-09-17T08:23:36.525462+0000","last_peered":"2024-09-17T08:23:36.525462+0000","last_clean":"2024-09-17T08:23:36.525462+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:23:36.525462+0000","last_undegraded":"2024-09-17T08:23:36.525462+0000","last_fullsized":"2024-09-17T08:23:36.525462+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:30.062098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:22:30.062098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:58.782037+0000","scrub_duration":0.0090326120000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":308,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524145+0000","last_change":"2024-09-17T08:23:15.916351+0000","last_active":"2024-09-17T08:23:36.524145+0000","last_peered":"2024-09-17T08:23:36.524145+0000","last_clean":"2024-09-17T08:23:36.524145+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:23:36.524145+0000","last_undegraded":"2024-09-17T08:23:36.524145+0000","last_fullsized":"2024-09-17T08:23:36.524145+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:15.915725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:23:15.915725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.985380+0000","scrub_duration":0.0045721490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"281'16360","reported_seq":16542,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.790874+0000","last_change":"2024-09-17T08:22:31.507556+0000","last_active":"2024-09-17T08:23:38.790874+0000","last_peered":"2024-09-17T08:23:38.790874+0000","last_clean":"2024-09-17T08:23:38.790874+0000","last_became_active":"2024-09-17T08:20:07.346402+0000","last_became_peered":"2024-09-17T08:20:07.346402+0000","last_unstale":"2024-09-17T08:23:38.790874+0000","last_undegraded":"2024-09-17T08:23:38.790874+0000","last_fullsized":"2024-09-17T08:23:38.790874+0000","mapping_epoch":221,"log_start":"265'10400","ondisk_log_start":"265'10400","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'11557","last_scrub_stamp":"2024-09-17T08:22:31.507376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.507376+0000","objects_scrubbed":4853,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:46.972003+0000","scrub_duration":1.513714271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":353,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.943700+0000","last_change":"2024-09-17T08:22:29.458182+0000","last_active":"2024-09-17T08:23:40.943700+0000","last_peered":"2024-09-17T08:23:40.943700+0000","last_clean":"2024-09-17T08:23:40.943700+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:23:40.943700+0000","last_undegraded":"2024-09-17T08:23:40.943700+0000","last_fullsized":"2024-09-17T08:23:40.943700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:29.458054+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:22:29.458054+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040122279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":282,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523214+0000","last_change":"2024-09-17T08:22:10.889503+0000","last_active":"2024-09-17T08:23:36.523214+0000","last_peered":"2024-09-17T08:23:36.523214+0000","last_clean":"2024-09-17T08:23:36.523214+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:23:36.523214+0000","last_undegraded":"2024-09-17T08:23:36.523214+0000","last_fullsized":"2024-09-17T08:23:36.523214+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:10.889256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:10.889256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006339183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":469,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:37.994132+0000","last_change":"2024-09-17T08:22:33.388252+0000","last_active":"2024-09-17T08:23:37.994132+0000","last_peered":"2024-09-17T08:23:37.994132+0000","last_clean":"2024-09-17T08:23:37.994132+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:23:37.994132+0000","last_undegraded":"2024-09-17T08:23:37.994132+0000","last_fullsized":"2024-09-17T08:23:37.994132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.388161+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:59.477432+0000","scrub_duration":0.002025004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":135,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523731+0000","last_change":"2024-09-17T08:23:18.663195+0000","last_active":"2024-09-17T08:23:36.523731+0000","last_peered":"2024-09-17T08:23:36.523731+0000","last_clean":"2024-09-17T08:23:36.523731+0000","last_became_active":"2024-09-17T08:23:18.662935+0000","last_became_peered":"2024-09-17T08:23:18.662935+0000","last_unstale":"2024-09-17T08:23:36.523731+0000","last_undegraded":"2024-09-17T08:23:36.523731+0000","last_fullsized":"2024-09-17T08:23:36.523731+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:06.237871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.237871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"281'16318","reported_seq":16432,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:38.774816+0000","last_change":"2024-09-17T08:23:17.601158+0000","last_active":"2024-09-17T08:23:38.774816+0000","last_peered":"2024-09-17T08:23:38.774816+0000","last_clean":"2024-09-17T08:23:38.774816+0000","last_became_active":"2024-09-17T08:20:07.154215+0000","last_became_peered":"2024-09-17T08:20:07.154215+0000","last_unstale":"2024-09-17T08:23:38.774816+0000","last_undegraded":"2024-09-17T08:23:38.774816+0000","last_fullsized":"2024-09-17T08:23:38.774816+0000","mapping_epoch":221,"log_start":"267'11300","ondisk_log_start":"267'11300","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'14733","last_scrub_stamp":"2024-09-17T08:23:17.600902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:06.139182+0000","last_clean_scrub_stamp":"2024-09-17T08:23:17.600902+0000","objects_scrubbed":1617,"log_size":5018,"log_dups_size":0,"ondisk_log_size":5018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:25.519166+0000","scrub_duration":0.64697504699999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16318,"num_write_kb":522176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":371,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524238+0000","last_change":"2024-09-17T08:22:39.204171+0000","last_active":"2024-09-17T08:23:36.524238+0000","last_peered":"2024-09-17T08:23:36.524238+0000","last_clean":"2024-09-17T08:23:36.524238+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:23:36.524238+0000","last_undegraded":"2024-09-17T08:23:36.524238+0000","last_fullsized":"2024-09-17T08:23:36.524238+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:22:39.204047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.204047+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:03.816885+0000","scrub_duration":0.061163012000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":306,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525932+0000","last_change":"2024-09-17T08:22:15.001277+0000","last_active":"2024-09-17T08:23:36.525932+0000","last_peered":"2024-09-17T08:23:36.525932+0000","last_clean":"2024-09-17T08:23:36.525932+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:23:36.525932+0000","last_undegraded":"2024-09-17T08:23:36.525932+0000","last_fullsized":"2024-09-17T08:23:36.525932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:22:15.001149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:15.001149+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054858570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":286,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523446+0000","last_change":"2024-09-17T08:21:17.762815+0000","last_active":"2024-09-17T08:23:36.523446+0000","last_peered":"2024-09-17T08:23:36.523446+0000","last_clean":"2024-09-17T08:23:36.523446+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:23:36.523446+0000","last_undegraded":"2024-09-17T08:23:36.523446+0000","last_fullsized":"2024-09-17T08:23:36.523446+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:17.762302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:21:17.762302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054597830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":101,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522003+0000","last_change":"2024-09-17T08:21:32.565481+0000","last_active":"2024-09-17T08:23:36.522003+0000","last_peered":"2024-09-17T08:23:36.522003+0000","last_clean":"2024-09-17T08:23:36.522003+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:23:36.522003+0000","last_undegraded":"2024-09-17T08:23:36.522003+0000","last_fullsized":"2024-09-17T08:23:36.522003+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:32.565367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:32.565367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022451039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":325,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526492+0000","last_change":"2024-09-17T08:23:11.841207+0000","last_active":"2024-09-17T08:23:36.526492+0000","last_peered":"2024-09-17T08:23:36.526492+0000","last_clean":"2024-09-17T08:23:36.526492+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:23:36.526492+0000","last_undegraded":"2024-09-17T08:23:36.526492+0000","last_fullsized":"2024-09-17T08:23:36.526492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:23:11.841049+0000","last_deep_scrub":"85'13","last_deep_scrub_stamp":"2024-09-17T08:11:22.027118+0000","last_clean_scrub_stamp":"2024-09-17T08:23:11.841049+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:16.910133+0000","scrub_duration":0.012088834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":300,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523510+0000","last_change":"2024-09-17T08:22:21.972879+0000","last_active":"2024-09-17T08:23:36.523510+0000","last_peered":"2024-09-17T08:23:36.523510+0000","last_clean":"2024-09-17T08:23:36.523510+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:23:36.523510+0000","last_undegraded":"2024-09-17T08:23:36.523510+0000","last_fullsized":"2024-09-17T08:23:36.523510+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:21.972582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.972582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084090889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":301,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516093+0000","last_change":"2024-09-17T08:23:23.852256+0000","last_active":"2024-09-17T08:23:36.516093+0000","last_peered":"2024-09-17T08:23:36.516093+0000","last_clean":"2024-09-17T08:23:36.516093+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:23:36.516093+0000","last_undegraded":"2024-09-17T08:23:36.516093+0000","last_fullsized":"2024-09-17T08:23:36.516093+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:23.852111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:23.852111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:37.848490+0000","scrub_duration":0.006997774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":273,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522269+0000","last_change":"2024-09-17T08:21:30.595600+0000","last_active":"2024-09-17T08:23:36.522269+0000","last_peered":"2024-09-17T08:23:36.522269+0000","last_clean":"2024-09-17T08:23:36.522269+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:23:36.522269+0000","last_undegraded":"2024-09-17T08:23:36.522269+0000","last_fullsized":"2024-09-17T08:23:36.522269+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:21:30.595456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002658952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":195,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524366+0000","last_change":"2024-09-17T08:23:18.666816+0000","last_active":"2024-09-17T08:23:36.524366+0000","last_peered":"2024-09-17T08:23:36.524366+0000","last_clean":"2024-09-17T08:23:36.524366+0000","last_became_active":"2024-09-17T08:23:18.666495+0000","last_became_peered":"2024-09-17T08:23:18.666495+0000","last_unstale":"2024-09-17T08:23:36.524366+0000","last_undegraded":"2024-09-17T08:23:36.524366+0000","last_fullsized":"2024-09-17T08:23:36.524366+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:27.392168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:27.392168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:47.852339+0000","scrub_duration":0.0030218530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":348,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516286+0000","last_change":"2024-09-17T08:23:29.829264+0000","last_active":"2024-09-17T08:23:36.516286+0000","last_peered":"2024-09-17T08:23:36.516286+0000","last_clean":"2024-09-17T08:23:36.516286+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:23:36.516286+0000","last_undegraded":"2024-09-17T08:23:36.516286+0000","last_fullsized":"2024-09-17T08:23:36.516286+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:23:29.829064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:29.829064+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:53.411214+0000","scrub_duration":0.0065936679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":204,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512769+0000","last_change":"2024-09-17T08:22:22.621469+0000","last_active":"2024-09-17T08:23:36.512769+0000","last_peered":"2024-09-17T08:23:36.512769+0000","last_clean":"2024-09-17T08:23:36.512769+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:23:36.512769+0000","last_undegraded":"2024-09-17T08:23:36.512769+0000","last_fullsized":"2024-09-17T08:23:36.512769+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.621350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:41.649184+0000","scrub_duration":0.011642807999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":303,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.517006+0000","last_change":"2024-09-17T08:23:17.424922+0000","last_active":"2024-09-17T08:23:36.517006+0000","last_peered":"2024-09-17T08:23:36.517006+0000","last_clean":"2024-09-17T08:23:36.517006+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:23:36.517006+0000","last_undegraded":"2024-09-17T08:23:36.517006+0000","last_fullsized":"2024-09-17T08:23:36.517006+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:17.424502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:17.424502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:20.282960+0000","scrub_duration":0.002219094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":357,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513319+0000","last_change":"2024-09-17T08:22:12.398758+0000","last_active":"2024-09-17T08:23:36.513319+0000","last_peered":"2024-09-17T08:23:36.513319+0000","last_clean":"2024-09-17T08:23:36.513319+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:23:36.513319+0000","last_undegraded":"2024-09-17T08:23:36.513319+0000","last_fullsized":"2024-09-17T08:23:36.513319+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:12.398295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.398295+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011555360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":244,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514015+0000","last_change":"2024-09-17T08:23:26.810186+0000","last_active":"2024-09-17T08:23:36.514015+0000","last_peered":"2024-09-17T08:23:36.514015+0000","last_clean":"2024-09-17T08:23:36.514015+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:23:36.514015+0000","last_undegraded":"2024-09-17T08:23:36.514015+0000","last_fullsized":"2024-09-17T08:23:36.514015+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:26.810075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:23:26.810075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:51.910527+0000","scrub_duration":0.0080033750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":282,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.574962+0000","last_change":"2024-09-17T08:21:29.598725+0000","last_active":"2024-09-17T08:23:40.574962+0000","last_peered":"2024-09-17T08:23:40.574962+0000","last_clean":"2024-09-17T08:23:40.574962+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:23:40.574962+0000","last_undegraded":"2024-09-17T08:23:40.574962+0000","last_fullsized":"2024-09-17T08:23:40.574962+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:21:29.598601+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:21:29.598601+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021201536999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":292,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526182+0000","last_change":"2024-09-17T08:22:25.070949+0000","last_active":"2024-09-17T08:23:36.526182+0000","last_peered":"2024-09-17T08:23:36.526182+0000","last_clean":"2024-09-17T08:23:36.526182+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:23:36.526182+0000","last_undegraded":"2024-09-17T08:23:36.526182+0000","last_fullsized":"2024-09-17T08:23:36.526182+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:25.070821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:25.070821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076690669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":239,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.520868+0000","last_change":"2024-09-17T08:22:09.487846+0000","last_active":"2024-09-17T08:23:36.520868+0000","last_peered":"2024-09-17T08:23:36.520868+0000","last_clean":"2024-09-17T08:23:36.520868+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:23:36.520868+0000","last_undegraded":"2024-09-17T08:23:36.520868+0000","last_fullsized":"2024-09-17T08:23:36.520868+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:09.487639+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:22:09.487639+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094462299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":298,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.527691+0000","last_change":"2024-09-17T08:22:26.034397+0000","last_active":"2024-09-17T08:23:36.527691+0000","last_peered":"2024-09-17T08:23:36.527691+0000","last_clean":"2024-09-17T08:23:36.527691+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:23:36.527691+0000","last_undegraded":"2024-09-17T08:23:36.527691+0000","last_fullsized":"2024-09-17T08:23:36.527691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:22:26.033887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:50.236921+0000","scrub_duration":0.011752558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":475,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-09-17T08:23:40.943158+0000","last_change":"2024-09-17T08:23:13.790341+0000","last_active":"2024-09-17T08:23:40.943158+0000","last_peered":"2024-09-17T08:23:40.943158+0000","last_clean":"2024-09-17T08:23:40.943158+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:23:40.943158+0000","last_undegraded":"2024-09-17T08:23:40.943158+0000","last_fullsized":"2024-09-17T08:23:40.943158+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:23:13.786345+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:23:13.786345+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:20.202757+0000","scrub_duration":0.0036625709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":297,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525611+0000","last_change":"2024-09-17T08:23:30.540527+0000","last_active":"2024-09-17T08:23:36.525611+0000","last_peered":"2024-09-17T08:23:36.525611+0000","last_clean":"2024-09-17T08:23:36.525611+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:23:36.525611+0000","last_undegraded":"2024-09-17T08:23:36.525611+0000","last_fullsized":"2024-09-17T08:23:36.525611+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:30.540408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:30.540408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:50.599919+0000","scrub_duration":0.0023462510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":209,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.510382+0000","last_change":"2024-09-17T08:21:12.364191+0000","last_active":"2024-09-17T08:23:36.510382+0000","last_peered":"2024-09-17T08:23:36.510382+0000","last_clean":"2024-09-17T08:23:36.510382+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:23:36.510382+0000","last_undegraded":"2024-09-17T08:23:36.510382+0000","last_fullsized":"2024-09-17T08:23:36.510382+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:21:12.364059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:12.364059+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076608910000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":176,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513337+0000","last_change":"2024-09-17T08:23:18.671761+0000","last_active":"2024-09-17T08:23:36.513337+0000","last_peered":"2024-09-17T08:23:36.513337+0000","last_clean":"2024-09-17T08:23:36.513337+0000","last_became_active":"2024-09-17T08:23:18.671502+0000","last_became_peered":"2024-09-17T08:23:18.671502+0000","last_unstale":"2024-09-17T08:23:36.513337+0000","last_undegraded":"2024-09-17T08:23:36.513337+0000","last_fullsized":"2024-09-17T08:23:36.513337+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:12.321279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:12.321279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":301,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516008+0000","last_change":"2024-09-17T08:22:16.495330+0000","last_active":"2024-09-17T08:23:36.516008+0000","last_peered":"2024-09-17T08:23:36.516008+0000","last_clean":"2024-09-17T08:23:36.516008+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:23:36.516008+0000","last_undegraded":"2024-09-17T08:23:36.516008+0000","last_fullsized":"2024-09-17T08:23:36.516008+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:16.495149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:22:16.495149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001489045,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524033+0000","last_change":"2024-09-17T08:22:14.202171+0000","last_active":"2024-09-17T08:23:36.524033+0000","last_peered":"2024-09-17T08:23:36.524033+0000","last_clean":"2024-09-17T08:23:36.524033+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:23:36.524033+0000","last_undegraded":"2024-09-17T08:23:36.524033+0000","last_fullsized":"2024-09-17T08:23:36.524033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:14.201855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:14.201855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018711583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":370,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524732+0000","last_change":"2024-09-17T08:23:13.382907+0000","last_active":"2024-09-17T08:23:36.524732+0000","last_peered":"2024-09-17T08:23:36.524732+0000","last_clean":"2024-09-17T08:23:36.524732+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:23:36.524732+0000","last_undegraded":"2024-09-17T08:23:36.524732+0000","last_fullsized":"2024-09-17T08:23:36.524732+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:23:13.382756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:13.382756+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:18.001037+0000","scrub_duration":0.0048298940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514689+0000","last_change":"2024-09-17T08:22:18.499171+0000","last_active":"2024-09-17T08:23:36.514689+0000","last_peered":"2024-09-17T08:23:36.514689+0000","last_clean":"2024-09-17T08:23:36.514689+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:23:36.514689+0000","last_undegraded":"2024-09-17T08:23:36.514689+0000","last_fullsized":"2024-09-17T08:23:36.514689+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:18.499009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:22:18.499009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020240600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.511888+0000","last_change":"2024-09-17T08:23:27.084326+0000","last_active":"2024-09-17T08:23:36.511888+0000","last_peered":"2024-09-17T08:23:36.511888+0000","last_clean":"2024-09-17T08:23:36.511888+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:23:36.511888+0000","last_undegraded":"2024-09-17T08:23:36.511888+0000","last_fullsized":"2024-09-17T08:23:36.511888+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.084153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.084153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:53.339612+0000","scrub_duration":0.013939082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":281,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512022+0000","last_change":"2024-09-17T08:21:33.941971+0000","last_active":"2024-09-17T08:23:36.512022+0000","last_peered":"2024-09-17T08:23:36.512022+0000","last_clean":"2024-09-17T08:23:36.512022+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:23:36.512022+0000","last_undegraded":"2024-09-17T08:23:36.512022+0000","last_fullsized":"2024-09-17T08:23:36.512022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:33.941822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:21:33.941822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057366700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":342,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512678+0000","last_change":"2024-09-17T08:22:06.283965+0000","last_active":"2024-09-17T08:23:36.512678+0000","last_peered":"2024-09-17T08:23:36.512678+0000","last_clean":"2024-09-17T08:23:36.512678+0000","last_became_active":"2024-09-17T08:16:33.322519+0000","last_became_peered":"2024-09-17T08:16:33.322519+0000","last_unstale":"2024-09-17T08:23:36.512678+0000","last_undegraded":"2024-09-17T08:23:36.512678+0000","last_fullsized":"2024-09-17T08:23:36.512678+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:22:06.283841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:06.283841+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044434549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":171,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522221+0000","last_change":"2024-09-17T08:23:22.570064+0000","last_active":"2024-09-17T08:23:36.522221+0000","last_peered":"2024-09-17T08:23:36.522221+0000","last_clean":"2024-09-17T08:23:36.522221+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:23:36.522221+0000","last_undegraded":"2024-09-17T08:23:36.522221+0000","last_fullsized":"2024-09-17T08:23:36.522221+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:22.569630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:23:22.569630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:50.302672+0000","scrub_duration":0.0031370999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.524110+0000","last_change":"2024-09-17T08:23:18.671544+0000","last_active":"2024-09-17T08:23:36.524110+0000","last_peered":"2024-09-17T08:23:36.524110+0000","last_clean":"2024-09-17T08:23:36.524110+0000","last_became_active":"2024-09-17T08:23:18.671271+0000","last_became_peered":"2024-09-17T08:23:18.671271+0000","last_unstale":"2024-09-17T08:23:36.524110+0000","last_undegraded":"2024-09-17T08:23:36.524110+0000","last_fullsized":"2024-09-17T08:23:36.524110+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:11.444312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:23:11.444312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:32.027633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":307,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515718+0000","last_change":"2024-09-17T08:23:25.489000+0000","last_active":"2024-09-17T08:23:36.515718+0000","last_peered":"2024-09-17T08:23:36.515718+0000","last_clean":"2024-09-17T08:23:36.515718+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:23:36.515718+0000","last_undegraded":"2024-09-17T08:23:36.515718+0000","last_fullsized":"2024-09-17T08:23:36.515718+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.488862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:25.488862+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.488862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:52.549945+0000","scrub_duration":0.002265043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":316,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512277+0000","last_change":"2024-09-17T08:22:31.627432+0000","last_active":"2024-09-17T08:23:36.512277+0000","last_peered":"2024-09-17T08:23:36.512277+0000","last_clean":"2024-09-17T08:23:36.512277+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:23:36.512277+0000","last_undegraded":"2024-09-17T08:23:36.512277+0000","last_fullsized":"2024-09-17T08:23:36.512277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:31.627297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:31.627297+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:49.425907+0000","scrub_duration":0.0041108009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":143,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522960+0000","last_change":"2024-09-17T08:23:18.669551+0000","last_active":"2024-09-17T08:23:36.522960+0000","last_peered":"2024-09-17T08:23:36.522960+0000","last_clean":"2024-09-17T08:23:36.522960+0000","last_became_active":"2024-09-17T08:23:18.669127+0000","last_became_peered":"2024-09-17T08:23:18.669127+0000","last_unstale":"2024-09-17T08:23:36.522960+0000","last_undegraded":"2024-09-17T08:23:36.522960+0000","last_fullsized":"2024-09-17T08:23:36.522960+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:22:21.329888+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:22:21.329888+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010030713,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":294,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515046+0000","last_change":"2024-09-17T08:21:31.458346+0000","last_active":"2024-09-17T08:23:36.515046+0000","last_peered":"2024-09-17T08:23:36.515046+0000","last_clean":"2024-09-17T08:23:36.515046+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:23:36.515046+0000","last_undegraded":"2024-09-17T08:23:36.515046+0000","last_fullsized":"2024-09-17T08:23:36.515046+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.458200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.458200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020112709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":281,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.515929+0000","last_change":"2024-09-17T08:22:39.655694+0000","last_active":"2024-09-17T08:23:36.515929+0000","last_peered":"2024-09-17T08:23:36.515929+0000","last_clean":"2024-09-17T08:23:36.515929+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:23:36.515929+0000","last_undegraded":"2024-09-17T08:23:36.515929+0000","last_fullsized":"2024-09-17T08:23:36.515929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:39.655581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.655581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:53.268378+0000","scrub_duration":0.0042238659999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":309,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516720+0000","last_change":"2024-09-17T08:23:32.379866+0000","last_active":"2024-09-17T08:23:36.516720+0000","last_peered":"2024-09-17T08:23:36.516720+0000","last_clean":"2024-09-17T08:23:36.516720+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:23:36.516720+0000","last_undegraded":"2024-09-17T08:23:36.516720+0000","last_fullsized":"2024-09-17T08:23:36.516720+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:23:32.379674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:32.379674+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:47.937610+0000","scrub_duration":0.002887091,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.523985+0000","last_change":"2024-09-17T08:23:15.383301+0000","last_active":"2024-09-17T08:23:36.523985+0000","last_peered":"2024-09-17T08:23:36.523985+0000","last_clean":"2024-09-17T08:23:36.523985+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:23:36.523985+0000","last_undegraded":"2024-09-17T08:23:36.523985+0000","last_fullsized":"2024-09-17T08:23:36.523985+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:15.383186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:23:15.383186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:36.275565+0000","scrub_duration":0.0036412580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":327,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526409+0000","last_change":"2024-09-17T08:22:20.129709+0000","last_active":"2024-09-17T08:23:36.526409+0000","last_peered":"2024-09-17T08:23:36.526409+0000","last_clean":"2024-09-17T08:23:36.526409+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:23:36.526409+0000","last_undegraded":"2024-09-17T08:23:36.526409+0000","last_fullsized":"2024-09-17T08:23:36.526409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:22:20.129556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.129556+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010294539,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":393,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:37.993712+0000","last_change":"2024-09-17T08:23:18.424749+0000","last_active":"2024-09-17T08:23:37.993712+0000","last_peered":"2024-09-17T08:23:37.993712+0000","last_clean":"2024-09-17T08:23:37.993712+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:23:37.993712+0000","last_undegraded":"2024-09-17T08:23:37.993712+0000","last_fullsized":"2024-09-17T08:23:37.993712+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:23:18.424605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:18.424605+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:41.930241+0000","scrub_duration":0.0087369089999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514423+0000","last_change":"2024-09-17T08:23:18.949410+0000","last_active":"2024-09-17T08:23:36.514423+0000","last_peered":"2024-09-17T08:23:36.514423+0000","last_clean":"2024-09-17T08:23:36.514423+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:23:36.514423+0000","last_undegraded":"2024-09-17T08:23:36.514423+0000","last_fullsized":"2024-09-17T08:23:36.514423+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:18.949212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:18.949212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:27.357161+0000","scrub_duration":0.0077693420000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":287,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513079+0000","last_change":"2024-09-17T08:21:31.941666+0000","last_active":"2024-09-17T08:23:36.513079+0000","last_peered":"2024-09-17T08:23:36.513079+0000","last_clean":"2024-09-17T08:23:36.513079+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:23:36.513079+0000","last_undegraded":"2024-09-17T08:23:36.513079+0000","last_fullsized":"2024-09-17T08:23:36.513079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:21:31.941534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066151480000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":112,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.513916+0000","last_change":"2024-09-17T08:23:18.676582+0000","last_active":"2024-09-17T08:23:36.513916+0000","last_peered":"2024-09-17T08:23:36.513916+0000","last_clean":"2024-09-17T08:23:36.513916+0000","last_became_active":"2024-09-17T08:23:18.676257+0000","last_became_peered":"2024-09-17T08:23:18.676257+0000","last_unstale":"2024-09-17T08:23:36.513916+0000","last_undegraded":"2024-09-17T08:23:36.513916+0000","last_fullsized":"2024-09-17T08:23:36.513916+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:39.435291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.435291+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.816543+0000","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":323,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526333+0000","last_change":"2024-09-17T08:23:27.862145+0000","last_active":"2024-09-17T08:23:36.526333+0000","last_peered":"2024-09-17T08:23:36.526333+0000","last_clean":"2024-09-17T08:23:36.526333+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:23:36.526333+0000","last_undegraded":"2024-09-17T08:23:36.526333+0000","last_fullsized":"2024-09-17T08:23:36.526333+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.862032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.862032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:39.005322+0000","scrub_duration":0.0029577050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":282,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.526767+0000","last_change":"2024-09-17T08:22:33.121286+0000","last_active":"2024-09-17T08:23:36.526767+0000","last_peered":"2024-09-17T08:23:36.526767+0000","last_clean":"2024-09-17T08:23:36.526767+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:23:36.526767+0000","last_undegraded":"2024-09-17T08:23:36.526767+0000","last_fullsized":"2024-09-17T08:23:36.526767+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:33.121209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:22:33.121209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:54.285318+0000","scrub_duration":0.0012454370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":250,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525032+0000","last_change":"2024-09-17T08:23:36.477221+0000","last_active":"2024-09-17T08:23:36.525032+0000","last_peered":"2024-09-17T08:23:36.525032+0000","last_clean":"2024-09-17T08:23:36.525032+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:23:36.525032+0000","last_undegraded":"2024-09-17T08:23:36.525032+0000","last_fullsized":"2024-09-17T08:23:36.525032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:36.477146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:23:36.477146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:39.260867+0000","scrub_duration":0.0013854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":404,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.522337+0000","last_change":"2024-09-17T08:22:20.983964+0000","last_active":"2024-09-17T08:23:36.522337+0000","last_peered":"2024-09-17T08:23:36.522337+0000","last_clean":"2024-09-17T08:23:36.522337+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:23:36.522337+0000","last_undegraded":"2024-09-17T08:23:36.522337+0000","last_fullsized":"2024-09-17T08:23:36.522337+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:22:20.983823+0000","last_deep_scrub":"87'22","last_deep_scrub_stamp":"2024-09-17T08:11:34.992118+0000","last_clean_scrub_stamp":"2024-09-17T08:22:20.983823+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:48.519128+0000","scrub_duration":0.010983009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":177,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.514556+0000","last_change":"2024-09-17T08:22:22.507255+0000","last_active":"2024-09-17T08:23:36.514556+0000","last_peered":"2024-09-17T08:23:36.514556+0000","last_clean":"2024-09-17T08:23:36.514556+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:23:36.514556+0000","last_undegraded":"2024-09-17T08:23:36.514556+0000","last_fullsized":"2024-09-17T08:23:36.514556+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:22.507142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:01.636849+0000","last_clean_scrub_stamp":"2024-09-17T08:22:22.507142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:52.303965+0000","scrub_duration":0.0043685180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":282,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516024+0000","last_change":"2024-09-17T08:23:25.793139+0000","last_active":"2024-09-17T08:23:36.516024+0000","last_peered":"2024-09-17T08:23:36.516024+0000","last_clean":"2024-09-17T08:23:36.516024+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:23:36.516024+0000","last_undegraded":"2024-09-17T08:23:36.516024+0000","last_fullsized":"2024-09-17T08:23:36.516024+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.793012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.793012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:47.678897+0000","scrub_duration":0.0072178959999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":298,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.516204+0000","last_change":"2024-09-17T08:23:27.801465+0000","last_active":"2024-09-17T08:23:36.516204+0000","last_peered":"2024-09-17T08:23:36.516204+0000","last_clean":"2024-09-17T08:23:36.516204+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:23:36.516204+0000","last_undegraded":"2024-09-17T08:23:36.516204+0000","last_fullsized":"2024-09-17T08:23:36.516204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.801348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.801348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:41.770147+0000","scrub_duration":0.0046643070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":359,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.512704+0000","last_change":"2024-09-17T08:21:27.998120+0000","last_active":"2024-09-17T08:23:36.512704+0000","last_peered":"2024-09-17T08:23:36.512704+0000","last_clean":"2024-09-17T08:23:36.512704+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:23:36.512704+0000","last_undegraded":"2024-09-17T08:23:36.512704+0000","last_fullsized":"2024-09-17T08:23:36.512704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:21:27.997982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:21:27.997982+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010572906999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":310,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-09-17T08:23:36.525253+0000","last_change":"2024-09-17T08:23:31.575041+0000","last_active":"2024-09-17T08:23:36.525253+0000","last_peered":"2024-09-17T08:23:36.525253+0000","last_clean":"2024-09-17T08:23:36.525253+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:23:36.525253+0000","last_undegraded":"2024-09-17T08:23:36.525253+0000","last_fullsized":"2024-09-17T08:23:36.525253+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:31.574717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:31.574717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:38.924282+0000","scrub_duration":0.0019245600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262394,"num_write_kb":8396545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2199,"num_bytes_recovered":144113664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82669,"ondisk_log_size":82669,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":491,"num_read_kb":486,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070635,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":519720,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":518398,"kb_avail":93225432,"statfs":{"total":95995035648,"available":95462842368,"internally_reserved":0,"allocated":1339392,"data_stored":1037067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":530840011},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2010000000000001,"5min":1.927,"15min":1.319},"min":{"1min":0.28000000000000003,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":13.446,"5min":62.018000000000001,"15min":62.018000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":2.2559999999999998,"5min":1.948,"15min":1.395},"min":{"1min":0.313,"5min":0.22500000000000001,"15min":0.21199999999999999},"max":{"1min":13.417,"5min":61.896999999999998,"15min":61.896999999999998},"last":3.2490000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":2.6789999999999998,"15min":1.446},"min":{"1min":0.23899999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":14.731,"5min":62.067,"15min":62.067},"last":1.264},{"interface":"front","average":{"1min":3.2639999999999998,"5min":2.722,"15min":1.4079999999999999},"min":{"1min":0.32300000000000001,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":14.401999999999999,"5min":62.194000000000003,"15min":62.194000000000003},"last":1.5820000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.0289999999999999,"15min":0.91100000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":4.4169999999999998,"5min":4.4169999999999998,"15min":4.4169999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.82299999999999995,"15min":0.64600000000000002},"min":{"1min":0.28799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.3559999999999999,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":0.42999999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":3.1040000000000001,"5min":1.9370000000000001,"15min":1.2749999999999999},"min":{"1min":0.35499999999999998,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":17.356000000000002,"5min":44.606000000000002,"15min":44.606000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":3.1909999999999998,"5min":2.1200000000000001,"15min":1.4159999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":17.315999999999999,"5min":44.591000000000001,"15min":44.591000000000001},"last":3.7730000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":0.88300000000000001,"15min":0.64000000000000001},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":4.391,"5min":5.335,"15min":5.335},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.56699999999999995},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":3.798,"5min":5.9669999999999996,"15min":5.9669999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":0.94799999999999995,"15min":0.74399999999999999},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.8450000000000002,"5min":8.6400000000000006,"15min":16.808},"last":0.314},{"interface":"front","average":{"1min":1.2210000000000001,"5min":0.91100000000000003,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":11.842000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:23:01 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.01,"15min":0.74099999999999999},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.26600000000000001},"max":{"1min":4.4640000000000004,"5min":7.2359999999999998,"15min":7.2359999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.77500000000000002,"15min":0.61699999999999999},"min":{"1min":0.35099999999999998,"5min":0.246,"15min":0.246},"max":{"1min":3.3849999999999998,"5min":5.9400000000000004,"15min":5.9400000000000004},"last":0.40799999999999997}]}]},{"osd":6,"up_from":50,"seq":214748364976,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":335128,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":93410024,"statfs":{"total":95995035648,"available":95651864576,"internally_reserved":0,"allocated":1323008,"data_stored":1036327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":341834183},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":2.0470000000000002,"15min":2.0190000000000001},"min":{"1min":0.24099999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.2189999999999999,"5min":81.007000000000005,"15min":81.007000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":2.0579999999999998,"15min":2.0710000000000002},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.19900000000000001},"max":{"1min":4.8710000000000004,"5min":80.310000000000002,"15min":80.310000000000002},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":2.516,"15min":1.5369999999999999},"min":{"1min":0.28699999999999998,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":3.7869999999999999,"5min":80.813000000000002,"15min":80.813000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.101,"5min":2.5190000000000001,"15min":1.5820000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5259999999999998,"5min":80.281000000000006,"15min":80.281000000000006},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":3.2050000000000001,"5min":1.1899999999999999,"15min":0.81200000000000006},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":57.305999999999997,"5min":57.305999999999997,"15min":57.305999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":3.2650000000000001,"5min":1.1259999999999999,"15min":0.72799999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":57.289999999999999,"5min":57.289999999999999,"15min":57.289999999999999},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":2.294,"15min":1.8420000000000001},"min":{"1min":0.32000000000000001,"5min":0.29899999999999999,"15min":0.28699999999999998},"max":{"1min":3.8250000000000002,"5min":80.427000000000007,"15min":80.427000000000007},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1699999999999999,"5min":2.125,"15min":1.764},"min":{"1min":0.36899999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.2930000000000001,"5min":80.379999999999995,"15min":80.379999999999995},"last":0.39400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67900000000000005,"15min":0.56899999999999995},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.215},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":4.6219999999999999},"last":0.379},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.51800000000000002},"min":{"1min":0.30299999999999999,"5min":0.26800000000000002,"15min":0.25},"max":{"1min":2.4060000000000001,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:22:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.80700000000000005,"15min":0.61599999999999999},"min":{"1min":0.33100000000000002,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":5.5659999999999998,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.57899999999999996},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1869999999999998,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:23:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78800000000000003,"15min":0.67000000000000004},"min":{"1min":0.30199999999999999,"5min":0.246,"15min":0.246},"max":{"1min":2.544,"5min":3.988,"15min":4.774},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.79400000000000004,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.29799999999999999,"15min":0.28999999999999998},"max":{"1min":3.1859999999999999,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":0.58699999999999997}]}]},{"osd":5,"up_from":41,"seq":176093659317,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":505176,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":504446,"kb_avail":93239976,"statfs":{"total":95995035648,"available":95477735424,"internally_reserved":0,"allocated":733184,"data_stored":445908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":516553158},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:23:18 2024","interfaces":[{"interface":"back","average":{"1min":4.3659999999999997,"5min":2.0339999999999998,"15min":1.6140000000000001},"min":{"1min":0.223,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":65.941000000000003,"5min":65.941000000000003,"15min":65.941000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":4.5270000000000001,"5min":2.1269999999999998,"15min":1.573},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":66.135000000000005,"5min":66.135000000000005,"15min":66.135000000000005},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:23:18 2024","interfaces":[{"interface":"back","average":{"1min":6.6870000000000003,"5min":2.3660000000000001,"15min":1.323},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":66.981999999999999,"5min":66.981999999999999,"15min":66.981999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":6.601,"5min":2.3700000000000001,"15min":1.379},"min":{"1min":0.25700000000000001,"5min":0.248,"15min":0.22},"max":{"1min":66.316000000000003,"5min":66.316000000000003,"15min":66.316000000000003},"last":0.54800000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:23:29 2024","interfaces":[{"interface":"back","average":{"1min":7.5800000000000001,"5min":2.1459999999999999,"15min":1.202},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":131.727,"5min":131.727,"15min":131.727},"last":0.49199999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":2.319,"15min":1.5109999999999999},"min":{"1min":0.27400000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":131.50399999999999,"5min":131.50399999999999,"15min":131.50399999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Tue Sep 17 08:23:18 2024","interfaces":[{"interface":"back","average":{"1min":6.3879999999999999,"5min":2.121,"15min":1.6559999999999999},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":67.676000000000002,"5min":67.676000000000002,"15min":75.286000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":6.4139999999999997,"5min":2.1629999999999998,"15min":1.637},"min":{"1min":0.27200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":67.691999999999993,"5min":67.691999999999993,"15min":77.206999999999994},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:23:18 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.629,"15min":0.68999999999999995},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.19},"max":{"1min":2.5270000000000001,"5min":3.9849999999999999,"15min":42.421999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.63800000000000001,"15min":0.70199999999999996},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":2.472,"5min":2.879,"15min":42.191000000000003},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.57499999999999996},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27700000000000002},"max":{"1min":3.4769999999999999,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.69699999999999995,"15min":0.58499999999999996},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.254},"max":{"1min":1.2909999999999999,"5min":5.4400000000000004,"15min":5.4400000000000004},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:23:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":1.349,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.79000000000000004,"15min":0.64900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":3.0059999999999998,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.62}]}]},{"osd":4,"up_from":32,"seq":137438953658,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":346852,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":346110,"kb_avail":93398300,"statfs":{"total":95995035648,"available":95639859200,"internally_reserved":0,"allocated":745472,"data_stored":446537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":354417094},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.91200000000000003},"min":{"1min":0.24099999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":23.786000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.90200000000000002,"15min":0.94599999999999995},"min":{"1min":0.29199999999999998,"5min":0.219,"15min":0.219},"max":{"1min":2.2890000000000001,"5min":3.7839999999999998,"15min":23.245000000000001},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":1.5640000000000001,"15min":1.0860000000000001},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":9.3789999999999996,"5min":14.164,"15min":23.135000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.401,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.246,"15min":0.24099999999999999},"max":{"1min":9.0719999999999992,"5min":9.0719999999999992,"15min":23.434999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:23:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66000000000000003,"15min":0.59399999999999997},"min":{"1min":0.26000000000000001,"5min":0.25,"15min":0.25},"max":{"1min":1.869,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.8069999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:22:50 2024","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.353,"15min":1.7969999999999999},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.226,"5min":10.012,"15min":154.137},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.208,"15min":1.6779999999999999},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":5.556,"5min":9.5180000000000007,"15min":153.54400000000001},"last":0.502}]},{"osd":5,"last update":"Tue Sep 17 08:23:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78700000000000003,"15min":0.57999999999999996},"min":{"1min":0.313,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":3.202,"5min":6.2329999999999997,"15min":6.2329999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.069,"5min":0.89800000000000002,"15min":0.64200000000000002},"min":{"1min":0.34000000000000002,"5min":0.26800000000000002,"15min":0.25900000000000001},"max":{"1min":4.0229999999999997,"5min":4.5300000000000002,"15min":4.5300000000000002},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Sep 17 08:23:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.56299999999999994},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":2.387,"5min":4.827,"15min":4.827},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.59999999999999998},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.7230000000000001,"5min":3.601,"15min":6.0010000000000003},"last":0.84999999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:23:02 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":0.878,"15min":0.68700000000000006},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.0259999999999998,"5min":4.875,"15min":4.875},"last":0.751},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.875,"15min":0.65400000000000003},"min":{"1min":0.30399999999999999,"5min":0.28199999999999997,"15min":0.27300000000000002},"max":{"1min":2.7949999999999999,"5min":4.6950000000000003,"15min":4.6950000000000003},"last":0.78100000000000003}]}]},{"osd":3,"up_from":27,"seq":115964117182,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":545392,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":544638,"kb_avail":93199760,"statfs":{"total":95995035648,"available":95436554240,"internally_reserved":0,"allocated":757760,"data_stored":446014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":557709768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:23:36 2024","interfaces":[{"interface":"back","average":{"1min":4.2249999999999996,"5min":2.1680000000000001,"15min":1.6080000000000001},"min":{"1min":0.182,"5min":0.156,"15min":0.156},"max":{"1min":65.819000000000003,"5min":65.819000000000003,"15min":65.819000000000003},"last":0.23799999999999999},{"interface":"front","average":{"1min":3.9660000000000002,"5min":1.8200000000000001,"15min":1.466},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":65.873999999999995,"5min":65.873999999999995,"15min":65.873999999999995},"last":0.32200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:23:36 2024","interfaces":[{"interface":"back","average":{"1min":5.718,"5min":2.4089999999999998,"15min":1.6359999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.186},"max":{"1min":71.459999999999994,"5min":71.459999999999994,"15min":71.459999999999994},"last":0.438},{"interface":"front","average":{"1min":5.9349999999999996,"5min":2.4900000000000002,"15min":1.639},"min":{"1min":0.247,"5min":0.192,"15min":0.188},"max":{"1min":71.438999999999993,"5min":71.438999999999993,"15min":71.438999999999993},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:23:30 2024","interfaces":[{"interface":"back","average":{"1min":3.8450000000000002,"5min":1.7210000000000001,"15min":1.2889999999999999},"min":{"1min":0.22800000000000001,"5min":0.215,"15min":0.215},"max":{"1min":60.423999999999999,"5min":60.423999999999999,"15min":60.423999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":3.3399999999999999,"5min":2.052,"15min":1.778},"min":{"1min":0.30099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":60.368000000000002,"5min":60.368000000000002,"15min":60.368000000000002},"last":0.28100000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.6339999999999999,"15min":1.214},"min":{"1min":0.28499999999999998,"5min":0.28100000000000003,"15min":0.252},"max":{"1min":4.3339999999999996,"5min":35.841000000000001,"15min":35.841000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.5820000000000001,"15min":1.165},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.9359999999999999,"5min":35.857999999999997,"15min":35.857999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:23:14 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3029999999999999,"15min":1.153},"min":{"1min":0.33200000000000002,"5min":0.311,"15min":0.29799999999999999},"max":{"1min":3.5350000000000001,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.657,"5min":1.54,"15min":1.3029999999999999},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":9.7289999999999992,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.626}]},{"osd":6,"last update":"Tue Sep 17 08:22:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.27,"15min":1.087},"min":{"1min":0.35199999999999998,"5min":0.31,"15min":0.29499999999999998},"max":{"1min":7.6669999999999998,"5min":7.6669999999999998,"15min":10.945},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6870000000000001,"15min":1.302},"min":{"1min":0.39000000000000001,"5min":0.314,"15min":0.311},"max":{"1min":9.6600000000000001,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:23:02 2024","interfaces":[{"interface":"back","average":{"1min":2.4670000000000001,"5min":1.5780000000000001,"15min":1.236},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.316},"max":{"1min":38.936999999999998,"5min":38.936999999999998,"15min":38.936999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":2.802,"5min":1.6399999999999999,"15min":1.2350000000000001},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.28999999999999998},"max":{"1min":36.634999999999998,"5min":36.634999999999998,"15min":36.634999999999998},"last":0.61599999999999999}]}]},{"osd":2,"up_from":227,"seq":974957576233,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":136796,"kb_used_data":720,"kb_used_omap":4,"kb_used_meta":136059,"kb_avail":93608356,"statfs":{"total":95995035648,"available":95854956544,"internally_reserved":0,"allocated":737280,"data_stored":446306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":139325091},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1759999999999999,"15min":1.04},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":6.1669999999999998,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":0.372},{"interface":"front","average":{"1min":0.628,"5min":1.2270000000000001,"15min":1.385},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":2.0979999999999999,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":4.4500000000000002,"5min":1.9330000000000001,"15min":1.4650000000000001},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":41.405000000000001,"5min":41.405000000000001,"15min":41.405000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":4.3860000000000001,"5min":1.931,"15min":1.4910000000000001},"min":{"1min":0.23699999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":41.25,"5min":41.25,"15min":41.25},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":2.718,"5min":1.722,"15min":1.524},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":44.185000000000002,"5min":44.185000000000002,"15min":44.185000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":3.133,"5min":1.8160000000000001,"15min":1.4850000000000001},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":43.994,"5min":43.994,"15min":43.994},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.88,"15min":0.84299999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.4910000000000001,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.84999999999999998,"15min":0.82999999999999996},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3500000000000001,"5min":3.411,"15min":3.411},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.96899999999999997,"15min":0.94999999999999996},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.589,"5min":4.915,"15min":4.915},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.93300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.4630000000000001,"5min":3.4489999999999998,"15min":3.4489999999999998},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.85699999999999998,"15min":0.73899999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":6.1790000000000003,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.498},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.92800000000000005,"15min":0.89400000000000002},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.54,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:23:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1,"15min":0.98199999999999998},"min":{"1min":0.40600000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.5339999999999998,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.089,"5min":0.98399999999999999,"15min":0.90500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.52600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":591352,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":590590,"kb_avail":93153800,"statfs":{"total":95995035648,"available":95389491200,"internally_reserved":0,"allocated":765952,"data_stored":446604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":604764614},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:22:54 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.508,"15min":1.407},"min":{"1min":0.21199999999999999,"5min":0.155,"15min":0.155},"max":{"1min":9.5730000000000004,"5min":9.5730000000000004,"15min":31.585999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.7450000000000001,"15min":1.5860000000000001},"min":{"1min":0.248,"5min":0.19500000000000001,"15min":0.17199999999999999},"max":{"1min":5.3129999999999997,"5min":16.812000000000001,"15min":35.807000000000002},"last":0.29499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:23:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.226,"15min":0.93700000000000006},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":4.7720000000000002,"5min":22.943000000000001,"15min":22.943000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":1.526,"5min":1.506,"15min":1.492},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":9.6110000000000007,"5min":9.6110000000000007,"15min":9.6110000000000007},"last":0.34799999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:23:36 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.552,"15min":1.3129999999999999},"min":{"1min":0.23200000000000001,"5min":0.184,"15min":0.184},"max":{"1min":4.5679999999999996,"5min":7.7350000000000003,"15min":8.0559999999999992},"last":0.32100000000000001},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.899,"15min":1.51},"min":{"1min":0.374,"5min":0.23699999999999999,"15min":0.189},"max":{"1min":4.7969999999999997,"5min":22.984999999999999,"15min":22.984999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 17 08:23:00 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.4610000000000001,"15min":1.1759999999999999},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":3.2919999999999998,"5min":16.884,"15min":16.884},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.409,"15min":1.075},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.25700000000000001},"max":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":10.208},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:23:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.3280000000000001,"15min":1.105},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5,"5min":7.5439999999999996,"15min":7.9139999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.361,"5min":1.486,"15min":1.2829999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":4.8810000000000002,"5min":16.853999999999999,"15min":16.853999999999999},"last":0.433}]},{"osd":6,"last update":"Tue Sep 17 08:23:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.5660000000000001,"15min":1.208},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.28199999999999997},"max":{"1min":9.7159999999999993,"5min":9.7159999999999993,"15min":10.253},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.252,"5min":1.387,"15min":1.1970000000000001},"min":{"1min":0.34200000000000003,"5min":0.33800000000000002,"15min":0.27200000000000002},"max":{"1min":4.5369999999999999,"5min":7.6079999999999997,"15min":7.9459999999999997},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:23:09 2024","interfaces":[{"interface":"back","average":{"1min":4.194,"5min":2.177,"15min":1.47},"min":{"1min":0.42599999999999999,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":47.442,"5min":47.442,"15min":47.442},"last":0.50700000000000001},{"interface":"front","average":{"1min":4.2329999999999997,"5min":2.0409999999999999,"15min":1.3680000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.378},"max":{"1min":47.475000000000001,"5min":47.475000000000001,"15min":47.475000000000001},"last":0.58899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738570,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":425380,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":93319772,"statfs":{"total":95995035648,"available":95559446528,"internally_reserved":0,"allocated":1335296,"data_stored":1036751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":434239942},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":2.3239999999999998,"15min":1.6950000000000001},"min":{"1min":0.20399999999999999,"5min":0.189,"15min":0.189},"max":{"1min":4.9459999999999997,"5min":43.493000000000002,"15min":61.649000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.538,"5min":2.4460000000000002,"15min":1.845},"min":{"1min":0.247,"5min":0.20200000000000001,"15min":0.18099999999999999},"max":{"1min":5.3840000000000003,"5min":40.436,"15min":61.281999999999996},"last":0.375}]},{"osd":2,"last update":"Tue Sep 17 08:23:31 2024","interfaces":[{"interface":"back","average":{"1min":6.3209999999999997,"5min":2.4199999999999999,"15min":1.5620000000000001},"min":{"1min":0.19900000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":99.120999999999995,"5min":99.120999999999995,"15min":99.120999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":6.524,"5min":2.4420000000000002,"15min":1.403},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":99.256,"5min":99.256,"15min":99.256},"last":0.42299999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:23:38 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.6970000000000001,"15min":1.901},"min":{"1min":0.22700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":7.1349999999999998,"5min":38.697000000000003,"15min":61.926000000000002},"last":0.495},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.45,"15min":1.6950000000000001},"min":{"1min":0.29699999999999999,"5min":0.223,"15min":0.223},"max":{"1min":5.173,"5min":38.950000000000003,"15min":61.616},"last":0.46300000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:22:55 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.401,"15min":1.236},"min":{"1min":0.32200000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.0800000000000001,"5min":41.158000000000001,"15min":41.158000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.4430000000000001,"15min":1.2529999999999999},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.9670000000000001,"5min":38.978000000000002,"15min":38.978000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:23:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.3759999999999999,"15min":1.1930000000000001},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.327,"5min":39.058,"15min":39.058},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.5680000000000001,"15min":1.3380000000000001},"min":{"1min":0.32600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":3.5680000000000001,"5min":43.485999999999997,"15min":43.485999999999997},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:22:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.7450000000000001,"15min":1.1879999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.919,"5min":38.917999999999999,"15min":38.917999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.9810000000000001,"15min":1.331},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":5.1239999999999997,"5min":41.189,"15min":41.189},"last":0.53400000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:23:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.5269999999999999,"15min":1.4279999999999999},"min":{"1min":0.308,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.9319999999999999,"5min":39.045999999999999,"15min":39.045999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.111,"5min":1.4930000000000001,"15min":1.3700000000000001},"min":{"1min":0.40200000000000002,"5min":0.34399999999999997,"15min":0.307},"max":{"1min":3.7570000000000001,"5min":39.082999999999998,"15min":39.082999999999998},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:23:47.866 INFO:tasks.thrashosds.thrasher:Setting 4.10 to [4, 5, 6] 2024-09-17T08:23:47.866 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.10', '4', '5', '6'] 2024-09-17T08:23:47.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 4.10 4 5 6 2024-09-17T08:23:47.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:47.930 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:47.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:47.931 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:48.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:48.053 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:48.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.054 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:48.155 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:48.181 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:48.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:48.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:48.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:48.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:48.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:48.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:48.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[40649]: pgmap v608: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 639 op/s 2024-09-17T08:23:48.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:48.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:48.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:48 smithi079 ceph-mon[29565]: pgmap v608: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 639 op/s 2024-09-17T08:23:48.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:48.434 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:48.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:48.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:48.559 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:48.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:48.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:48.687 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:48.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:48.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:23:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:23:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:48 smithi173 ceph-mon[33187]: pgmap v608: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 639 op/s 2024-09-17T08:23:48.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:48.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:48.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:48.913 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:48.936 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:48.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:48.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:49.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:49.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:49.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:49.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:49.182 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:49.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:49.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:23:49.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:49.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.305 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:49.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:49.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:49.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:49.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:49.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:49.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:49.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.551 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:49.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:49.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.675 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:49.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:49.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:49.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:49.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:49 smithi079 ceph-mon[29565]: pgmap v609: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 70 op/s 2024-09-17T08:23:49.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:49 smithi079 ceph-mon[40649]: pgmap v609: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 70 op/s 2024-09-17T08:23:49.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:49.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:49.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:49.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:49 smithi173 ceph-mon[33187]: pgmap v609: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 70 op/s 2024-09-17T08:23:50.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:50.046 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:50.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:50.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:50.172 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:50.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:50.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:50.296 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:50.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:50.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:50.421 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:50.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:50.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:50.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:50.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:50.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:50.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:50.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:50.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:50.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:50.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.793 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:50.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:50.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:50.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:50.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:51.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:51.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:51.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:51.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:51.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:51.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:51.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:51.296 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:51.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:51.326 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:23:51.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:51.422 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:51.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:51.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/15413485' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.468 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.469 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:51.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/15413485' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.469 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:51 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:51.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:51.544 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:51.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:51.646 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:51.669 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:51.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:51.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:51 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/15413485' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:23:51.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:51 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:23:51.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:51.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:51.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:51.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:51.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:51.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:52.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:52.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:52.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.043 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:52.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:52.165 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:52.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:52.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:52.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:52.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:52.341 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_2' removed 2024-09-17T08:23:52.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:52.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:52.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:52.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:52.541 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:52.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[29565]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T08:23:52.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[29565]: pgmap v611: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 52 op/s 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3501330649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[40649]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[40649]: pgmap v611: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 52 op/s 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3501330649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:52 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:52.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.668 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:52.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:52.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:52 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:23:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:52 smithi173 ceph-mon[33187]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T08:23:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:52 smithi173 ceph-mon[33187]: pgmap v611: 148 pgs: 148 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 52 op/s 2024-09-17T08:23:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:52 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3501330649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:52 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:23:52.741 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:52.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:52.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:52.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:52.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:52.915 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:52.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:52.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:53.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:53.038 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:53.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.039 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:53.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:53.162 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:53.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:53.264 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:53.286 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:53.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:53.335 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:23:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:23:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:23:53.335 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:23:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:23:53.388 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:53.409 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:53.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:53.410 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:53.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:53.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:53 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:23:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:53 smithi079 ceph-mon[29565]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T08:23:53.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:53 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:23:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:53 smithi079 ceph-mon[40649]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T08:23:53.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:53.658 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:53.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:53 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:23:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:53 smithi173 ceph-mon[33187]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T08:23:53.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:53.782 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:53.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:53.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:53.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:53.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:53.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:53.997 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-17T08:23:53.998 INFO:tasks.radosbench:Beginning radosbench... 2024-09-17T08:23:53.998 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-17T08:23:53.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_3 16 2024-09-17T08:23:54.007 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:54.029 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:54.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:54.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:54.153 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:54.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:54.255 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:54.277 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:54.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:54.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:54.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:54.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:54.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:54.519 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:54.519 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:54 smithi079 ceph-mon[40649]: pgmap v613: 132 pgs: 132 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:54.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:54.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:54 smithi079 ceph-mon[29565]: pgmap v613: 132 pgs: 132 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:54.525 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:54.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:54.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:54.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:54.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:23:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:54 smithi173 ceph-mon[33187]: pgmap v613: 132 pgs: 132 active+clean; 582 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:54.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:54.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:23:54.770 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:54.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:54.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:54.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:54.894 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:54.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:54.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:55.021 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:55.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:55.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:55.144 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:55.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:55.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:55.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:55.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:55.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:55.394 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:55.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.395 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:55.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:55.522 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:55.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:55.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:55 smithi079 ceph-mon[40649]: pgmap v614: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:55.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:55 smithi079 ceph-mon[29565]: pgmap v614: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:55.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:55.645 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:55.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:55.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:55.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11680 9.6 9.a107be96 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.33807216, 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98902988199999997, 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11680 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492093+0000", 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492153+0000", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0398e-05 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492169+0000", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5747e-05 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492178+0000", 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9889999999999998e-06 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.500907+0000", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0087288000000000001 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.500917+0000", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0227999999999999e-05 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481114+0000", 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98019719299999997 2024-09-17T08:23:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481125+0000", 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0677e-05 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11685 9.6 9.95710a26 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.33796730099999, 2024-09-17T08:23:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98896512400000003, 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11685 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492199+0000", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492221+0000", 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1826000000000001e-05 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492229+0000", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5189999999999992e-06 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492236+0000", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1709999999999999e-06 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501314+0000", 2024-09-17T08:23:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090785840000000007 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501321+0000", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8669999999999996e-06 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481157+0000", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97983637499999998 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481165+0000", 2024-09-17T08:23:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4070000000000004e-06 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11687 9.6 9.1931af16 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.337925692, 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98896536099999999, 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11687 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492264+0000", 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3332e-05 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492271+0000", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.233e-06 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492276+0000", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6779999999999999e-06 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.502199+0000", 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099226100000000001 2024-09-17T08:23:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.502206+0000", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2259999999999999e-06 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481200+0000", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97899387199999999 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481207+0000", 2024-09-17T08:23:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8940000000000004e-06 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11722 9.6 9.1e34edf6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.320995334, 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97205906399999997, 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11722 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509169+0000", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509202+0000", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2230000000000001e-05 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509219+0000", 2024-09-17T08:23:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7748e-05 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509229+0000", 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.73e-06 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509283+0000", 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4252000000000002e-05 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509290+0000", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6680000000000004e-06 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481224+0000", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97193378699999999 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481231+0000", 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11734 9.6 9.e7adb46 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.31464602, 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.96575857899999995, 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11734 2024-09-17T08:23:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515519+0000", 2024-09-17T08:23:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515551+0000", 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2396e-05 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515573+0000", 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1594999999999999e-05 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515582+0000", 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4490000000000005e-06 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515650+0000", 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7528999999999997e-05 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515660+0000", 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0559e-05 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481272+0000", 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.96561206600000005 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481280+0000", 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0960000000000003e-06 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11755 9.6 9.44f58cd6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.30041420800001, 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95155133000000003, 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11755 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529750+0000", 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999971 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529787+0000", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6903999999999998e-05 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529807+0000", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9723000000000001e-05 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529816+0000", 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.516e-06 2024-09-17T08:23:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529876+0000", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9246999999999997e-05 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529884+0000", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6770000000000001e-06 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481297+0000", 2024-09-17T08:23:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95141305200000004 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481304+0000", 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11758 9.6 9.46988236 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.297445528, 2024-09-17T08:23:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.948607588, 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11758 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532720+0000", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532745+0000", 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4783999999999999e-05 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532760+0000", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4521999999999999e-05 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532769+0000", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4909999999999994e-06 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532827+0000", 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.783e-05 2024-09-17T08:23:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532834+0000", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3180000000000001e-06 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481322+0000", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.94848791200000004 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481329+0000", 2024-09-17T08:23:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7900000000000002e-06 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1632 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94434111868160 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 212.87678549699999, 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53571094799999996, 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:23:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1632 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953379+0000", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953387+0000", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7030000000000006e-06 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953400+0000", 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-17T08:23:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953412+0000", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2456999999999999e-05 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953473+0000", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0698000000000001e-05 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953483+0000", 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4730000000000006e-06 2024-09-17T08:23:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.479366+0000", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.52588388500000005 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.479544+0000", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017710100000000001 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.489092+0000", 2024-09-17T08:23:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0095489370000000004 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:187779 9.6 9:6df68798:::benchmark_data_smithi079_143589_object56580:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e267)", 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:33.181106+0000", 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.649061352000004, 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.155905974, 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 187779 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181106+0000", 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181106+0000", 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181104+0000", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181111+0000", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7299999999999999e-06 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181124+0000", 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2986999999999999e-05 2024-09-17T08:23:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181131+0000", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4189999999999996e-06 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181617+0000", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048542399999999998 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181749+0000", 2024-09-17T08:23:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013229299999999999 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181887+0000", 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013843300000000001 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.189766+0000", 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078785539999999994 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.189775+0000", 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3260000000000002e-06 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.336933+0000", 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.147158022 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.336976+0000", 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2719000000000002e-05 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.337012+0000", 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5578999999999999e-05 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:187781 9.6 9:6300154b:::benchmark_data_smithi079_143589_object56582:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e267)", 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:33.181905+0000", 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.648262224999996, 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15527395099999999, 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 187781 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181905+0000", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181905+0000", 2024-09-17T08:23:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181904+0000", 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181909+0000", 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3120000000000001e-06 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181920+0000", 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0919e-05 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.181927+0000", 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1999999999999997e-06 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.186070+0000", 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041426299999999996 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.186170+0000", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9727000000000002e-05 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.186269+0000", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9819000000000001e-05 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.191194+0000", 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049245909999999999 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.193015+0000", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001821037 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.337102+0000", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14408646899999999 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.337144+0000", 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2475999999999998e-05 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:33.337179+0000", 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4728999999999999e-05 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210605 9.0 e272/230 9:018ae7c4:::benchmark_data_smithi079_143589_object79406:head v 272'12909, mlcod=272'12906)", 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.890281725000001, 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37561313400000002, 2024-09-17T08:23:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939884+0000", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939892+0000", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.187e-06 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939902+0000", 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0917000000000001e-05 2024-09-17T08:23:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939910+0000", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3420000000000002e-06 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939949+0000", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9501e-05 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939970+0000", 2024-09-17T08:23:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0380000000000001e-05 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315468+0000", 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37549839200000001 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315498+0000", 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0312999999999999e-05 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210614 9.2 e272/230 9:448427a4:::benchmark_data_smithi079_143589_object79415:head v 272'13602, mlcod=272'13600)", 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.884004287000003, 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36429002500000002, 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946161+0000", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946171+0000", 2024-09-17T08:23:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303862+0000", 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35769168000000001 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303871+0000", 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8189999999999998e-06 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304030+0000", 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158679 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304053+0000", 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3189000000000001e-05 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310428+0000", 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063754440000000001 2024-09-17T08:23:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310453+0000", 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4318e-05 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210615 9.6 9:6555ccd0:::benchmark_data_smithi079_143589_object79416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.883516782999997, 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36942082399999998, 2024-09-17T08:23:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210615 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946648+0000", 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946656+0000", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303849+0000", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35719317699999997 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303864+0000", 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5011000000000001e-05 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303913+0000", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9240999999999998e-05 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304058+0000", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014518800000000001 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304185+0000", 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126952 2024-09-17T08:23:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312414+0000", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008228806 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312468+0000", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3854000000000003e-05 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315985+0000", 2024-09-17T08:23:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035164839999999998 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316026+0000", 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1301000000000003e-05 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316071+0000", 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5207999999999998e-05 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213094 9.6 9:6c0b11e2:::benchmark_data_smithi079_143589_object81895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.492801901, 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.31323531599999999, 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213094 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337363+0000", 2024-09-17T08:23:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337371+0000", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6359999999999994e-06 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337385+0000", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4183e-05 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337392+0000", 2024-09-17T08:23:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4719999999999998e-06 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337442+0000", 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9378000000000001e-05 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337568+0000", 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012602300000000001 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337689+0000", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012135599999999999 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341229+0000", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003539979 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.345283+0000", 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040540510000000004 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650515+0000", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30523122800000002 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650560+0000", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5611000000000001e-05 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650600+0000", 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0299000000000002e-05 2024-09-17T08:23:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213100 9.6 9:65f8bbfc:::benchmark_data_smithi079_143589_object81901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.488948123999997, 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30948176700000002, 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213100 2024-09-17T08:23:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341217+0000", 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341225+0000", 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.328e-06 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341239+0000", 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4051e-05 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341247+0000", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3319999999999992e-06 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341301+0000", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3996000000000003e-05 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341442+0000", 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141328 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341562+0000", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011956 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.346195+0000", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046330629999999998 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.347443+0000", 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001248292 2024-09-17T08:23:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650634+0000", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30319073099999999 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650668+0000", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4360999999999998e-05 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650701+0000", 2024-09-17T08:23:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2305999999999999e-05 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213109 9.6 9:690187cb:::benchmark_data_smithi079_143589_object81910:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.47934558, 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29998592099999999, 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213109 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350820+0000", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350826+0000", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5889999999999999e-06 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350840+0000", 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3474e-05 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350848+0000", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9829999999999996e-06 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350896+0000", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8618000000000001e-05 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351004+0000", 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108012 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351110+0000", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105951 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355185+0000", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040743740000000004 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355199+0000", 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-05 2024-09-17T08:23:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650732+0000", 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29553320199999999 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650766+0000", 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4260000000000001e-05 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650807+0000", 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0850999999999999e-05 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213119 9.0 e272/230 9:0a967850:::benchmark_data_smithi079_143589_object81920:head v 272'13077, mlcod=272'13075)", 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.467489340999997, 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29685224700000001, 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362677+0000", 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362684+0000", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1459999999999999e-06 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362696+0000", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687999999999999e-05 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362703+0000", 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7259999999999996e-06 2024-09-17T08:23:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362762+0000", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8424000000000002e-05 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362787+0000", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5799000000000002e-05 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.659042+0000", 2024-09-17T08:23:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29625426799999999 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.659530+0000", 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048823999999999997 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213120 9.0 e272/230 9:090e3aab:::benchmark_data_smithi079_143589_object81921:head v 272'13078, mlcod=272'13075)", 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.464099502000003, 2024-09-17T08:23:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29576923700000002, 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366067+0000", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366076+0000", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0210000000000005e-06 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650376+0000", 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28430049099999999 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650392+0000", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6096999999999999e-05 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650464+0000", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1775999999999999e-05 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650490+0000", 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.552e-05 2024-09-17T08:23:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.661805+0000", 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011315725 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.661837+0000", 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1464000000000002e-05 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213167 9.2 e272/230 9:4746aeca:::benchmark_data_smithi079_143589_object81968:head v 272'13735, mlcod=272'13733)", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.424641090999998, 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.252615176, 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405524+0000", 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405535+0000", 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811e-05 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650412+0000", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.244876661 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650423+0000", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1259e-05 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650465+0000", 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1532000000000001e-05 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650490+0000", 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.516e-05 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.658002+0000", 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075121060000000002 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.658141+0000", 2024-09-17T08:23:55.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013928899999999999 2024-09-17T08:23:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:232708 9.6 9:6c502600:::benchmark_data_smithi079_143589_object101509:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:14.386946+0000", 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.443221368000003, 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.158553528, 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:23:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 232708 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.386946+0000", 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.386946+0000", 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.386945+0000", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.386950+0000", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1630000000000004e-06 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.523148+0000", 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.136198387 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.523168+0000", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0136000000000002e-05 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.525976+0000", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028080230000000002 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.526113+0000", 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013641400000000001 2024-09-17T08:23:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.526245+0000", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013231899999999999 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.537853+0000", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011607519 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.537908+0000", 2024-09-17T08:23:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5287000000000001e-05 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.545416+0000", 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075075209999999996 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.545460+0000", 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4524000000000003e-05 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:23:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:14.545499+0000", 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9103000000000002e-05 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:23:55.961 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:23:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:55 smithi173 ceph-mon[33187]: pgmap v614: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:55.982 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:55.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:55.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:56.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:56.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:56.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:56.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:56.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:56.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:56.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:56.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.356 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:56.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:56.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:56.478 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:56.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:56.580 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:56.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.606 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:56.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:56.707 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:56.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:56.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:56.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:56.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:56.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:56.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:23:56.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:23:56.980 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:56.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:56.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:23:57.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:57.103 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:57.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:57.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:57.226 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:57.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:57.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:57.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:57.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:57.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:57.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:57.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:57.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:57.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:57.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:57.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:57.735 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:57.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:57.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:57 smithi079 ceph-mon[40649]: pgmap v615: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:57.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:57 smithi079 ceph-mon[29565]: pgmap v615: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:57.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:57.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:57.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:57.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:57 smithi173 ceph-mon[33187]: pgmap v615: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T08:23:57.986 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:57.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:57.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:58.088 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:23:58.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:58.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:23:58.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:58.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:58.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:58.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:23:58.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:58.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:23:58.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:58.491 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:58.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:58.581 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:58.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:23:58.619 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:58.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:23:58.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:58.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:58.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:58.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:58.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.869 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:58.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:58.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:58.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:58.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:58.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:59.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:23:59.114 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:23:59.183 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:23:59.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:59.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:59.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:59.361 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:59.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:23:59.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:23:59.488 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:23:59.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:23:59.611 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:23:59.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:59.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:23:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:23:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:59 smithi173 ceph-mon[33187]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T08:23:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:23:59 smithi173 ceph-mon[33187]: pgmap v617: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[40649]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[40649]: pgmap v617: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:23:59.725 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[29565]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T08:23:59.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:23:59 smithi079 ceph-mon[29565]: pgmap v617: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:23:59.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.736 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:59.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:59.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:23:59.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:23:59.982 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:23:59.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:23:59.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:00.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:00.261 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:00.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:00.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:00.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:00.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:00.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:00.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:00.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:00.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:00.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:00.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:00.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:00.763 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:00.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:00.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:00.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:00.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:00.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:00.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:01.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:01.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:01.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:01.257 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:01.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:01.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:01.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:01.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:01.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:01.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:01.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:01.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:01.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:01.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:01.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:01.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:01.889 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:24:01.889 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:24:01.890 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:24:01.890 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:24:01.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:24:01.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:24:01.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:24:01.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:24:01.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:24:01.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:24:01.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:24:01.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:24:01.896 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:24:01.896 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:24:01.912 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:24:01.913 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:24:01.980 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:02.006 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:02.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:02.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:02.129 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:02.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:02.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:02.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.258 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:02.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:02.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:02.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:02.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:02.434 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:24:02.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:02.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.509 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:02.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:02.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[40649]: pgmap v618: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:24:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3177215899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/738482882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:02.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[29565]: pgmap v618: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:24:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3177215899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/738482882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:02.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:02.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:02.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:02 smithi173 ceph-mon[33187]: pgmap v618: 132 pgs: 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:24:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3177215899' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:02 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:24:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/738482882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:02.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:02.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.761 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:02.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:02.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:02.888 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:02.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:02.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:02.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:03.013 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:03.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:03.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:03.138 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:03.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.140 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:03.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:03.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:03.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:03.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:03.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:03.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:03.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:03.442 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_3' created 2024-09-17T08:24:03.492 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:03.517 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:03.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:03.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:03.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.641 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:03.641 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2123166314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:24:03.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/101286522' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2123166314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:24:03.691 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/101286522' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.692 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:03 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2123166314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/101286522' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:03 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]: dispatch 2024-09-17T08:24:03.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:03.767 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:03.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:03.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:03.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:03.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:03.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:04.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:04.026 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.027 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:04.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:04.154 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:04.249 INFO:teuthology.orchestra.run.smithi079.stderr:set 4.10 pg_upmap mapping to [4,5,6] 2024-09-17T08:24:04.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:04.281 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:04.383 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:04.408 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:04.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:04.535 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]': finished 2024-09-17T08:24:04.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: pgmap v621: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:24:04.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]': finished 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: pgmap v621: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:24:04.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:04 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:24:04.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:04.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:04.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:04.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-17T08:24:04.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.10", "id": [4, 5, 6]}]': finished 2024-09-17T08:24:04.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T08:24:04.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: pgmap v621: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:24:04.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:24:04.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:04 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:24:04.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:04.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:04.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:04.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:04.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:05.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:05.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:05.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:05.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-17T08:24:05.137 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:05.162 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:05.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:05.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:05.286 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:05.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:05.388 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:05.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:05.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:05.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:05.537 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:05.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:05.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:24:05.639 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:05.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:05.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:24:05.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:05 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:24:05.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:05.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:05.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:24:05.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:24:05.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:24:05.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:05 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:24:05.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:05.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:05.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:05.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:06.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:06.041 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:06.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:06.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:06.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:06.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:06.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:06.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:06.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:06.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:06.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:06.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.427 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:06.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:06.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:06.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:06.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:06.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:06.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[40649]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T08:24:06.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:24:06.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:24:06.673 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[40649]: pgmap v624: 148 pgs: 9 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:24:06.674 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[29565]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T08:24:06.674 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:24:06.674 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:24:06.674 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:06 smithi079 ceph-mon[29565]: pgmap v624: 148 pgs: 9 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:24:06.675 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:06 smithi173 ceph-mon[33187]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T08:24:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:06 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:24:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:06 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:24:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:06 smithi173 ceph-mon[33187]: pgmap v624: 148 pgs: 9 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:24:06.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:06.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:06.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:06.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:06.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:06.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:07.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:07.042 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:24:07.042 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:24:07.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:07.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:07.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:07.178 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:07.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:07.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:07.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:07.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:07.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:07.423 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:07.433 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:07.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:07.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:07.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:07.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:07.661 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:07.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:07.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:07 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:24:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:07 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:24:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:07 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:24:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:07 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:24:07.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.810 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:07.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:24:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:24:07.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:07 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:24:07.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:24:07.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:07.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:07.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:07.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:08.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:08.074 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:08.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:08.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:08.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:08.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:08.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:08.325 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:08.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:08.427 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:08.452 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:08.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:08.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:08.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:08.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:08.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:24:08.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[29565]: pgmap v625: 148 pgs: 9 creating+peering, 4 unknown, 135 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[40649]: pgmap v625: 148 pgs: 9 creating+peering, 4 unknown, 135 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:24:08.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:08 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:24:08.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:24:08.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:08 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:24:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:08 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:24:08.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:08 smithi173 ceph-mon[33187]: pgmap v625: 148 pgs: 9 creating+peering, 4 unknown, 135 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:24:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:08 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:24:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:08 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:24:08.740 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:08.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:08.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:08.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:08.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:08.991 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:08.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:08.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:09.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:09.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:09.191 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:24:09.191 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:24:09.191 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:24:09.191 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:24:09.199 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:24:09.199 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:24:09.214 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:24:09.214 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:24:09.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:09.231 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:24:09.231 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:24:09.243 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:09.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:09.247 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:24:09.248 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:24:09.264 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:24:09.264 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:24:09.280 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:24:09.280 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:24:09.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:09.370 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:09.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:09.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:09.493 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:09.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:09.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:09.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.620 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:09.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:09.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2496755615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:09.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:09 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:24:09.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:09.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:09.789 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:24:09.789 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:24:09.789 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:24:09.789 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2496755615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:24:09.790 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:24:09.791 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2496755615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:09.791 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:09 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:24:09.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:09.868 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:09.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:09.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:09.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:09.993 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:09.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:10.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:10.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:10.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:10.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:10.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:10.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:10.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:10.350 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:10.368 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:10.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:10.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:10.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:10.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:10.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:10.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:10.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:10.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:10.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:10 smithi173 ceph-mon[33187]: pgmap v626: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:10 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:24:10.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:10 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:24:10.740 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:10.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:10.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[40649]: pgmap v626: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:10.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:24:10.802 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:24:10.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[29565]: pgmap v626: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:10.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:24:10.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:10 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:24:10.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:10.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.864 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:10.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:10.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:10.992 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:10.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:10.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:11.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:24:11.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-17T08:24:11.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:11.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.116 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:11.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:11.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:11.239 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:11.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:11.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:11.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.363 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:11.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:11.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:11.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:11.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:11.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:11.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:11.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:11.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:11.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:11 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:24:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:11 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:24:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:11 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:24:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:11 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:24:11.733 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:11.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:11.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:11.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.858 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:11.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:24:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:24:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:24:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:24:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:24:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:24:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:24:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:11 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:24:11.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:11.980 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:11.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:11.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:12.082 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:12.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.105 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:12.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:12.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:12.228 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:12.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:12.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:12.352 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:12.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:12.454 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:12.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.478 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:12.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:12.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:12.600 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:12.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:12.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:12.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.724 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:12.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:12.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:12 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:24:12.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:12 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:24:12.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:12 smithi173 ceph-mon[33187]: pgmap v627: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:12.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:24:12.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:24:12.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[40649]: pgmap v627: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:12.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:24:12.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:24:12.763 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:12 smithi079 ceph-mon[29565]: pgmap v627: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:12.825 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:24:12.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:12.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.850 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:12.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:12.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:12.973 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:12.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:12.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:13.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:13.097 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:13.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:13.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:13.216 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:13.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:13.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:13.240 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:13.244 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:13.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:13.343 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:13.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:13.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:13.358 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:13.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:13.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:13.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:13.473 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-17T08:24:13.473 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:24:13.570 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:13.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:13.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:13.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/349099843' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/295121242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/349099843' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:24:13.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:24:13.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/295121242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:13.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.719 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:13.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:24:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/349099843' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/295121242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:13.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:13.842 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:13.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:13.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:13.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:13.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:13.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:14.068 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:14.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:14.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.091 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:14.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:14.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:14.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:14.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:14.336 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:14.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.338 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:14.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:14.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:14.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:14.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:14.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:14.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:14.685 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:14.706 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:14.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: pgmap v629: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 635 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:14.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:14.761 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: pgmap v629: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 635 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:24:14.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:14 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:24:14.808 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:14.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:14.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:14.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:14.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:14.953 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:14.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: pgmap v629: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 635 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:24:14.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:14 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:24:15.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:15.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:15.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:15.178 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:15.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:15.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:15.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:15.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:15.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:15.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:15.544 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:15.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:15.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:15.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.668 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:15.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:15.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:15.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:15.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:15.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:15 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:15.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:15.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:15.917 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:15.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:15 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:16.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:16.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.040 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:16.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:16.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.164 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:16.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:16.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.287 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:16.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:16.410 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.411 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:16.512 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:16.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:16.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:16.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:16.660 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:16.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:16.784 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.785 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:16.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[40649]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[40649]: pgmap v631: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[29565]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[29565]: pgmap v631: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:24:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:16 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:24:16.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:16.907 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:16.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:16.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:16.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:16 smithi173 ceph-mon[33187]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T08:24:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:16 smithi173 ceph-mon[33187]: pgmap v631: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-17T08:24:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:16 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:24:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:16 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:24:17.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:17.035 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:17.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:17.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:17.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:17.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:17.287 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:17.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:17.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:17.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.411 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:17.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:17.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:17.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:17.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:17.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:17.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:17.657 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:17.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:17.780 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:17.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:17.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:17.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:17.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:24:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:17.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:18.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:18.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:18.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:18.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:18.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:18.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:18.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:18.278 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:18.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:18.301 DEBUG:teuthology.orchestra.run.smithi079:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-17T08:24:18.322 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-17T08:24:18.322 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-17T08:24:18.377 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:hints = 1 2024-09-17T08:24:18.377 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-17T08:24:18.377 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object prefix: benchmark_data_smithi079_172150 2024-09-17T08:24:18.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:24:18.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 0 0 0 0 0 0 - 0 2024-09-17T08:24:18.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:18.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:18.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:18.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:18.557 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:18.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:18.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:24:18.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:18.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[40649]: pgmap v632: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:24:18.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:24:18.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:24:18.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[29565]: pgmap v632: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:24:18.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:24:18.675 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:18 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:24:18.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:18.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:18.809 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:18.836 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:18.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:18 smithi173 ceph-mon[33187]: pgmap v632: 148 pgs: 148 active+clean; 582 KiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:24:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:18 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:24:18.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:18 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:24:18.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:18.968 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:18.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:18.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:19.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:19.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:19.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:19.247 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:19.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:19.377 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 1 16 2808 2792 174.492 174.5 0.00528917 0.00571103 2024-09-17T08:24:19.387 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:19.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:19.511 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T08:24:19.526 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:19.625 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:24:19.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:24:19.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:24:19.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:24:19.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3638663831' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:19 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:19.666 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:19.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:19.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:19.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3638663831' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:24:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:24:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:24:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:24:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3638663831' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:19 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:24:19.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:19.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:19.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:20.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:20.082 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:20.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:20.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:20.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:20.233 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:24:20.233 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:24:20.233 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:24:20.233 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:24:20.233 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:24:20.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:20.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.352 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:20.377 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 2 16 5637 5621 175.645 176.812 0.00541422 0.00568363 2024-09-17T08:24:20.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:20.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:20.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:20.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:20.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:20.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:20.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:20.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: pgmap v634: 148 pgs: 148 active+clean; 26 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.2 MiB/s wr, 68 op/s 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:24:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T08:24:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: pgmap v634: 148 pgs: 148 active+clean; 26 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.2 MiB/s wr, 68 op/s 2024-09-17T08:24:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:24:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:20 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:24:20.887 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:20.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:20.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: pgmap v634: 148 pgs: 148 active+clean; 26 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.2 MiB/s wr, 68 op/s 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:24:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:20 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:24:20.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:21.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:21.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:21.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:21.161 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:21.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:21.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:21.294 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:21.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.296 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:21.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 3 15 8945 8930 186.028 206.812 0.00624712 0.00536686 2024-09-17T08:24:21.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:21.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:21.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:21.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:24:21.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:21.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:21.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:21.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:21.695 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:21.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:21.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:21.832 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:21.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.833 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:21.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:24:21.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:24:21.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[40649]: 4.f deep-scrub starts 2024-09-17T08:24:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[40649]: 4.f deep-scrub ok 2024-09-17T08:24:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:24:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:24:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[29565]: 4.f deep-scrub starts 2024-09-17T08:24:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:21 smithi079 ceph-mon[29565]: 4.f deep-scrub ok 2024-09-17T08:24:21.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:21.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-17T08:24:21.963 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:21.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:21.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:21 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:24:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:21 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:24:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:21 smithi173 ceph-mon[33187]: 4.f deep-scrub starts 2024-09-17T08:24:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:21 smithi173 ceph-mon[33187]: 4.f deep-scrub ok 2024-09-17T08:24:22.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:22.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.111 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:22.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:22.212 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:22.237 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:22.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:22.308 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:22.312 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:22.312 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:24:22.312 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:24:22.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:22.356 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:24:22.356 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:24:22.364 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:22.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:22.376 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:22.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 4 16 11573 11557 180.556 164.188 0.00305971 0.00552494 2024-09-17T08:24:22.394 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:24:22.394 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:24:22.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:24:22.428 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:24:22.460 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:24:22.460 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:24:22.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:22.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:24:22.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:24:22.505 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:22.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:22.525 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:24:22.526 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:24:22.557 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:24:22.557 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:24:22.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:22.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:22.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: pgmap v635: 148 pgs: 148 active+clean; 131 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2044018204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:24:22.733 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:24:22.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: pgmap v635: 148 pgs: 148 active+clean; 131 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-17T08:24:22.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:24:22.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:24:22.734 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2044018204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:22.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:22.785 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:22.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:22.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:22.919 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:22.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:22.920 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: pgmap v635: 148 pgs: 148 active+clean; 131 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 MiB/s wr, 261 op/s 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:24:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2044018204' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:23.021 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:23.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.053 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.053 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:23.154 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:23.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.186 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:23.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:23.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:23.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 5 16 14242 14226 177.806 166.812 0.00754489 0.00561719 2024-09-17T08:24:23.415 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:23.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:23.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:23.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:23.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:23.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:23.710 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:23.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:23.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:23.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:24:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:24:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:23 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:24:23.950 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:24:23.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:23 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:24:23.982 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:23.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:23.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:24.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:24.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:24.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:24.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:24.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.250 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:24.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:24.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:24.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 6 15 17070 17055 177.636 176.812 0.00469729 0.00562413 2024-09-17T08:24:24.387 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:24.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:24.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:24.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:24.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:24.658 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:24.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:24.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:24.796 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:24.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:24 smithi079 ceph-mon[29565]: pgmap v636: 148 pgs: 148 active+clean; 323 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 638 B/s rd, 40 MiB/s wr, 643 op/s 2024-09-17T08:24:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:24 smithi079 ceph-mon[40649]: pgmap v636: 148 pgs: 148 active+clean; 323 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 638 B/s rd, 40 MiB/s wr, 643 op/s 2024-09-17T08:24:24.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:24.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:24.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:24.925 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:24 smithi173 ceph-mon[33187]: pgmap v636: 148 pgs: 148 active+clean; 323 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 638 B/s rd, 40 MiB/s wr, 643 op/s 2024-09-17T08:24:25.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:25.055 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:25.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:25.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:25.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:25.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:25.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:25.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.316 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:25.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:25.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 7 16 19305 19289 172.204 139.625 0.00125912 0.00578429 2024-09-17T08:24:25.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:25.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:25.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:25.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:25.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:25.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:25.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:25.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:25.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:25.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:25.851 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:25.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 4.1c deep-scrub starts 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 4.1c deep-scrub ok 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 1.0 deep-scrub starts 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: 1.0 deep-scrub ok 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[29565]: pgmap v637: 148 pgs: 148 active+clean; 823 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 4.1c deep-scrub starts 2024-09-17T08:24:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 4.1c deep-scrub ok 2024-09-17T08:24:25.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 1.0 deep-scrub starts 2024-09-17T08:24:25.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: 1.0 deep-scrub ok 2024-09-17T08:24:25.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:25 smithi079 ceph-mon[40649]: pgmap v637: 148 pgs: 148 active+clean; 823 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-17T08:24:25.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 4.1c deep-scrub starts 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 4.1c deep-scrub ok 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 1.0 deep-scrub starts 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: 1.0 deep-scrub ok 2024-09-17T08:24:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:25 smithi173 ceph-mon[33187]: pgmap v637: 148 pgs: 148 active+clean; 823 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-17T08:24:25.992 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:25.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:25.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:26.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:26.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:26.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:26.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:26.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:26.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:26.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:26.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 8 16 21745 21729 169.739 152.5 0.00167056 0.00586492 2024-09-17T08:24:26.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:26.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:26.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:26.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:26.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:26.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:26.658 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:26.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:26.759 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:26.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.781 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:26.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:26 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:24:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:26 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:24:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:26 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:24:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:26 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:24:26.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:26.919 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:26.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:26.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:26 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:24:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:26 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:24:27.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:27.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:27.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:27.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:27.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:27.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:27.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:27.309 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:27.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.310 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:27.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 9 16 23506 23490 163.108 110.062 0.0134621 0.00612346 2024-09-17T08:24:27.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:27.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.448 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:27.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:27.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:27.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.579 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:27.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:27.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:27.716 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:27.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:27.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:27.848 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:27.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[40649]: pgmap v638: 148 pgs: 148 active+clean; 968 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:24:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:24:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:24:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:24:27.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:24:27.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:27.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:27.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:27 smithi079 ceph-mon[29565]: pgmap v638: 148 pgs: 148 active+clean; 968 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:24:27.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:24:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:24:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:24:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:24:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:27 smithi173 ceph-mon[33187]: pgmap v638: 148 pgs: 148 active+clean; 968 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-17T08:24:27.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:27.989 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:27.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:28.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:28.120 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:28.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:28.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:28.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.259 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:28.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:28.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:28.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 10 16 25947 25931 162.052 152.562 0.0067315 0.00616491 2024-09-17T08:24:28.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:28.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.397 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:28.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:28.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:28.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:28.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:28.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:28.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:28.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:28.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:28.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:28.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:24:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:24:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[29565]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T08:24:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:24:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:24:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:28 smithi079 ceph-mon[40649]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T08:24:28.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:28.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:28.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:28.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:28 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:24:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:28 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:24:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:28 smithi173 ceph-mon[33187]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T08:24:29.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:29.055 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:29.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:29.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:29.185 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:29.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:29.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:29.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:29.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:29.378 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 11 16 27846 27830 158.109 118.688 0.00277226 0.00631966 2024-09-17T08:24:29.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:29.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:29.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:29.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:29.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:29.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:29.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:29.733 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:29.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:29.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:29.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:24:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:24:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[29565]: pgmap v640: 148 pgs: 148 active+clean; 1.5 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-17T08:24:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:24:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:24:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:29 smithi079 ceph-mon[40649]: pgmap v640: 148 pgs: 148 active+clean; 1.5 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-17T08:24:29.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:29.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:29.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:29.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:29 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:24:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:29 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:24:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:29 smithi173 ceph-mon[33187]: pgmap v640: 148 pgs: 148 active+clean; 1.5 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-17T08:24:30.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:30.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:30.111 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:30.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.142 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:30.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:30.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:30.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:30.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:30.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 12 15 30226 30211 157.333 148.812 0.00718802 0.00635131 2024-09-17T08:24:30.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:30.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:30.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:30.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:30.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:30.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:30.659 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:30.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:30.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:24:30.718 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:24:30.719 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:30 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:24:30.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:30.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.830 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:30.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:30.837 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:24:30.837 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:24:30.837 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:24:30.838 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:24:30.838 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:24:30.838 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:24:30.838 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:24:30.838 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:30 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:24:30.931 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:30.964 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:30.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:30.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:31.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:31.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:31.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:31.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:31.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.253 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:31.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:31.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:31.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 13 16 32423 32407 155.787 137.25 0.00663539 0.00641386 2024-09-17T08:24:31.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:31.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:31.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:31.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:31.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:31.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:31.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:31.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:31.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:31.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:31.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:31.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[40649]: 5.f deep-scrub starts 2024-09-17T08:24:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[40649]: 5.f deep-scrub ok 2024-09-17T08:24:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[40649]: pgmap v641: 148 pgs: 148 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-17T08:24:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[29565]: 5.f deep-scrub starts 2024-09-17T08:24:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[29565]: 5.f deep-scrub ok 2024-09-17T08:24:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:31 smithi079 ceph-mon[29565]: pgmap v641: 148 pgs: 148 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-17T08:24:31.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:31.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:31.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:31.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:31 smithi173 ceph-mon[33187]: 5.f deep-scrub starts 2024-09-17T08:24:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:31 smithi173 ceph-mon[33187]: 5.f deep-scrub ok 2024-09-17T08:24:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:31 smithi173 ceph-mon[33187]: pgmap v641: 148 pgs: 148 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-17T08:24:32.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:32.069 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:32.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:32.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:32.205 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:32.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:32.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:32.348 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:32.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:32.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 14 16 34570 34554 154.24 134.188 0.00438389 0.0064783 2024-09-17T08:24:32.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:32.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:32.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:32.585 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:32.613 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:32.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:32.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:32.748 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:32.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:32.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:24:32.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:24:32.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:24:32.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[40649]: 2.7 deep-scrub starts 2024-09-17T08:24:32.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[40649]: 2.7 deep-scrub ok 2024-09-17T08:24:32.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:32.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:24:32.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:24:32.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[29565]: 2.7 deep-scrub starts 2024-09-17T08:24:32.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[29565]: 2.7 deep-scrub ok 2024-09-17T08:24:32.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:32.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:32.896 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:32.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:32.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:32 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:24:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:32 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:24:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:32 smithi173 ceph-mon[33187]: 2.7 deep-scrub starts 2024-09-17T08:24:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:32 smithi173 ceph-mon[33187]: 2.7 deep-scrub ok 2024-09-17T08:24:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:32.998 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:33.026 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:33.120 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:33.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:33.169 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:33.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:33.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:33.308 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:33.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:33.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 15 16 36511 36495 152.044 121.312 0.00269217 0.00656656 2024-09-17T08:24:33.410 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:33.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.432 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:33.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:33.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:33.571 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:33.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:24:33.623 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:24:33.624 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:33.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:24:33.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:24:33.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:33.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/718187988' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[29565]: pgmap v642: 148 pgs: 148 active+clean; 1.8 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.44k op/s 2024-09-17T08:24:33.628 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:24:33.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:33.705 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:33.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:33.837 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/718187988' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:33 smithi079 ceph-mon[40649]: pgmap v642: 148 pgs: 148 active+clean; 1.8 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.44k op/s 2024-09-17T08:24:33.939 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/718187988' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:24:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:33 smithi173 ceph-mon[33187]: pgmap v642: 148 pgs: 148 active+clean; 1.8 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 153 MiB/s wr, 2.44k op/s 2024-09-17T08:24:33.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:33.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:33.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:34.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:34.112 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:34.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:34.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:34.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:34.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:34.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:34.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 16 16 38247 38231 149.322 108.5 0.00642933 0.00669173 2024-09-17T08:24:34.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:34.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:34.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:34.519 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:34.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:34.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:34.652 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:34.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:34.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:34.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:34.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[29565]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T08:24:34.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:24:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:24:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:24:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:34 smithi079 ceph-mon[40649]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T08:24:34.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:34.911 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:34.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:34.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:34 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:24:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:34 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:24:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:34 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:34 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:24:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:34 smithi173 ceph-mon[33187]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T08:24:35.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:35.033 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:35.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:35.135 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:35.166 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:35.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:35.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:35.289 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:35.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:35.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 17 15 40076 40061 147.266 114.375 0.0187842 0.0067852 2024-09-17T08:24:35.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:35.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:35.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:35.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:35.550 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:35.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:35.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:35.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:35.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:35.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:35.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:35.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:35.916 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:35.917 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:35.917 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[40649]: pgmap v644: 148 pgs: 148 active+clean; 2.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:24:35.917 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:35.917 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:35.917 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:35 smithi079 ceph-mon[29565]: pgmap v644: 148 pgs: 148 active+clean; 2.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:24:35.937 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:35.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:35.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:35.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:35 smithi173 ceph-mon[33187]: pgmap v644: 148 pgs: 148 active+clean; 2.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-17T08:24:36.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:36.108 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:36.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:36.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:36.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:36.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 18 16 41975 41959 145.674 118.625 0.0102141 0.0068581 2024-09-17T08:24:36.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:36.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:36.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:36.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:36.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:36.755 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:36 smithi079 ceph-mon[29565]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T08:24:36.755 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:36 smithi079 ceph-mon[29565]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T08:24:36.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:36 smithi079 ceph-mon[40649]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T08:24:36.755 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:36 smithi079 ceph-mon[40649]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T08:24:36.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:36.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:36.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:36.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:36.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:36.948 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:36.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:36.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:36 smithi173 ceph-mon[33187]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T08:24:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:36 smithi173 ceph-mon[33187]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T08:24:37.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:37.080 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:37.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:37.086 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:24:37.086 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:24:37.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:37.216 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:37.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:37.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:37.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:37.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:37.379 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 19 16 43827 43811 144.099 115.75 0.0047138 0.00693407 2024-09-17T08:24:37.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:37.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.490 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:37.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:37.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:37.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:37.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:37.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:37.765 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:37 smithi079 ceph-mon[29565]: pgmap v647: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-17T08:24:37.765 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:37 smithi079 ceph-mon[40649]: pgmap v647: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-17T08:24:37.769 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:37.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:37.871 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:37.905 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:37.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:37.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:37 smithi173 ceph-mon[33187]: pgmap v647: 148 pgs: 148 active+clean; 2.3 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-17T08:24:38.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:38.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:38.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:38.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:38.157 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:38.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:38.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:38.298 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:38.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:38.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.331 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:38.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:24:38.380516+0000 min lat: 0.00114729 max lat: 0.178956 avg lat: 0.0069992 2024-09-17T08:24:38.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:24:38.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 20 16 45684 45668 142.696 116.062 0.00391014 0.0069992 2024-09-17T08:24:38.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:38.455 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:38.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:38.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:24:38.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:38.587 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:38.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:38.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:38.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:38.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:38.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:38.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:38.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:38.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:38.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:39.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:39.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:39.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:39.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:39.270 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:39.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:39.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 21 16 47296 47280 140.698 100.75 0.00518998 0.00710265 2024-09-17T08:24:39.398 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:39.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:39.522 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:39.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:39.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:39.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:39.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:39.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:39 smithi079 ceph-mon[40649]: pgmap v648: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:24:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:39 smithi079 ceph-mon[29565]: pgmap v648: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:24:39.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:39.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:39.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:39.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:39 smithi173 ceph-mon[33187]: pgmap v648: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 159 MiB/s wr, 2.54k op/s 2024-09-17T08:24:40.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:40.061 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:40.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:40.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:40.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:40.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:40.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:40.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:40.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 22 16 48763 48747 138.47 91.6875 0.00487588 0.00721692 2024-09-17T08:24:40.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:40.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:40.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:40.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:40.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.596 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:40.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:40.698 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:40.730 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:40.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:40.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:40.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:40.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:40.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:40.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:41.012 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:41.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:41.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:41.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.153 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:41.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:41.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:41.292 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:41.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:41.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 23 16 50499 50483 137.167 108.5 0.00586772 0.00728572 2024-09-17T08:24:41.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:41.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:41.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:41.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:41.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:41.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:24:41.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:41.701 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:41.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:41.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:41.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:41.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:41.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:41.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.011 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:42.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:42.152 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:42.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:42.286 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:42.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:42.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 24 16 52061 52045 135.519 97.625 0.0160312 0.00737398 2024-09-17T08:24:42.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T08:24:42.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:42.558 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:42.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:42.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:42.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:42.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:42 smithi173 ceph-mon[33187]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T08:24:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:42 smithi173 ceph-mon[33187]: pgmap v650: 148 pgs: 148 active+clean; 2.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-17T08:24:42.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:24:42.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[40649]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T08:24:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[40649]: pgmap v650: 148 pgs: 148 active+clean; 2.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-17T08:24:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:24:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[29565]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T08:24:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[29565]: pgmap v650: 148 pgs: 148 active+clean; 2.8 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-17T08:24:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:24:42.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:42.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:42.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:42.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:43.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:43.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:43.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:43.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:43.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:43.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:43.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:43.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:43.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:43.380 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 25 16 53494 53478 133.68 89.5625 0.00812692 0.00747552 2024-09-17T08:24:43.452 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:43.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:43.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.498 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:43.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:43.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:43.633 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:43.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:43 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:24:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:43 smithi173 ceph-mon[33187]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T08:24:43.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:43.768 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:43.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:24:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:43 smithi079 ceph-mon[29565]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T08:24:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:43 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/95461163' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:24:43.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:43 smithi079 ceph-mon[40649]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T08:24:43.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:43.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:43.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:43.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:43.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:44.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:44.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:44.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:44.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:44.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:44.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:44.382 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 26 16 54923 54907 131.963 89.3125 0.0125486 0.00757289 2024-09-17T08:24:44.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:44.463 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:44.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:44.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:44.594 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:44.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:44 smithi079 ceph-mon[40649]: pgmap v652: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 721 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:24:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:44 smithi079 ceph-mon[29565]: pgmap v652: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 721 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:24:44.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:44.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:44 smithi173 ceph-mon[33187]: pgmap v652: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 721 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:24:44.727 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:44.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:44.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:44.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.861 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:44.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:44.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:44.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:44.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:44.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:45.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:45.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-17T08:24:45.126 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:45.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:45.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:45.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.278 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:45.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:45.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:45.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 27 16 56591 56575 130.936 104.25 0.00494528 0.00763241 2024-09-17T08:24:45.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:45.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:45.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:45.557 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:45.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:45.582 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:45.659 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:45.688 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:45.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.689 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:45.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:45.823 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:45.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:45 smithi079 ceph-mon[40649]: pgmap v653: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:24:45.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:45 smithi079 ceph-mon[29565]: pgmap v653: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:24:45.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:45.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:45.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:45.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:45 smithi173 ceph-mon[33187]: pgmap v653: 148 pgs: 148 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-17T08:24:46.074 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:46.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.109 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:46.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:46.210 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:46.383 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 28 16 58108 58092 129.644 94.8125 0.0111953 0.00770791 2024-09-17T08:24:46.393 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:46.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:46.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:46.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.545 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:46.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:46.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:46.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:46.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.774 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[40649]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.775 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:46 smithi079 ceph-mon[29565]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T08:24:46.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:46.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:46.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:46.971 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:46.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:46.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:24:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:24:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:46 smithi173 ceph-mon[33187]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T08:24:47.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:47.105 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:47.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.109 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:47.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:47.210 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:47.242 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:47.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:47.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:47.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:47.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:47.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 29 16 59644 59628 128.481 96 0.027063 0.00777695 2024-09-17T08:24:47.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:47.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:47.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:47.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:47.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:47.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:47.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:47.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:47.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:47.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:47.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:47.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:48.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:48.071 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:48.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:48.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:48.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:48.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:48.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:48.337 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:48.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:48.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 30 16 61226 61210 127.494 98.875 0.0171964 0.00783795 2024-09-17T08:24:48.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:48.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:48.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:48.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:48.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:48.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[29565]: pgmap v655: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 840 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:48.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:48 smithi079 ceph-mon[40649]: pgmap v655: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 840 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-17T08:24:48.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:24:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:24:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:48 smithi173 ceph-mon[33187]: pgmap v655: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 840 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-17T08:24:48.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.735 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:48.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:48.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:48.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:48.877 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:48.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:48.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:49.007 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:49.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:49.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:49.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:49.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:49.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:49.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.285 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:49.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:49.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 31 16 62607 62591 126.165 86.3125 0.00829313 0.00792105 2024-09-17T08:24:49.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:49.417 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:49.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:49.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:49.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:49.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:49.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:49.683 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:49.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:49.785 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:49.816 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:49.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:49.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:49 smithi079 ceph-mon[40649]: pgmap v656: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:24:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:49 smithi079 ceph-mon[29565]: pgmap v656: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:24:49.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:49.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:49.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:49.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:49.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:49 smithi173 ceph-mon[33187]: pgmap v656: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-17T08:24:50.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:50.078 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:50.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:50.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:50.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:50.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:50.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:50.346 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:50.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.347 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:50.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 32 16 63837 63821 124.625 76.875 0.023503 0.00801786 2024-09-17T08:24:50.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:50.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.490 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:50.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:50.579 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:24:50.579 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:24:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:50.592 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:50.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:24:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:24:50.617 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:50.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:50.642 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:24:50.642 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:24:50.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:24:50.679 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:24:50.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:24:50.715 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:24:50.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:50.748 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:50.748 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:24:50.749 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:24:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:50.782 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:24:50.783 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:24:50.816 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:24:50.816 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:24:50.850 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:50.857 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2028260089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:50.857 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2028260089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:50.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:50.881 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:50.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2028260089' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:24:50.983 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:51.006 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:51.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:51.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:51.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:51.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:51.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:51.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 33 16 65084 65068 123.21 77.9375 0.00570506 0.00811068 2024-09-17T08:24:51.402 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:51.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:51.537 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:51.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:51.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.684 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:51.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:51.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:51.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.829 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:51.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:51.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:51.959 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:51.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:51.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:52.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:52.097 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:52.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:52.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:52.230 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:52.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:52.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:52.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:52.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:52.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 34 16 66526 66510 122.237 90.125 0.00664614 0.00817479 2024-09-17T08:24:52.476 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:52.512 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:52.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:52.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:52 smithi079 ceph-mon[29565]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T08:24:52.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:52 smithi079 ceph-mon[29565]: pgmap v658: 148 pgs: 148 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:24:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:52 smithi079 ceph-mon[40649]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T08:24:52.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:52 smithi079 ceph-mon[40649]: pgmap v658: 148 pgs: 148 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:24:52.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:52.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:52 smithi173 ceph-mon[33187]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T08:24:52.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:52 smithi173 ceph-mon[33187]: pgmap v658: 148 pgs: 148 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:24:52.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:52.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:52.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:52.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:52.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:52.934 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:52.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:53.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:53.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:53.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:53.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:24:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:24:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:24:53.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:53.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:53.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.215 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:53.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:53.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:53.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:53.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 35 16 67947 67931 121.282 88.8125 0.0135547 0.00824005 2024-09-17T08:24:53.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:53.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:24:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:24:53.480 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:24:53.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:53.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:53.586 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:53.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.617 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:53.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1402387628' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:53 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1402387628' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:53 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.707 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:68369 10.b 10:d809e126:::benchmark_data_smithi079_172150_object68368:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e301)", 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:24:53.702778+0000", 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0047370620000000002, 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047605160000000002, 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:24:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 68369 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702778+0000", 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702778+0000", 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702777+0000", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702825+0000", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8306999999999997e-05 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702838+0000", 2024-09-17T08:24:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2751e-05 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702845+0000", 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9029999999999998e-06 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.702893+0000", 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8177999999999998e-05 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:24:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.703036+0000", 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014331700000000001 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.703181+0000", 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144523 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.705988+0000", 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028076379999999999 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.706004+0000", 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5787e-05 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:68372 10.2 10:44198d90:::benchmark_data_smithi079_172150_object68371:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e301)", 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:24:53.704278+0000", 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0032365689999999999, 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033137230000000002, 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:24:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 68372 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704278+0000", 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:24:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704278+0000", 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704277+0000", 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:24:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704301+0000", 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4544000000000001e-05 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704314+0000", 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2959000000000001e-05 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704321+0000", 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1930000000000004e-06 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704368+0000", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7052999999999998e-05 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704489+0000", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012066300000000001 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,4", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.704611+0000", 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012166 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:24:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:24:53.707153+0000", 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025418440000000001 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 2 2024-09-17T08:24:53.718 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:24:53.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1402387628' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:53 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:24:53.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:53.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:53.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:53.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:53.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:53.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:53.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:54.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:54.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:54.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:54.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:54.142 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:54.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:54.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:54.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:54.377 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:54.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 36 16 69269 69253 120.208 82.625 0.0106867 0.00831418 2024-09-17T08:24:54.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:54.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:54.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:54.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:54.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:54.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:54.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:54.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:54.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T08:24:54.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: pgmap v660: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 722 B/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-17T08:24:54.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:24:54.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:54 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:24:54.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:54.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:24:54.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:54.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:54.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:54.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:54.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: pgmap v660: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 722 B/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T08:24:54.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: pgmap v660: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 722 B/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-17T08:24:54.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:24:54.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:54 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:24:54.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:54.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:54.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:54.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:55.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:55.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.102 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:55.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:55.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:55.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:55.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:55.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:55.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:55.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 37 16 70689 70673 119.357 88.75 0.0120243 0.00837348 2024-09-17T08:24:55.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:55.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.539 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:55.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:55.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:55.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:55.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:55.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:55.814 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:55.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:55.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:55 smithi079 ceph-mon[29565]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T08:24:55.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:55 smithi079 ceph-mon[40649]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T08:24:55.916 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:55.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:55.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:55.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:55 smithi173 ceph-mon[33187]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T08:24:56.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:56.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.083 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:56.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:56.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:24:56.215 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:56.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:24:56.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:56.348 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:56.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:56.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 38 16 72129 72113 118.585 90 0.00817555 0.00842739 2024-09-17T08:24:56.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:56.497 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:56.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:56.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:56.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:56.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:56.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:56.774 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:56.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:56 smithi079 ceph-mon[40649]: pgmap v662: 148 pgs: 148 active+clean; 4.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:24:56.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:56 smithi079 ceph-mon[29565]: pgmap v662: 148 pgs: 148 active+clean; 4.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:24:56.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:56.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:56.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:56.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:56 smithi173 ceph-mon[33187]: pgmap v662: 148 pgs: 148 active+clean; 4.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:24:57.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:57.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:57.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:57.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.214 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:57.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:57.350 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:57.384 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 39 16 73435 73419 117.637 81.625 0.0130087 0.00849586 2024-09-17T08:24:57.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:24:57.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.487 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:24:57.518 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T08:24:57.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:24:57.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:57.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:57.779 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1676681587' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:57 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:57.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1676681587' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:57.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:57 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:57.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:57.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:57.928 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:57.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1676681587' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:57 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:24:58.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:58.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:58.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:58.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:24:58.213 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:58.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:58.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:58.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:58.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:58.381 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:24:58.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:24:58.385559+0000 min lat: 0.00114729 max lat: 0.178956 avg lat: 0.00855932 2024-09-17T08:24:58.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:24:58.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 40 16 74746 74730 116.744 81.9375 0.0148447 0.00855932 2024-09-17T08:24:58.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-17T08:24:58.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:58.494 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:58.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:58.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:58.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.629 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:58.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:58.730 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:24:58.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.756 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:58.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:24:58.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:58.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:24:58.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[29565]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T08:24:58.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[29565]: pgmap v664: 148 pgs: 148 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:24:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:24:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[40649]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T08:24:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:24:58 smithi079 ceph-mon[40649]: pgmap v664: 148 pgs: 148 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:24:58.891 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:58.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:58.892 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:58 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:24:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:58 smithi173 ceph-mon[33187]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T08:24:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:24:58 smithi173 ceph-mon[33187]: pgmap v664: 148 pgs: 148 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:24:58.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:24:59.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.031 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:59.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:24:59.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:59.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.174 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:59.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:59.275 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:59.315 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:59.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 41 16 75972 75956 115.766 76.625 0.0133933 0.0086332 2024-09-17T08:24:59.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:24:59.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.455 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:59.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:24:59.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:24:59.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.598 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:59.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:24:59.700 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:24:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:24:59.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:24:59.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:24:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:24:59.878 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:24:59.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:24:59.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:00.008 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:00.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:00.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:00.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:00.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:00.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:00.264 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:00.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:00.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:00.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 42 16 77219 77203 114.865 77.9375 0.00842504 0.00870095 2024-09-17T08:25:00.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:00.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:00.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:00.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.538 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:00.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:00.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:00.677 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:00.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:00.779 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:00.817 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:00.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:00.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:00 smithi079 ceph-mon[29565]: pgmap v665: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:25:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:00 smithi079 ceph-mon[40649]: pgmap v665: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:25:00.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:00.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:00.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:00.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:00 smithi173 ceph-mon[33187]: pgmap v665: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.77k op/s 2024-09-17T08:25:01.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:01.093 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.094 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:01.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:01.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.237 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:01.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:01.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:01.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:01.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:01.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 43 16 78440 78424 113.968 76.3125 0.0140926 0.00876935 2024-09-17T08:25:01.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:01.503 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:01.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:01.605 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:01.635 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:01.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:01.701 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:01.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:01.775 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:01.779 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:01.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.784 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:01.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:01.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:01.916 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:01.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:01.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:02.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:02.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.070 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:02.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:02.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:02.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:02.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:02.305 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:02.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.337 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:02.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:02.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 44 16 79711 79695 113.183 79.4375 0.00530059 0.00882971 2024-09-17T08:25:02.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:02.472 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.473 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:02.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[29565]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T08:25:02.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[29565]: pgmap v667: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 727 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:25:02.541 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[40649]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T08:25:02.541 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[40649]: pgmap v667: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 727 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:25:02.542 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:02 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:02.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:02.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:02.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:02.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:02 smithi173 ceph-mon[33187]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T08:25:02.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:02 smithi173 ceph-mon[33187]: pgmap v667: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 727 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:25:02.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:02 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:02.746 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:02.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:02.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:02.882 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:02.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:02.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:02.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:03.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.020 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:03.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:03.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:03.123 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:03.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:03.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:03.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:03.299 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:03.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:03.385 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 45 16 81032 81016 112.503 82.5625 0.0246787 0.00888404 2024-09-17T08:25:03.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:03.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.438 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:03.438 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:03.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:03.539 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:03.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.576 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:03.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:03.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:03.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.718 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:03.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:03.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:03.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:03.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:03 smithi079 ceph-mon[40649]: pgmap v668: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:03 smithi079 ceph-mon[29565]: pgmap v668: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:03.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:03 smithi173 ceph-mon[33187]: pgmap v668: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:03.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:03.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:03.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:04.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:04.143 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:04.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:04.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:04.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:04.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:04.387 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 46 16 82356 82340 111.854 82.75 0.0120517 0.0089353 2024-09-17T08:25:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:04.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:04.555 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:04.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:04.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.701 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:04.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:04.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:04.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:04.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:04.975 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:04.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:05.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:05.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:05.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:05.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:05.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:05.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:05.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:05.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:05.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:05.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 47 16 83659 83643 111.207 81.4375 0.0157467 0.00898729 2024-09-17T08:25:05.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.536 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:05.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:05.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:05.669 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:05.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:05.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:05.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:05.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:05.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:05.951 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:05.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:05.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:06.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:06.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:06.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:06.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:06.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:06.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:06.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:06.356 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:06.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:06.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 48 16 84822 84806 110.404 72.6875 0.0105663 0.00905288 2024-09-17T08:25:06.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:06.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:06.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:06.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:06.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:06.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:06.624 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[29565]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[29565]: pgmap v670: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3175517701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[40649]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[40649]: pgmap v670: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:25:06.625 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3175517701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:06.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:06 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:06 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:06 smithi173 ceph-mon[33187]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T08:25:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:06 smithi173 ceph-mon[33187]: pgmap v670: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-17T08:25:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3175517701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:06.728 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:06.751 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:06.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:06.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:06.891 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:06.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:06.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:06.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:07.031 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:07.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:07.133 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:07.160 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.161 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:07.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:07.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:07.385 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:07.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 49 16 85774 85758 109.366 59.5 0.0161268 0.00913886 2024-09-17T08:25:07.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:07.466 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T08:25:07.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:07.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:07.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:25:07.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:25:07.663 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:25:07.691 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:07.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:07 smithi173 ceph-mon[33187]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T08:25:07.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2320981839' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:07 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:07 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:07.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:07 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:07.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:07.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[40649]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T08:25:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2320981839' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[29565]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T08:25:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2320981839' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:25:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:07 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:07.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:07.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:07.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:07.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:08.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:08.107 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:08.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.108 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:08.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:08.243 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:08.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:08.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:08.378 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:08.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:08.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 50 16 87089 87073 108.822 82.1875 0.0126492 0.00918465 2024-09-17T08:25:08.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:08.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:08.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:08.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:08.652 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:08.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:08 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:25:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:08 smithi173 ceph-mon[33187]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T08:25:08.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:08 smithi173 ceph-mon[33187]: pgmap v673: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:08.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:08.782 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:08.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:25:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[29565]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T08:25:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[29565]: pgmap v673: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:25:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[40649]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T08:25:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:08 smithi079 ceph-mon[40649]: pgmap v673: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:08.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:08.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:08.927 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:08.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:09.028 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:09.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:09.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:09.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:09.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.202 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:09.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:09.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:09.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:09.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:09.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 51 16 88357 88341 108.242 79.25 0.0136214 0.00923424 2024-09-17T08:25:09.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:09.479 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:09.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:09.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:09.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:09.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:09.755 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:09.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:09.772 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:09 smithi079 ceph-mon[29565]: pgmap v674: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:25:09.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:09 smithi079 ceph-mon[40649]: pgmap v674: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:25:09.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:09.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:09.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:09.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:09.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:09 smithi173 ceph-mon[33187]: pgmap v674: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:25:09.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:10.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:10.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:10.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:10.258 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:10.291 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:10.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 52 16 89514 89498 107.551 72.3125 0.0198415 0.00929289 2024-09-17T08:25:10.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:10.435 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:10.537 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:10.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:10.680 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:10.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.709 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:10.810 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:10.837 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:25:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:25:10.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:25:10.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:10.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:10.974 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:10.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:25:11.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:11.103 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:11.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:11.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:11.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:11.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:11.323 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:25:11.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:11.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:11.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:11.386 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 53 16 90728 90712 106.953 75.875 0.0121488 0.00934558 2024-09-17T08:25:11.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:11.513 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:11.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:11.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:11.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.661 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:11.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:11.752 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-17T08:25:11.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:11.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.841 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:11.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:11.860 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:11.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:25:11.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[40649]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T08:25:11.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[40649]: pgmap v676: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:11.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:11.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:25:11.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[29565]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T08:25:11.862 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:11 smithi079 ceph-mon[29565]: pgmap v676: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:11.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:11 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:11 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/739732790' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:25:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:11 smithi173 ceph-mon[33187]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T08:25:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:11 smithi173 ceph-mon[33187]: pgmap v676: 148 pgs: 148 active+clean; 5.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:25:11.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:11.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:11.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:12.014 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:25:12.014 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:25:12.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:12.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.143 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:12.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:12.244 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:12.274 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:12.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:12.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:12.387 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 54 16 91905 91889 106.335 73.5625 0.0083746 0.00939923 2024-09-17T08:25:12.414 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:12.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:12.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:12.547 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:12.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:12.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:12.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:12.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:12.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:12.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:12.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:25:12.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:12 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:25:12.876 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:12.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:12 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:25:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:12 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:25:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:12 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:25:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:12 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:25:12.977 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:12.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:12.979 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:13.079 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:13.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:13.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:13.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.254 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:13.311 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:13.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:25:13.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:13.369 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:13.379 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:13.387 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 55 16 93163 93147 105.829 78.625 0.0216041 0.00944427 2024-09-17T08:25:13.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:13.481 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:13.509 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:13.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:13.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:13.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:13.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:13.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:13.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:13.920 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:13.920 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:13.920 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:25:13.921 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:25:13.922 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:13.922 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[40649]: pgmap v678: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 841 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:13.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:13.923 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:13.923 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:25:13.924 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:25:13.925 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:13.925 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:13 smithi079 ceph-mon[29565]: pgmap v678: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 841 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:13 smithi173 ceph-mon[33187]: pgmap v678: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 841 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:14.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:14.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:14.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:14.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:14.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.187 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:14.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:14.288 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:14.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:14.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:14.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 56 16 94315 94299 105.225 72 0.0117786 0.00949889 2024-09-17T08:25:14.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:14.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.454 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:14.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:14.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:14.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:14.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:14.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:25:14.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:14.724 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:14.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:14.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:14 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:25:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:14 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:25:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:14 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:25:14.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:14 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:25:14.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:14.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:14.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:14 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:25:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:14 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:25:14.974 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:15.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:15.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:15.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:15.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:15.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:15.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:15.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:15.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 57 16 95470 95454 104.645 72.1875 0.00853752 0.00955078 2024-09-17T08:25:15.420 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:15.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:15.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:15.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:15.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:15.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:15.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:15.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:15.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:15.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:15.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:15.920 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:15.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:15.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:15.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:25:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:15 smithi173 ceph-mon[33187]: pgmap v679: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-17T08:25:16.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:16.099 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:16.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[29565]: pgmap v679: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:25:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:25:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:25:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:25:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:25:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:25:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:15 smithi079 ceph-mon[40649]: pgmap v679: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-17T08:25:16.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:16.235 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:16.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:16.337 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:16.369 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:16.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:16.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 58 16 96681 96665 104.146 75.6875 0.00751396 0.00959704 2024-09-17T08:25:16.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:16.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:16.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:16.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:16.639 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:16.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:16.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:16.776 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:16.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:16.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:16.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:16.989 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:16 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:25:16.989 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:16 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:25:16.989 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:16 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:25:16.989 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:16 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:25:16.989 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:16 smithi173 ceph-mon[33187]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T08:25:17.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.026 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:17.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[40649]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:25:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:16 smithi079 ceph-mon[29565]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T08:25:17.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:17.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:17.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:17.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:17.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:17.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:17.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 59 16 97774 97758 103.539 68.3125 0.00980687 0.00965349 2024-09-17T08:25:17.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:17.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:17.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:17.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:17.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:17.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:17.700 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:17.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:17.802 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:17.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:17.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:17.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:25:17.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:25:17.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[29565]: pgmap v681: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 739 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:25:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:25:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:25:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:17 smithi079 ceph-mon[40649]: pgmap v681: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 739 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:25:17.932 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:17.965 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:17.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:17.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:17 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:25:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:17 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:25:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:17 smithi173 ceph-mon[33187]: pgmap v681: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 739 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-17T08:25:18.066 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:18.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:18.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:18.234 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:18.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:18.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:18.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:25:18.388962+0000 min lat: 0.00114729 max lat: 0.178956 avg lat: 0.00969159 2024-09-17T08:25:18.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:25:18.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 60 16 99043 99027 103.135 79.3125 0.0105814 0.00969159 2024-09-17T08:25:18.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:18.498 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:18.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:18.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:18.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:18.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.765 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:18.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:18.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:18.902 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:18.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:18.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:18 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:25:18.940 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:18 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:25:18.940 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:18 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:25:18.940 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:18 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:25:18.948 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:18 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:25:18.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:18 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:25:19.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:19.030 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:19.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.051 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:19.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:19.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:19.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:19.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:19.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:19.329 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:19.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.330 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:19.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 61 16 100159 100143 102.587 69.75 0.0186905 0.00974187 2024-09-17T08:25:19.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:19.463 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:19.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:19.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:19.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:19.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:19.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:19.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:19.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:19.853 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:19.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[40649]: pgmap v682: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:25:19.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:25:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:25:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:25:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:19 smithi079 ceph-mon[29565]: pgmap v682: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:25:19.955 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:19.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:19 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:25:19.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:19 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:25:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:19 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:25:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:19 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:25:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:19 smithi173 ceph-mon[33187]: pgmap v682: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-17T08:25:19.980 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:19.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:19.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:20.082 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:20.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:20.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:20.212 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:20.241 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:20.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:20.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 62 16 101225 101209 102.007 66.625 0.00966391 0.00979762 2024-09-17T08:25:20.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:20.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:20.491 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:20.521 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:20.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:20.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:20.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.656 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:20.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:20.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:20.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:20.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:20.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:25:20.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:20 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:25:20.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:20.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:20.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:20.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:20 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:25:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:20 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:25:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:20 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:25:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:20 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:25:21.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:21.077 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:21.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:21.215 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:21.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:21.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:21.358 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:21.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:21.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 63 15 102363 102348 101.518 71.1875 0.0149702 0.00984606 2024-09-17T08:25:21.460 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:21.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:21.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:21.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:21.619 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:21.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:21.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:21.755 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:21.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:21.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:21.885 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:21.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:21.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:21.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:22.015 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:22.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:22.117 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:22.146 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:22.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:22.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:22.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.284 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:22.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:22.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:22.388 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 64 16 103493 103477 101.034 70.5625 0.0105962 0.00989286 2024-09-17T08:25:22.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:22.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:22.480 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T08:25:22.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:22.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:22.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:22.656 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:22.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:22.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:22.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:25:22.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:25:22.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[40649]: pgmap v683: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:25:22.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1350415634' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[29565]: pgmap v683: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1350415634' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:22 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:22.839 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:22.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.840 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:22.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:22.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:22 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:25:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:22 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:25:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:22 smithi173 ceph-mon[33187]: pgmap v683: 148 pgs: 148 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:25:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1350415634' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:22 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:25:22.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:22.989 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:22.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:23.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:23.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:23.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:23.271 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:23.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:23.336 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:23.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:23.389 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 65 16 104649 104633 100.591 72.25 0.013663 0.00993615 2024-09-17T08:25:23.402 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:23.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.403 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:23.445 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:25:23.445 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:25:23.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:23.479 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:25:23.479 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:25:23.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:23.522 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:25:23.523 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:25:23.535 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:23.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:23.558 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:25:23.558 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:25:23.589 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:25:23.589 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:25:23.589 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:25:23.589 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:25:23.589 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1091225478' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:25:23.590 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:25:23.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T08:25:23.591 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:23 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1091225478' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:23.602 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:25:23.603 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:25:23.636 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:25:23.636 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:25:23.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:23.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:23.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:25:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:25:23.706 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:25:23.707 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:25:23.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:23.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:23.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:23.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:23.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:23.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:23.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T08:25:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:23 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1091225478' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:24.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:24.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:24.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:24.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:24.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:24.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11696 9.0 9:09772a79:::benchmark_data_smithi079_143589_object11695:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:25:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.494907+0000", 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.76497423199999, 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1012250789999998, 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11696 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494907+0000", 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494907+0000", 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494907+0000", 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494927+0000", 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0435999999999999e-05 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494935+0000", 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5909999999999998e-06 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.494940+0000", 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5550000000000002e-06 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.495097+0000", 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015707399999999999 2024-09-17T08:25:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.495109+0000", 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2062000000000001e-05 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485784+0000", 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.99067411299999997 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485798+0000", 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4011e-05 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592463+0000", 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.106665287 2024-09-17T08:25:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592577+0000", 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011404 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592685+0000", 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107941 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.595465+0000", 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027805999999999998 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.595541+0000", 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5859000000000007e-05 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.595584+0000", 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2444000000000002e-05 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.595638+0000", 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3931000000000003e-05 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596132+0000", 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049482400000000005 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11698 9.0 9:06751af7:::benchmark_data_smithi079_143589_object11697:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.496296+0000", 2024-09-17T08:25:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.76358587999999, 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.101318655, 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11698 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496296+0000", 2024-09-17T08:25:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496296+0000", 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496295+0000", 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496318+0000", 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3793000000000001e-05 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496329+0000", 2024-09-17T08:25:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0919e-05 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496350+0000", 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0690999999999999e-05 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496456+0000", 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105883 2024-09-17T08:25:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.496477+0000", 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1121e-05 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485822+0000", 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98934501500000005 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485831+0000", 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9420000000000001e-06 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593798+0000", 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1079673999999999 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593942+0000", 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014400700000000001 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594123+0000", 2024-09-17T08:25:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018013600000000001 2024-09-17T08:25:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596180+0000", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020575010000000002 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597136+0000", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095619699999999999 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597192+0000", 2024-09-17T08:25:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5770999999999998e-05 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597228+0000", 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5954999999999997e-05 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597614+0000", 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038642700000000001 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11709 9.0 9:054176e7:::benchmark_data_smithi079_143589_object11708:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.504521+0000", 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.755360634, 2024-09-17T08:25:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.093545717, 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11709 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504521+0000", 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504521+0000", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504457+0000", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999366 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504545+0000", 2024-09-17T08:25:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7591999999999998e-05 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504555+0000", 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0146000000000001e-05 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504562+0000", 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.353e-06 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504671+0000", 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109881 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.504682+0000", 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1048e-05 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485853+0000", 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98117045400000003 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485862+0000", 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8429999999999999e-06 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594700+0000", 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1088385060000001 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594851+0000", 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015060099999999999 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594987+0000", 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013584799999999999 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596225+0000", 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001238615 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597144+0000", 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091914999999999998 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597664+0000", 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051980699999999995 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597717+0000", 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2550999999999999e-05 2024-09-17T08:25:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598067+0000", 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034989 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11728 9.0 9:066b6040:::benchmark_data_smithi079_143589_object11727:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.513294+0000", 2024-09-17T08:25:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.74658760800003, 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0852627199999998, 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11728 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513294+0000", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513294+0000", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513287+0000", 2024-09-17T08:25:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999933 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513349+0000", 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1618000000000005e-05 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513405+0000", 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6507e-05 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513425+0000", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9876000000000001e-05 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513517+0000", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1976000000000002e-05 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.513532+0000", 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5116999999999999e-05 2024-09-17T08:25:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485887+0000", 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97235489399999997 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.485898+0000", 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0818999999999999e-05 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596258+0000", 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1103604170000001 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596387+0000", 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012838300000000001 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.596540+0000", 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153024 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598129+0000", 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015889680000000001 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598186+0000", 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7108000000000003e-05 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598227+0000", 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1477999999999998e-05 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598278+0000", 2024-09-17T08:25:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-17T08:25:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.598557+0000", 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027906300000000002 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1623 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch ping cookie 94434125129984 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e225)", 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952168+0000", 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.30771338800002, 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54602349699999997, 2024-09-17T08:25:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1623 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952168+0000", 2024-09-17T08:25:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952168+0000", 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952168+0000", 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952172+0000", 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-17T08:25:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952180+0000", 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7249999999999997e-06 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952186+0000", 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.173e-06 2024-09-17T08:25:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952373+0000", 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018653800000000001 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952384+0000", 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0737000000000001e-05 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.497957+0000", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54557372599999998 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.498104+0000", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146835 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.498192+0000", 2024-09-17T08:25:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7581000000000001e-05 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1630 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94434125129984 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952758+0000", 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.30712372099998, 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.547138499, 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1630 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952758+0000", 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952758+0000", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952757+0000", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952763+0000", 2024-09-17T08:25:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7179999999999999e-06 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952788+0000", 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4300000000000001e-05 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952797+0000", 2024-09-17T08:25:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4590000000000004e-06 2024-09-17T08:25:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952888+0000", 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0958000000000006e-05 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952899+0000", 2024-09-17T08:25:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0669000000000001e-05 2024-09-17T08:25:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.498506+0000", 2024-09-17T08:25:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54560718600000002 2024-09-17T08:25:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.498597+0000", 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1073e-05 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.499896+0000", 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001299316 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24418.0:613 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e226)", 2024-09-17T08:25:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.129344+0000", 2024-09-17T08:25:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.130537726, 2024-09-17T08:25:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36067623799999998, 2024-09-17T08:25:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24418", 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.173:0/3469022300", 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 613 2024-09-17T08:25:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129344+0000", 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129344+0000", 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129341+0000", 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999971 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129355+0000", 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.412e-05 2024-09-17T08:25:24.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129386+0000", 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0812999999999998e-05 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129400+0000", 2024-09-17T08:25:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.42e-05 2024-09-17T08:25:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129516+0000", 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115611 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.129526+0000", 2024-09-17T08:25:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9350000000000006e-06 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.489858+0000", 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36033199100000002 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.489959+0000", 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100987 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490020+0000", 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1317999999999998e-05 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11680 9.6 9:697de085:::benchmark_data_smithi079_143589_object11679:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484425+0000", 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77545636600001, 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1159076800000001, 2024-09-17T08:25:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11680 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484425+0000", 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484425+0000", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484424+0000", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484482+0000", 2024-09-17T08:25:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8424000000000002e-05 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484496+0000", 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3397000000000001e-05 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484504+0000", 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.49e-06 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487808+0000", 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033036699999999999 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487817+0000", 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3489999999999998e-06 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594624+0000", 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.106807273 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for degraded object", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.595596+0000", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097173099999999998 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.599217+0000", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036210140000000001 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.600333+0000", 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001115867 2024-09-17T08:25:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11685 9.6 9:64508ea9:::benchmark_data_smithi079_143589_object11684:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484514+0000", 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77536786000002, 2024-09-17T08:25:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.119124402, 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11685 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484514+0000", 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484514+0000", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484513+0000", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484564+0000", 2024-09-17T08:25:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0844000000000003e-05 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484573+0000", 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3610000000000007e-06 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484580+0000", 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.861e-06 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487842+0000", 2024-09-17T08:25:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032613970000000001 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487848+0000", 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2349999999999996e-06 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597122+0000", 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1092739030000001 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for degraded object", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.597890+0000", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076869500000000001 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603547+0000", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564850000000002 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603638+0000", 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1187999999999995e-05 2024-09-17T08:25:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11687 9.6 9:68f58c98:::benchmark_data_smithi079_143589_object11686:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484587+0000", 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77529416800002, 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.121878731, 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11687 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484587+0000", 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484587+0000", 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484587+0000", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484611+0000", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4311000000000002e-05 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484619+0000", 2024-09-17T08:25:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4499999999999998e-06 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484624+0000", 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3959999999999997e-06 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487870+0000", 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032459490000000001 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487876+0000", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0719999999999996e-06 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.599705+0000", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1118290390000001 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.599845+0000", 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013999800000000001 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.599984+0000", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138708 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603502+0000", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035184610000000001 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603722+0000", 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000219645 2024-09-17T08:25:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606070+0000", 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023482199999999998 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606125+0000", 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5441e-05 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606466+0000", 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000340694 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11722 9.6 9:6fb72c78:::benchmark_data_smithi079_143589_object11721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484629+0000", 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77525249899998, 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.122240313, 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11722 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484629+0000", 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484629+0000", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484629+0000", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484653+0000", 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4026999999999999e-05 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484660+0000", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6839999999999996e-06 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484666+0000", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5890000000000002e-06 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487897+0000", 2024-09-17T08:25:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032308609999999998 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487903+0000", 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2099999999999998e-06 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.600659+0000", 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.112756176 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.600800+0000", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014123799999999999 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.600959+0000", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015812199999999999 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603510+0000", 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025517729999999998 2024-09-17T08:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603772+0000", 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026130599999999999 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606512+0000", 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027400390000000001 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606564+0000", 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1997999999999999e-05 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606869+0000", 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030577399999999998 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11734 9.6 9:62db5e70:::benchmark_data_smithi079_143589_object11733:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484671+0000", 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77521060399999, 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1263577570000001, 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11734 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484671+0000", 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484671+0000", 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484671+0000", 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484976+0000", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000305648 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484985+0000", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.312e-06 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484992+0000", 2024-09-17T08:25:24.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487923+0000", 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029302640000000001 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487929+0000", 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0619999999999997e-06 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.601768+0000", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113839018 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.601892+0000", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124633 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.602037+0000", 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014500299999999999 2024-09-17T08:25:24.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603637+0000", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015997330000000001 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603816+0000", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017857300000000001 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.610437+0000", 2024-09-17T08:25:24.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066217649999999999 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.610490+0000", 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2549000000000001e-05 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.611029+0000", 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053876300000000005 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11755 9.6 9:6b31af22:::benchmark_data_smithi079_143589_object11754:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.484999+0000", 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77488250900001, 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.133133261, 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11755 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.368 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484999+0000", 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484999+0000", 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.484998+0000", 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486413+0000", 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014145869999999999 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486424+0000", 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0978999999999999e-05 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486432+0000", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1189999999999997e-06 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487951+0000", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00151851 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487956+0000", 2024-09-17T08:25:24.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.711e-06 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603848+0000", 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1158912409999999 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.603968+0000", 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120432 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.604121+0000", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015338 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606018+0000", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018965830000000001 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606027+0000", 2024-09-17T08:25:24.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8750000000000006e-06 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.616994+0000", 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010967347000000001 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.617057+0000", 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2692000000000002e-05 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.618132+0000", 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001075473 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11758 9.6 9:6c411962:::benchmark_data_smithi079_143589_object11757:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e228)", 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:23.486440+0000", 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.77344132899998, 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.132201891, 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11758 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486440+0000", 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486440+0000", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486439+0000", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486470+0000", 2024-09-17T08:25:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0584999999999999e-05 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486477+0000", 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3649999999999998e-06 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.486483+0000", 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5249999999999996e-06 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487975+0000", 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014922 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.487981+0000", 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0900000000000001e-06 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.604700+0000", 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1167191030000001 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.604831+0000", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013087099999999999 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,5", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.604978+0000", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000147262 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.606033+0000", 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001054438 2024-09-17T08:25:24.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.607090+0000", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001057517 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.618198+0000", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011107334999999999 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.618248+0000", 2024-09-17T08:25:24.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0228000000000001e-05 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.618642+0000", 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039419200000000002 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210605 9.0 9:018ae7c4:::benchmark_data_smithi079_143589_object79406:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.939477+0000", 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.32040470199999, 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37674120900000002, 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210605 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939477+0000", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939477+0000", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939476+0000", 2024-09-17T08:25:24.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939480+0000", 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3050000000000003e-06 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939490+0000", 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.679e-06 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939497+0000", 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.708e-06 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939540+0000", 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2953e-05 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939667+0000", 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012783400000000001 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939808+0000", 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014090500000000001 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.942173+0000", 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023650250000000002 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946516+0000", 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043427769999999999 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316066+0000", 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36954946599999999 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316125+0000", 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9518999999999997e-05 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316218+0000", 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3078999999999994e-05 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210628 9.0 9:0dc80e1c:::benchmark_data_smithi079_143589_object79429:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.955016+0000", 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.30486539699999, 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36132987100000002, 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210628 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955016+0000", 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955016+0000", 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955015+0000", 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955021+0000", 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5570000000000003e-06 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955034+0000", 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3281999999999999e-05 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955040+0000", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2260000000000002e-06 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955082+0000", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2123000000000003e-05 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955165+0000", 2024-09-17T08:25:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2164999999999997e-05 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.955229+0000", 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4172999999999997e-05 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.957969+0000", 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405429999999998 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.958730+0000", 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076042899999999997 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316275+0000", 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35754498400000001 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316315+0000", 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9845999999999998e-05 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316346+0000", 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1551999999999999e-05 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210632 9.0 9:06ca537b:::benchmark_data_smithi079_143589_object79433:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.960018+0000", 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.29986407300001, 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35642721599999999, 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210632 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960018+0000", 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960018+0000", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960016+0000", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960022+0000", 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9630000000000003e-06 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960035+0000", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2693e-05 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960042+0000", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0759999999999997e-06 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960075+0000", 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3510999999999998e-05 2024-09-17T08:25:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960201+0000", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012512499999999999 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.960338+0000", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137525 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.966204+0000", 2024-09-17T08:25:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058660099999999996 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.966212+0000", 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5469999999999999e-06 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316384+0000", 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35017204699999999 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316416+0000", 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2563e-05 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316445+0000", 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-05 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238282 9.a 9:5c132a73:::benchmark_data_smithi079_143589_object107083:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.043499+0000", 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.216383003, 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.479805753, 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238282 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043499+0000", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043499+0000", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043497+0000", 2024-09-17T08:25:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043504+0000", 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7789999999999998e-06 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043515+0000", 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.119e-05 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043522+0000", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.922e-06 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043569+0000", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6786999999999998e-05 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043692+0000", 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123025 2024-09-17T08:25:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043831+0000", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013865000000000001 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046113+0000", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022828359999999999 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046748+0000", 2024-09-17T08:25:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00063450500000000005 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523211+0000", 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47646293899999997 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523268+0000", 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6910999999999998e-05 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523304+0000", 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6591000000000003e-05 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238291 9.a 9:5e6ad04a:::benchmark_data_smithi079_143589_object107092:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.052229+0000", 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.20765219499999, 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.471230025, 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238291 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052229+0000", 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052229+0000", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052228+0000", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052234+0000", 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6080000000000003e-06 2024-09-17T08:25:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052246+0000", 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1643e-05 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052250+0000", 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7910000000000004e-06 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052297+0000", 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6811999999999998e-05 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052419+0000", 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121341 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.052509+0000", 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0445000000000006e-05 2024-09-17T08:25:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.054552+0000", 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020428400000000002 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.054828+0000", 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027666399999999998 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523353+0000", 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.46852407200000001 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523425+0000", 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2477000000000004e-05 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.523459+0000", 2024-09-17T08:25:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4452000000000002e-05 2024-09-17T08:25:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:105066 10.2 10:480ffbfb:::benchmark_data_smithi079_172150_object105065:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e311)", 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:23.788120+0000", 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.47176137699999998, 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029791950000000001, 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:25:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105066 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788120+0000", 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788120+0000", 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788120+0000", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788160+0000", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0142000000000003e-05 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788170+0000", 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0316000000000001e-05 2024-09-17T08:25:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.788175+0000", 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0359999999999997e-06 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.791810+0000", 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036349360000000001 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.791938+0000", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000127744 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,4", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.792082+0000", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014367399999999999 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.795909+0000", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003826978 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:25:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.802543+0000", 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066344450000000001 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.817463+0000", 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014919577 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.817599+0000", 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013662799999999999 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:23.817912+0000", 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031297699999999998 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:24.423 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:25:24.423 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 66 16 105780 105764 100.138 70.6875 0.0111034 0.00998172 2024-09-17T08:25:24.454 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:24.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:24.510 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:25:24.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:24.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:24.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:24.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: pgmap v685: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3975303731' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:24.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: pgmap v685: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3975303731' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:24.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:24 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:24.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:24.728 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:24.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:24.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:24.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:24.867 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:24.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:24.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:25:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:25:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:24.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:25:24.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:25:24.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: pgmap v685: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-17T08:25:24.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3975303731' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:24.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:24 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:25.002 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:25.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:25.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.142 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:25.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:25.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:25.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.279 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:25.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:25.389 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 67 15 107019 107004 99.8003 77.5 0.0115491 0.0100151 2024-09-17T08:25:25.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:25.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:25.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:25.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:25.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:25.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.688 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:25.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:25.824 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:25:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[40649]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:25:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:25:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:25:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:25.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:25 smithi079 ceph-mon[29565]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T08:25:25.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:25.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:25.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:25.966 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:25 smithi173 ceph-mon[33187]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T08:25:26.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:26.109 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:26.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:26.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:25:26.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:26.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:26.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:26.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:26.389 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 68 16 108187 108171 99.4052 72.9375 0.0243386 0.0100556 2024-09-17T08:25:26.393 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:26.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:26.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:26.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-17T08:25:26.581 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:26.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:26.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:26.735 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:26.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:26.814 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:25:26.814 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:25:26.814 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[29565]: pgmap v687: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:25:26.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:25:26.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:25:26.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:26 smithi079 ceph-mon[40649]: pgmap v687: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:25:26.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:26.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:26.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:26.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:26 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:25:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:26 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:25:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:26 smithi173 ceph-mon[33187]: pgmap v687: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-17T08:25:26.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:27.010 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:27.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:27.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:27.142 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:27.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:27.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:27.283 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:27.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:27.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:27.393 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 69 16 109287 109271 98.9594 68.75 0.016329 0.0101004 2024-09-17T08:25:27.427 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:27.529 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:27.558 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:27.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:27.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:27.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.704 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:27.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:27.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:27.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:27.852 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:27.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:27.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:27 smithi079 ceph-mon[29565]: 10.3 scrub starts 2024-09-17T08:25:27.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:27 smithi079 ceph-mon[40649]: 10.3 scrub starts 2024-09-17T08:25:27.873 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:27.885 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:27.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:27 smithi173 ceph-mon[33187]: 10.3 scrub starts 2024-09-17T08:25:28.015 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:28.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:28.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:28.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:28.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:28.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:28.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:28.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:28.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 70 16 110353 110337 98.4974 66.625 0.00451043 0.0101474 2024-09-17T08:25:28.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:28.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:28.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:28.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:28.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:28.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:28.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[29565]: 6.12 deep-scrub starts 2024-09-17T08:25:28.597 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[29565]: 6.12 deep-scrub ok 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[29565]: pgmap v688: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 839 op/s 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[40649]: 6.12 deep-scrub starts 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[40649]: 6.12 deep-scrub ok 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:25:28.598 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:28 smithi079 ceph-mon[40649]: pgmap v688: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 839 op/s 2024-09-17T08:25:28.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:28.712 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:28.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:28.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:28.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:28.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:28.950 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:28 smithi173 ceph-mon[33187]: 6.12 deep-scrub starts 2024-09-17T08:25:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:28 smithi173 ceph-mon[33187]: 6.12 deep-scrub ok 2024-09-17T08:25:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:28 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:25:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:28 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:25:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:28 smithi173 ceph-mon[33187]: pgmap v688: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 839 op/s 2024-09-17T08:25:28.984 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:28.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:28.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:29.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:29.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:29.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:29.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:25:29.227 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:29.256 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:29.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:29.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:29.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 71 16 111537 111521 98.1523 74 0.00987639 0.0101837 2024-09-17T08:25:29.404 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:29.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:29.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:29.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:29.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:29.646 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:29.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:29.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:29.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:25:29.751 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:25:29.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:25:29.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[40649]: 6.1a deep-scrub starts 2024-09-17T08:25:29.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[40649]: 6.1a deep-scrub ok 2024-09-17T08:25:29.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[40649]: 10.3 scrub ok 2024-09-17T08:25:29.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:25:29.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:25:29.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[29565]: 6.1a deep-scrub starts 2024-09-17T08:25:29.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[29565]: 6.1a deep-scrub ok 2024-09-17T08:25:29.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:29 smithi079 ceph-mon[29565]: 10.3 scrub ok 2024-09-17T08:25:29.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:29.809 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:29.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:29.911 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:29.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:29.943 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:29.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:29 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:25:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:29 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:25:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:29 smithi173 ceph-mon[33187]: 6.1a deep-scrub starts 2024-09-17T08:25:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:29 smithi173 ceph-mon[33187]: 6.1a deep-scrub ok 2024-09-17T08:25:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:29 smithi173 ceph-mon[33187]: 10.3 scrub ok 2024-09-17T08:25:30.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:30.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.086 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:30.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:30.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:30.221 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:30.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:30.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:30.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.363 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:30.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:30.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 72 15 112674 112659 97.7768 71.125 0.00676234 0.0102232 2024-09-17T08:25:30.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:30.530 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:30.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:30.554 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:25:30.554 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:25:30.554 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:25:30.554 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:25:30.554 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[29565]: pgmap v689: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:25:30.598 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:30.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:30.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.677 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:30.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:30.778 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:30.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:30.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:25:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:25:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:25:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:25:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:30 smithi079 ceph-mon[40649]: pgmap v689: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:25:30.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:30.946 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:30.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:30.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:30 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:25:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:30 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:25:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:30 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:25:30.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:30 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:25:30.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:30 smithi173 ceph-mon[33187]: pgmap v689: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-17T08:25:31.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:31.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:31.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:31.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:31.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:31.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:31.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:31.353 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:31.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:31.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 73 16 113759 113743 97.3655 67.75 0.00858775 0.0102653 2024-09-17T08:25:31.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:31.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:31.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:31.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:31.610 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:113734 10.f e313/287)", 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.373570+0000", 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0050459770000000001, 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050695979999999998, 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373570+0000", 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373570+0000", 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373569+0000", 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373572+0000", 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1240000000000001e-06 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373581+0000", 2024-09-17T08:25:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4090000000000003e-06 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.373585+0000", 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2520000000000001e-06 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:113748 10.f 10.9d4cce0f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e313)", 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.376705+0000", 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001910669, 2024-09-17T08:25:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001984567, 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113748 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376705+0000", 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376705+0000", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376703+0000", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376730+0000", 2024-09-17T08:25:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6550000000000002e-05 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376747+0000", 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7408e-05 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.376757+0000", 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2879999999999998e-06 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:113749 10.f 10.9f8033f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e313)", 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.377156+0000", 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001459615, 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015842230000000001, 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113749 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377156+0000", 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377156+0000", 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377155+0000", 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377179+0000", 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3904e-05 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377190+0000", 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1243000000000001e-05 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.377197+0000", 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0049999999999998e-06 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:113738 10.f e313/287)", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.378497+0000", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00011814700000000001, 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029691199999999998, 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378497+0000", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378497+0000", 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378497+0000", 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378504+0000", 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8110000000000003e-06 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378517+0000", 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2193e-05 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.378523+0000", 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5089999999999999e-06 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:113728 10.4 e313/287 10:2655a9f3:::benchmark_data_smithi079_172150_object113727:head v 313'7081, mlcod=313'7079)", 2024-09-17T08:25:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.365888+0000", 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.012727871999999999, 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012953157999999999, 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365888+0000", 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365888+0000", 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365887+0000", 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365911+0000", 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4417999999999999e-05 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365928+0000", 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7142999999999998e-05 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.365936+0000", 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9379999999999992e-06 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.366066+0000", 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000130338 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.366122+0000", 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5331999999999998e-05 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:113734 10.f 10:f84a6d2b:::benchmark_data_smithi079_172150_object113733:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e313)", 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.367161+0000", 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.011454568, 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011739431999999999, 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113734 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367161+0000", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367161+0000", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367160+0000", 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367184+0000", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4893e-05 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367197+0000", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2619000000000001e-05 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367204+0000", 2024-09-17T08:25:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2080000000000003e-06 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367447+0000", 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000242484 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367829+0000", 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000381744 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.368282+0000", 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045397299999999999 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:113738 10.f 10:f4d5152a:::benchmark_data_smithi079_172150_object113737:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e313)", 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.367261+0000", 2024-09-17T08:25:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.011354161, 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011706072, 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113738 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367261+0000", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367261+0000", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367261+0000", 2024-09-17T08:25:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367281+0000", 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030999999999998e-05 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367290+0000", 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6370000000000005e-06 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.367295+0000", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8420000000000001e-06 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.374265+0000", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069693389999999997 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.374549+0000", 2024-09-17T08:25:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000283923 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.375007+0000", 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045841099999999999 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:113740 10.4 e313/287 10:2ab22eaa:::benchmark_data_smithi079_172150_object113739:head v 313'7082, mlcod=313'7079)", 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.370038+0000", 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0085771449999999996, 2024-09-17T08:25:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089922289999999992, 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370038+0000", 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370038+0000", 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370037+0000", 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370060+0000", 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2204e-05 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370075+0000", 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5271999999999999e-05 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.370082+0000", 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0090000000000001e-06 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371371+0000", 2024-09-17T08:25:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012893169999999999 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371426+0000", 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4966000000000002e-05 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:113734 10.f e313/287)", 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:31.371221+0000", 2024-09-17T08:25:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0073945699999999996, 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078605689999999995, 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371221+0000", 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:25:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371221+0000", 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371220+0000", 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371228+0000", 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1610000000000003e-06 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371257+0000", 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.885e-05 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:31.371265+0000", 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9990000000000008e-06 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 9 2024-09-17T08:25:31.645 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:25:31.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:31.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:31.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:31.779 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:31.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.780 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T08:25:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[29565]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T08:25:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:31 smithi079 ceph-mon[40649]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T08:25:31.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:31.919 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:31.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:31.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T08:25:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:31 smithi173 ceph-mon[33187]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T08:25:32.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:32.062 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:32.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:32.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:32.203 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:32.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:32.304 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:32.331 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:32.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:32.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:25:32.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 74 16 114988 114972 97.0877 76.8125 0.0147887 0.0102956 2024-09-17T08:25:32.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:32.456 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:32.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:32.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:32.600 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:32.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:32.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:32.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:32.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.748 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:32.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:32.849 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:32.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:25:32.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:25:32.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[29565]: pgmap v692: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 726 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[40649]: pgmap v692: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 726 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:32.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:32.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:32.871 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:32.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:32.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:32 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:25:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:32 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:25:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:32 smithi173 ceph-mon[33187]: pgmap v692: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 726 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-17T08:25:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:33.013 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:33.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:33.154 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:33.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.296 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:33.390 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 75 16 115970 115954 96.6115 61.375 0.00960165 0.0103461 2024-09-17T08:25:33.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:33.437 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:33.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:33.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:33.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:33.578 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T08:25:33.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.709 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:33.709 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:33.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:33.843 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:33.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:33.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2590648136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[29565]: pgmap v693: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2590648136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:33.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:33 smithi079 ceph-mon[40649]: pgmap v693: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-17T08:25:33.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:25:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:25:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2590648136' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:25:33.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:33.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:33 smithi173 ceph-mon[33187]: pgmap v693: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-17T08:25:33.976 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:33.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:33.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:34.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:34.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.112 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:34.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:34.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:34.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.251 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:34.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:34.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:34.382 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:34.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:34.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 76 16 117179 117163 96.3344 75.5625 0.0128528 0.0103763 2024-09-17T08:25:34.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:34.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.506 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:34.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:34.607 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:34.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:34.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:34.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.782 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:34.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 4.16 deep-scrub starts 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 4.16 deep-scrub ok 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:25:34.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[29565]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 4.16 deep-scrub starts 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 4.16 deep-scrub ok 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:25:34.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:34 smithi079 ceph-mon[40649]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T08:25:34.883 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:34.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:34.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:34.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:34.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 4.16 deep-scrub starts 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 4.16 deep-scrub ok 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:25:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:34 smithi173 ceph-mon[33187]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T08:25:35.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:35.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.056 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:35.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:35.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:35.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:35.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:35.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:35.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:35.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:35.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 77 16 118141 118125 95.8642 60.125 0.00862121 0.0104267 2024-09-17T08:25:35.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:35.460 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:35.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:35.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:35.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.601 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:35.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:35.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:35.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:35.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:35.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:35.864 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:35.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:35.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: 10.c scrub starts 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: 5.1d deep-scrub starts 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: 5.1d deep-scrub ok 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:25:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[40649]: pgmap v695: 148 pgs: 148 active+clean; 7.0 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 55 MiB/s wr, 877 op/s 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: 10.c scrub starts 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: 5.1d deep-scrub starts 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: 5.1d deep-scrub ok 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:25:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:35 smithi079 ceph-mon[29565]: pgmap v695: 148 pgs: 148 active+clean; 7.0 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 55 MiB/s wr, 877 op/s 2024-09-17T08:25:35.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: 10.c scrub starts 2024-09-17T08:25:35.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: 5.1d deep-scrub starts 2024-09-17T08:25:35.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: 5.1d deep-scrub ok 2024-09-17T08:25:35.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:25:35.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:25:35.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:35 smithi173 ceph-mon[33187]: pgmap v695: 148 pgs: 148 active+clean; 7.0 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 55 MiB/s wr, 877 op/s 2024-09-17T08:25:36.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:36.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:36.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:36.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.136 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:36.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:36.237 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:36.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.260 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:36.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:36.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:36.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:36.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:36.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 78 16 119039 119023 95.3547 56.125 0.00326359 0.0104574 2024-09-17T08:25:36.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:36.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:36.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:36.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:36.652 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:36.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:36.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:36.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:36.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:36.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:25:36.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:25:36.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[29565]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:25:36.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:36 smithi079 ceph-mon[40649]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T08:25:36.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:36.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:36.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:36.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:36 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:25:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:36 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:25:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:36 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:25:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:36 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:25:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:36 smithi173 ceph-mon[33187]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T08:25:37.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:37.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:37.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:37.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:37.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:37.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:37.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:37.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.333 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:37.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:37.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 79 16 120005 119989 94.9119 60.375 0.0036632 0.0105303 2024-09-17T08:25:37.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:37.470 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:37.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:37.572 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:37.606 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:37.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:37.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:37.742 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:37.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:37.844 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:37 smithi079 ceph-mon[29565]: 10.c scrub ok 2024-09-17T08:25:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:37 smithi079 ceph-mon[29565]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 840 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-17T08:25:37.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:37 smithi079 ceph-mon[40649]: 10.c scrub ok 2024-09-17T08:25:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:37 smithi079 ceph-mon[40649]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 840 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-17T08:25:37.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:37.878 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:37.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:37 smithi173 ceph-mon[33187]: 10.c scrub ok 2024-09-17T08:25:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:37 smithi173 ceph-mon[33187]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 840 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-17T08:25:37.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:38.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:38.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:38.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.160 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:38.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:38.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:25:38.391775+0000 min lat: 0.00114729 max lat: 0.263535 avg lat: 0.0105682 2024-09-17T08:25:38.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:25:38.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 80 16 121107 121091 94.5864 68.875 0.0149855 0.0105682 2024-09-17T08:25:38.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:38.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:38.570 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:38.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:38.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:38.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:38.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:38.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:38.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.842 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:38 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:25:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:38 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:25:38.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:38 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:25:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:38 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:25:38.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:38.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:38.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:38.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:38 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:25:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:38 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:25:39.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:39.091 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:39.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:39.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:39.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:39.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:39.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:39.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:39.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:39.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 81 15 122056 122041 94.1516 59.375 0.0112911 0.0106169 2024-09-17T08:25:39.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:39.496 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:39.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:39.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:39.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:39.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:39.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:39.776 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:39.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:25:39.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:25:39.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:39 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:25:39.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:39.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:39.914 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:39.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:39 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:25:39.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:39 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:25:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:39 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:25:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:39 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:25:40.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:40.056 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:40.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:40.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:40.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:40.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:40.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:40.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:40.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:40.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 82 16 123252 123236 93.9142 74.6875 0.00762396 0.0106434 2024-09-17T08:25:40.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:40.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-17T08:25:40.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.475 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:40.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:40.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:40.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:40.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:40.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:40.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.764 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:40.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:40.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:40.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 5.4 deep-scrub starts 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 5.4 deep-scrub ok 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[29565]: pgmap v698: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 5.4 deep-scrub starts 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 5.4 deep-scrub ok 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:25:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:25:40.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:25:40.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:25:40.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:40 smithi079 ceph-mon[40649]: pgmap v698: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:25:40.891 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:40.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:40.930 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:40.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 5.4 deep-scrub starts 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 5.4 deep-scrub ok 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:25:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:40 smithi173 ceph-mon[33187]: pgmap v698: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:25:41.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:41.077 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:41.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:41.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:41.229 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:41.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:41.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:41.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:41.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:41.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 83 16 124375 124359 93.6283 70.1875 0.00839233 0.0106752 2024-09-17T08:25:41.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:41.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:41.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:41.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:41.646 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:41.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:41.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:41.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:41.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:41.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:41.919 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:41.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:41.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:42.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:42.035 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:42.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:42.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:42.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:42.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:42.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:42.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:42.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:42.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:42.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 84 16 125496 125480 93.3477 70.0625 0.00826559 0.0107082 2024-09-17T08:25:42.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:42.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.732 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:42.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:42.733 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:25:42.833 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:42.864 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:42.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:25:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:25:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3617849879' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:25:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:25:42.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3617849879' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:42 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3617849879' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:42 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:25:42.997 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:42.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:42.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:43.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:43.137 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:43.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:43.239 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:43.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:43.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:43.299 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:43.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:43.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 85 16 126701 126685 93.1354 75.3125 0.0128899 0.0107327 2024-09-17T08:25:43.413 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:43.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.415 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:43.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:43.515 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:43.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:43.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: pgmap v699: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 859 op/s 2024-09-17T08:25:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[40649]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: pgmap v699: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 859 op/s 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:25:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:25:43.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:43.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:25:43.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:43 smithi079 ceph-mon[29565]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T08:25:43.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:43.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:43.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:43.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:43.820 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:43.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:43.957 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:43.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:43.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: pgmap v699: 148 pgs: 148 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 859 op/s 2024-09-17T08:25:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:25:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:43 smithi173 ceph-mon[33187]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T08:25:44.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:44.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:44.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:44.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:44.231 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:44.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:44.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:44.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:44.392 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 86 16 127928 127912 92.9441 76.6875 0.0119564 0.0107551 2024-09-17T08:25:44.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:44.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:44.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:44.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[40649]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T08:25:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:44.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:44 smithi079 ceph-mon[29565]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T08:25:44.637 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:44.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:44.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:44.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.775 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:44.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:44.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:44.915 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:44.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:44.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:44 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:44 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:44 smithi173 ceph-mon[33187]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T08:25:45.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:45.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.061 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:45.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:45.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:45.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:45.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:45.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:45.325 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:45.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:45.382 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:25:45.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:25:45.391 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 87 15 129143 129128 92.7493 76 0.0090174 0.0107772 2024-09-17T08:25:45.414 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:25:45.414 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:25:45.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:45.445 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:25:45.445 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:25:45.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:45.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:45.478 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:25:45.478 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:25:45.512 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:25:45.512 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:25:45.543 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:25:45.543 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:25:45.566 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:45.575 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:25:45.575 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:25:45.599 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:45.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:45.608 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:25:45.608 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:25:45.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:45.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:25:45.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:45.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:45.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:45 smithi079 ceph-mon[29565]: pgmap v702: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 721 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:45.778 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:45 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3996507202' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:45.778 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:45 smithi079 ceph-mon[40649]: pgmap v702: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 721 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:45.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:45 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3996507202' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:45.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:45.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:45.904 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:45.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:45 smithi173 ceph-mon[33187]: pgmap v702: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 721 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-17T08:25:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:45 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3996507202' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:25:46.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.048 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:46.048 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:46.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:46.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.181 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:46.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:46.282 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:46.318 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:46.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:46.392 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 88 16 130317 130301 92.5281 73.3125 0.0154173 0.0108032 2024-09-17T08:25:46.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:46.454 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:46.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:46.556 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:46.588 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:46.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:46.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:46.728 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:46.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:46.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:46.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:46.893 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:46.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:46.994 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:47.033 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:47.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:47.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:47.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:47.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:47.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:47.365 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:47.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:47.394 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 89 16 131366 131350 92.223 65.5625 0.0332233 0.0108375 2024-09-17T08:25:47.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:47.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.519 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:47.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: pgmap v703: 148 pgs: 148 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:25:47.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: pgmap v703: 148 pgs: 148 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:25:47.621 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:47.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:47.650 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:47.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:47.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:47.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:47.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:47.904 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:47.944 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:47.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:47.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: pgmap v703: 148 pgs: 148 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:25:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:25:48.046 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:48.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.080 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:48.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:48.181 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:48.211 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:48.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:48.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:48.342 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:48.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:48.394 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total time run: 90.0112 2024-09-17T08:25:48.394 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total writes made: 132529 2024-09-17T08:25:48.394 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Write size: 65536 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object size: 65536 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Bandwidth (MB/sec): 92.0226 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Bandwidth: 31.7852 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max bandwidth (MB/sec): 206.812 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min bandwidth (MB/sec): 56.125 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average IOPS: 1472 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev IOPS: 508.563 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max IOPS: 3309 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min IOPS: 898 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average Latency(s): 0.0108628 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Latency(s): 0.00910649 2024-09-17T08:25:48.395 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max latency(s): 0.263535 2024-09-17T08:25:48.396 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min latency(s): 0.00114729 2024-09-17T08:25:48.401 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T08:25:48.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:48.479 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:48.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:48.517 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T08:25:48.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:48.613 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:48.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:48.629 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:25:48.629 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:25:48.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.748 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:48.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:48.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:48.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:25:48.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:48.890 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:48.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:48.991 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:49.025 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:49.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:25:49.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:49.152 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:49.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:49.305 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:49.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:49.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:49.442 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:49.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-17T08:25:49.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:49.591 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:49.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:49.692 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:49.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[40649]: pgmap v705: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:25:49.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:25:49.715 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[40649]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T08:25:49.715 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[29565]: pgmap v705: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:25:49.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:25:49.716 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:49 smithi079 ceph-mon[29565]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T08:25:49.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:49.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:49.850 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:49.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:49.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:49 smithi173 ceph-mon[33187]: pgmap v705: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-17T08:25:49.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/77393277' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:25:49.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:49 smithi173 ceph-mon[33187]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T08:25:49.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:49.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:49.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:50.090 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:50.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:50.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:50.224 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.254 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:50.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:50.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:50.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.402 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:50.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:50.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:50.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:50.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:50.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:50.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:50.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:50.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:50.813 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:50.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:50.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:50.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:50.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:50.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:51.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:51.092 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:51.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:51.195 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:51.233 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:51.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:51.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:51.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:51.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:51.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:51.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.512 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:51.612 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:51.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.641 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:51.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:51.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:51.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.769 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:51.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:51.821 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:51 smithi079 ceph-mon[40649]: pgmap v707: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.6 KiB/s rd, 77 MiB/s wr, 1.26k op/s 2024-09-17T08:25:51.822 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:51 smithi079 ceph-mon[29565]: pgmap v707: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.6 KiB/s rd, 77 MiB/s wr, 1.26k op/s 2024-09-17T08:25:51.855 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:51.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:51.885 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:25:51.885 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:25:51.928 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:51.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:51.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:51 smithi173 ceph-mon[33187]: pgmap v707: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.6 KiB/s rd, 77 MiB/s wr, 1.26k op/s 2024-09-17T08:25:52.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:52.063 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:52.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:52.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:52.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.207 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:52.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:52.308 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:52.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:52.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:52.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:52.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:52.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:52.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:52.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:52.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:52.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:52.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:52.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:52.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:25:52.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:52.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:52.909 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:52.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:53.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:53.058 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:53.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:53.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:53.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:53.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:53.286 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:25:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:25:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:25:53.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:53.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:53.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:53.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:53.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:25:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:25:53.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:53.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:53.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:53.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:53.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:53.705 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:25:53.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:53.788 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:53.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:53.856 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:53 smithi079 ceph-mon[40649]: pgmap v708: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 956 op/s 2024-09-17T08:25:53.857 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:53 smithi079 ceph-mon[29565]: pgmap v708: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 956 op/s 2024-09-17T08:25:53.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:53.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:53.929 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:53.929 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:53 smithi173 ceph-mon[33187]: pgmap v708: 148 pgs: 148 active+clean; 8.0 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 956 op/s 2024-09-17T08:25:54.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:54.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:54.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.063 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:54.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:54.196 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:54.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:54.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:54.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:54.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:54.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:54.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.471 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:54.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:54.548 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:25:54.572 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:54.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.604 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:54.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:54.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:54.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:54.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:54.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:54.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:54.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3893001562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3893001562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:54 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:54.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:54.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:25:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3893001562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:54 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:25:54.974 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:55.000 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:55.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:55.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:55.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:55.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:55.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:55.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:55.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:55.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:55.415 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:55.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:55.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:55.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:55.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:55.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:55.683 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:55.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:55.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:55.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.822 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:55.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:55.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[40649]: pgmap v709: 148 pgs: 148 active+clean; 7.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 52 MiB/s wr, 1.25k op/s 2024-09-17T08:25:55.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[40649]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[29565]: pgmap v709: 148 pgs: 148 active+clean; 7.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 52 MiB/s wr, 1.25k op/s 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:55 smithi079 ceph-mon[29565]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T08:25:55.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:55.956 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:55.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:55.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:55 smithi173 ceph-mon[33187]: pgmap v709: 148 pgs: 148 active+clean; 7.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 52 MiB/s wr, 1.25k op/s 2024-09-17T08:25:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:55 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:25:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:25:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:55 smithi173 ceph-mon[33187]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T08:25:56.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:56.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:56.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:56.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:56.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:56.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:56.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:56.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.349 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:56.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:56.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:56.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:56.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:56.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:25:56.619 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:56.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:25:56.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:56.757 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:56.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:56.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:56.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:56 smithi079 ceph-mon[40649]: pgmap v711: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 32 MiB/s wr, 1.16k op/s 2024-09-17T08:25:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:56 smithi079 ceph-mon[29565]: pgmap v711: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 32 MiB/s wr, 1.16k op/s 2024-09-17T08:25:56.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:56.894 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:56.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:56 smithi173 ceph-mon[33187]: pgmap v711: 148 pgs: 148 active+clean; 7.8 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 32 MiB/s wr, 1.16k op/s 2024-09-17T08:25:56.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:57.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.017 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:57.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:57.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:57.142 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:57.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:57.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:57.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.267 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:57.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:57.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:57.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.391 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:57.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:57.492 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:57.526 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:57.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:57.628 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:25:57.659 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:57.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:25:57.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:25:57.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.798 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:57.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:25:57.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:57.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:57.987 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:57.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:58.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:58.123 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:58.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:58.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:58.261 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:58.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:58.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:58.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:58.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:58.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:58.522 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:58.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:58.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:58.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.655 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:58.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:58.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:25:58.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:58.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:25:58.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:58.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:58.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:58.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:25:58 smithi173 ceph-mon[33187]: pgmap v712: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2024-09-17T08:25:59.007 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:59.041 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:25:59.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:25:58 smithi079 ceph-mon[40649]: pgmap v712: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2024-09-17T08:25:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:25:58 smithi079 ceph-mon[29565]: pgmap v712: 148 pgs: 148 active+clean; 7.6 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 1.14k op/s 2024-09-17T08:25:59.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:59.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.182 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:59.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:59.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:59.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:59.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:59.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:59.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:25:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.593 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:25:59.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:25:59.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:25:59.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:25:59.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:25:59.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.857 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.858 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:25:59.959 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:25:59.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:25:59.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:25:59.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:00.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:00.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.144 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:00.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:00.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:00.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:00.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:00.368 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:00.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:00.404 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:00.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.405 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:00.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:00.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:00.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:00.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:00.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:00.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:00.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:00.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.814 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:00.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:00.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:00.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:00.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:00.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:00.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:00 smithi079 ceph-mon[29565]: pgmap v713: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:00.973 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:00 smithi079 ceph-mon[40649]: pgmap v713: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:00 smithi173 ceph-mon[33187]: pgmap v713: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:01.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:01.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:01.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:01.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:01.221 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:01.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:01.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:01.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.360 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:01.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:01.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:01.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:01.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:01.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:01.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:01.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:01.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:01.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:01.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:01.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:01.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:01.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:01.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:01.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:26:01.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:26:01.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:26:01.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:26:01.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:26:01.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:26:01.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:26:01.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:26:01.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:26:01.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:26:01.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:26:01.950 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:26:01.981 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:26:01.981 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:26:02.010 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:26:02.011 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:26:02.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:02.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:02.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:02.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:02.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:02.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:26:02.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:02.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:02.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.491 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:02.592 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:02.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:02.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:02.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:02.770 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/262479055' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:02.770 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/262479055' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:02.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:02.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:02.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:02.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:02.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/262479055' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:02.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:03.028 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:03.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:03.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:03.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:03.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:03.265 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:03.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.305 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:03.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:03.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:03.438 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:03.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:03.476 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:03.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:03.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.571 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:03.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:03.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:03.700 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:03.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:03.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:03.825 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:03 smithi079 ceph-mon[40649]: pgmap v714: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:03.825 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:03.825 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:03 smithi079 ceph-mon[29565]: pgmap v714: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:03.825 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:03.837 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:03.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:03.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:03.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:03.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:03 smithi173 ceph-mon[33187]: pgmap v714: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-09-17T08:26:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:04.074 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:04.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.109 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:04.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:04.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:04.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:04.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:04.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:04.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:04.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:04.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:04.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:04.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:04.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:04.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:04.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:04.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:04.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:04.798 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:04.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:04.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:04.934 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:04.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:04.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:05.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:05.083 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:05.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:05.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:05.227 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:05.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:05.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:05.361 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11661 9.2 9:4a142cba:::benchmark_data_smithi079_143589_object11660:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e226)", 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.59868460799999, 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1206362190000001, 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11661 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467002+0000", 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467028+0000", 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6152e-05 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467039+0000", 2024-09-17T08:26:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0975000000000001e-05 2024-09-17T08:26:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.467046+0000", 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7519999999999996e-06 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469412+0000", 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00236514 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469537+0000", 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012545899999999999 2024-09-17T08:26:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.469795+0000", 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025807199999999998 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.473388+0000", 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035933499999999999 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491273+0000", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017884118000000001 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491283+0000", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477118+0000", 2024-09-17T08:26:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98583439100000003 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477182+0000", 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3971000000000001e-05 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.587537+0000", 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1103552290000001 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.587638+0000", 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101394 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11677 9.2 9:4d7c77eb:::benchmark_data_smithi079_143589_object11676:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e226)", 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.59054025, 2024-09-17T08:26:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.117803313, 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11677 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:26:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475147+0000", 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475145+0000", 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475171+0000", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5361e-05 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475181+0000", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311e-05 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.475188+0000", 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8210000000000004e-06 2024-09-17T08:26:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491728+0000", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01654015 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491736+0000", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2849999999999995e-06 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477216+0000", 2024-09-17T08:26:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98547963500000002 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477224+0000", 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6859999999999995e-06 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588074+0000", 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1108491460000001 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588183+0000", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109447 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.588290+0000", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107101 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590678+0000", 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023879299999999999 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591490+0000", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081148099999999996 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592352+0000", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086275299999999996 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592413+0000", 2024-09-17T08:26:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0192999999999998e-05 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592950+0000", 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053737399999999997 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11689 9.2 9:44ca1e6e:::benchmark_data_smithi079_143589_object11688:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:26:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.57373095000003, 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1013493219999999, 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11689 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:26:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491956+0000", 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491954+0000", 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491975+0000", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0701999999999999e-05 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.491996+0000", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.056e-05 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492007+0000", 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1165000000000001e-05 2024-09-17T08:26:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492171+0000", 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016403199999999999 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492179+0000", 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3280000000000006e-06 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477251+0000", 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.985072155 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477260+0000", 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7460000000000006e-06 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589001+0000", 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1117409949999999 2024-09-17T08:26:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589129+0000", 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012832800000000001 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589302+0000", 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000172177 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591544+0000", 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022423090000000001 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591633+0000", 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9308000000000006e-05 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.592991+0000", 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013578959999999999 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593025+0000", 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4350000000000001e-05 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593305+0000", 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027980099999999999 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11704 9.2 9:41d452d7:::benchmark_data_smithi079_143589_object11703:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.56434745299998, 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0927345320000001, 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11704 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501339+0000", 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501338+0000", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501363+0000", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5066999999999999e-05 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501375+0000", 2024-09-17T08:26:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1566000000000001e-05 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501382+0000", 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0129999999999996e-06 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501433+0000", 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1002999999999999e-05 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501442+0000", 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8230000000000006e-06 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477286+0000", 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97584350399999997 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477294+0000", 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6529999999999994e-06 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589843+0000", 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.112549104 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.589966+0000", 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012277800000000001 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590114+0000", 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014866099999999999 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591640+0000", 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015257809999999999 2024-09-17T08:26:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591683+0000", 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3460000000000001e-05 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593630+0000", 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001946779 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.593684+0000", 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3808999999999999e-05 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594074+0000", 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038991200000000002 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11749 9.2 9:4b73f331:::benchmark_data_smithi079_143589_object11748:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.540459564, 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0692497840000001, 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11749 2024-09-17T08:26:05.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525227+0000", 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525226+0000", 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525248+0000", 2024-09-17T08:26:05.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1557999999999999e-05 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525258+0000", 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5680000000000005e-06 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525264+0000", 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3810000000000004e-06 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525422+0000", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015760700000000001 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.525431+0000", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5009999999999993e-06 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477317+0000", 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95188626399999998 2024-09-17T08:26:05.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.477326+0000", 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4749999999999993e-06 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590708+0000", 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113382469 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590842+0000", 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013373000000000001 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.590997+0000", 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015488500000000001 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591711+0000", 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071417900000000001 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.591929+0000", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000218123 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594137+0000", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022075100000000002 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594199+0000", 2024-09-17T08:26:05.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1870000000000002e-05 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:24.594477+0000", 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027855799999999998 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1625 5.14 5:2b04a3e9:::notify.6:head [watch ping cookie 94434111870464 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e225)", 2024-09-17T08:26:05.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.11355462699998, 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53841006800000002, 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1625 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952132+0000", 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:05.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952135+0000", 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6540000000000001e-06 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952142+0000", 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9369999999999998e-06 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952147+0000", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1019999999999996e-06 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952242+0000", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4555999999999994e-05 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952254+0000", 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1989e-05 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490303+0000", 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53804894000000003 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490454+0000", 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015072999999999999 2024-09-17T08:26:05.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490542+0000", 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8702000000000006e-05 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1631 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94434111870464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:26:05.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 342.11284678099997, 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54383248699999998, 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1631 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:26:05.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952840+0000", 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952839+0000", 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952846+0000", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6100000000000002e-06 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952865+0000", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9641999999999999e-05 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952872+0000", 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2840000000000001e-06 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952988+0000", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115812 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.952998+0000", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0182e-05 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490843+0000", 2024-09-17T08:26:05.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53784467899999999 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.490913+0000", 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0266999999999997e-05 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.496673+0000", 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057591739999999997 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210614 9.2 9:448427a4:::benchmark_data_smithi079_143589_object79415:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.11991794400001, 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36495978699999998, 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210614 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945768+0000", 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945772+0000", 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7929999999999997e-06 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945790+0000", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7125999999999998e-05 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945796+0000", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1890000000000004e-06 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945837+0000", 2024-09-17T08:26:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1242000000000003e-05 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945946+0000", 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010887600000000001 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946056+0000", 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110115 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.948721+0000", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026654339999999999 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.949343+0000", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062103199999999999 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310604+0000", 2024-09-17T08:26:05.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36126145799999998 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310682+0000", 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8459000000000002e-05 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310729+0000", 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6282000000000002e-05 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238260 9.c 9:3ad2fbde:::benchmark_data_smithi079_143589_object107061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.10697752499999, 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55530729599999995, 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238260 2024-09-17T08:26:05.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958708+0000", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958715+0000", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.007e-06 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958728+0000", 2024-09-17T08:26:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2736e-05 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958736+0000", 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3329999999999997e-06 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958824+0000", 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8157000000000003e-05 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958940+0000", 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011576199999999999 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959047+0000", 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010683699999999999 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964643+0000", 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055964830000000002 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967521+0000", 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028780609999999999 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.513927+0000", 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54640532900000005 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.513970+0000", 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3497e-05 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514017+0000", 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6619000000000002e-05 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238262 9.c 9:3e3399a0:::benchmark_data_smithi079_143589_object107063:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.10573954700001, 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55418753300000001, 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238262 2024-09-17T08:26:05.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959946+0000", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959952+0000", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4500000000000003e-06 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959967+0000", 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5302999999999999e-05 2024-09-17T08:26:05.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959975+0000", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4059999999999999e-06 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961568+0000", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001593265 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961685+0000", 2024-09-17T08:26:05.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117594 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961815+0000", 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012927899999999999 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967563+0000", 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057487989999999997 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967572+0000", 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4810000000000006e-06 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514062+0000", 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54648985500000002 2024-09-17T08:26:05.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514101+0000", 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9530000000000003e-05 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514135+0000", 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3488999999999997e-05 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238269 9.c 9:300a75d2:::benchmark_data_smithi079_143589_object107070:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.10288112800001, 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.551483222, 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238269 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962805+0000", 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:05.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962808+0000", 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8140000000000001e-06 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962814+0000", 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0260000000000004e-06 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962820+0000", 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5450000000000003e-06 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964667+0000", 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001847392 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964772+0000", 2024-09-17T08:26:05.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010490899999999999 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964884+0000", 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112475 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967590+0000", 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027056939999999998 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970332+0000", 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027415999999999999 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514219+0000", 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.543886961 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514254+0000", 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5525000000000003e-05 2024-09-17T08:26:05.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514289+0000", 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4709000000000003e-05 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238270 9.c 9:3be7327c:::benchmark_data_smithi079_143589_object107071:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:26:05.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.10188836899999, 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55059389999999997, 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238270 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963803+0000", 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963817+0000", 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3968000000000001e-05 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963824+0000", 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8909999999999998e-06 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967612+0000", 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037879839999999999 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967728+0000", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116365 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967860+0000", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131507 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970361+0000", 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025006899999999999 2024-09-17T08:26:05.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970432+0000", 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1725999999999997e-05 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514325+0000", 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54389226899999998 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514359+0000", 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570999999999998e-05 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514392+0000", 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3257999999999999e-05 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238273 9.1 9:8dab1ad1:::benchmark_data_smithi079_143589_object107074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.03245138099999, 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48495675599999999, 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238273 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033234+0000", 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033240+0000", 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2550000000000003e-06 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033252+0000", 2024-09-17T08:26:05.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1790000000000001e-05 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033258+0000", 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8449999999999999e-06 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033293+0000", 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5088e-05 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033384+0000", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1261999999999995e-05 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033498+0000", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113414 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.038146+0000", 2024-09-17T08:26:05.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046477230000000003 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.042772+0000", 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046263770000000001 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518104+0000", 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.475331644 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518151+0000", 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7188999999999998e-05 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518192+0000", 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1462999999999999e-05 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238276 9.1 9:8a6a764e:::benchmark_data_smithi079_143589_object107077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.031789444, 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48449216099999998, 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238276 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.459 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033901+0000", 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9870000000000001e-06 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033911+0000", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0243e-05 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033918+0000", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9349999999999997e-06 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035704+0000", 2024-09-17T08:26:05.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001785998 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035838+0000", 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134497 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035960+0000", 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122063 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.038305+0000", 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023450060000000002 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.042809+0000", 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045036399999999997 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518313+0000", 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.475503499 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518356+0000", 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2954000000000002e-05 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518390+0000", 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4028e-05 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238286 9.1 9:88a162d9:::benchmark_data_smithi079_143589_object107087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.02182048, 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47487866200000001, 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238286 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043865+0000", 2024-09-17T08:26:05.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043870+0000", 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7450000000000003e-06 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043881+0000", 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1239e-05 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043887+0000", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9859999999999999e-06 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043915+0000", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8085e-05 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043989+0000", 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3700000000000002e-05 2024-09-17T08:26:05.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.044053+0000", 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4226999999999993e-05 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046200+0000", 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002146448 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046890+0000", 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069050399999999999 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518492+0000", 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47160132500000002 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518708+0000", 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000216216 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518745+0000", 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7187000000000002e-05 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:119682 10.f 10:fc8c09c6:::benchmark_data_smithi079_172150_object119681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:37.027807+0000", 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.037880294000001, 2024-09-17T08:26:05.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107261417, 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119682 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027807+0000", 2024-09-17T08:26:05.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027807+0000", 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027805+0000", 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027827+0000", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2818000000000001e-05 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027841+0000", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3132999999999999e-05 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027848+0000", 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3440000000000003e-06 2024-09-17T08:26:05.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027891+0000", 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2765000000000001e-05 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.027978+0000", 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7315e-05 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.028060+0000", 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2010999999999998e-05 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.030311+0000", 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022512600000000002 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.030796+0000", 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048469099999999998 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.134989+0000", 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.104193548 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.135035+0000", 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5834000000000002e-05 2024-09-17T08:26:05.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.135068+0000", 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2685000000000003e-05 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:120318 10.7 10:ee4a6540:::benchmark_data_smithi079_172150_object120317:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:37.635568+0000", 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.430118816, 2024-09-17T08:26:05.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.071600394999999997, 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 120318 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635568+0000", 2024-09-17T08:26:05.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635568+0000", 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635568+0000", 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635589+0000", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1662999999999999e-05 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635595+0000", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2160000000000003e-06 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.635600+0000", 2024-09-17T08:26:05.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639126+0000", 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035257610000000001 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639244+0000", 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117825 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.695739+0000", 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.05649506 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.701665+0000", 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059260040000000003 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.702447+0000", 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078239100000000001 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707089+0000", 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046421639999999998 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707133+0000", 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3948e-05 2024-09-17T08:26:05.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707168+0000", 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5280000000000001e-05 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:120322 10.7 10:e9d5c671:::benchmark_data_smithi079_172150_object120321:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:37.639362+0000", 2024-09-17T08:26:05.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.426325040999998, 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.068022238999999998, 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 120322 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639362+0000", 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639362+0000", 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639361+0000", 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.639379+0000", 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7776999999999999e-05 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.696698+0000", 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057319428999999998 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.696710+0000", 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2048999999999999e-05 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.698536+0000", 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018258739999999999 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.698685+0000", 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000148629 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.698850+0000", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016493500000000001 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.702451+0000", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036015230000000001 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.702721+0000", 2024-09-17T08:26:05.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026929999999999999 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707258+0000", 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045376349999999999 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707351+0000", 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2884e-05 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:37.707384+0000", 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2982999999999999e-05 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:121824 10.3 e316/287 10:c7ef409d:::benchmark_data_smithi079_172150_object121823:head v 316'7682, mlcod=316'7679)", 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.012097+0000", 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.053589576, 2024-09-17T08:26:05.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18886142, 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.012097+0000", 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.012097+0000", 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.012097+0000", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.012119+0000", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2121e-05 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.183077+0000", 2024-09-17T08:26:05.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17095793300000001 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.183092+0000", 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959e-05 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.183128+0000", 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6128999999999999e-05 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.183145+0000", 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6773e-05 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.200735+0000", 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017590551999999999 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.200959+0000", 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022359300000000001 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:152293 10.f 10:f65539e8:::benchmark_data_smithi079_172150_object19761:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:04.906932+0000", 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.15875530299999999, 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072119107000000002, 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 152293 2024-09-17T08:26:05.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906932+0000", 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906932+0000", 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906930+0000", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906935+0000", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906944+0000", 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7690000000000002e-06 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.906950+0000", 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3249999999999996e-06 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.907169+0000", 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021844 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:05.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.907274+0000", 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105357 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.907399+0000", 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012526099999999999 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.910405+0000", 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030059990000000001 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.911985+0000", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001579865 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.978947+0000", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066962378000000003 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.979005+0000", 2024-09-17T08:26:05.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7444000000000001e-05 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:04.979051+0000", 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5720000000000003e-05 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:05.504 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:26:05.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.506 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:05.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:05.607 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[40649]: pgmap v715: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:05.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[40649]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T08:26:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[29565]: pgmap v715: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:05.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:05 smithi079 ceph-mon[29565]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T08:26:05.643 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:05.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:05.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:05.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.784 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:05.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:05.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:05.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:05.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:05.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:05 smithi173 ceph-mon[33187]: pgmap v715: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:05 smithi173 ceph-mon[33187]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T08:26:06.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:06.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:06.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:06.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:06.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:26:06.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:06.348 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:06.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:06.360 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:06.368 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:26:06.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:06.492 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.493 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:06.514 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T08:26:06.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:06.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:06.677 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2651030474' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:06 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:06.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.781 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2651030474' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:06 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2651030474' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:06 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:26:06.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:06.910 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:06.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:06.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:07.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:07.053 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:07.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:07.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:07.189 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:07.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:07.291 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:07.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:07.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:07.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:07.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.461 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:07.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:07.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:07.604 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:07.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:07.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:07.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:07.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[40649]: pgmap v717: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:26:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:26:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[40649]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T08:26:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[29565]: pgmap v717: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:26:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:26:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:07 smithi079 ceph-mon[29565]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T08:26:07.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:07.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:07.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:07 smithi173 ceph-mon[33187]: pgmap v717: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-17T08:26:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:07 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:26:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:07 smithi173 ceph-mon[33187]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T08:26:07.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:08.008 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:08.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:08.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:08.205 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:08.240 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:08.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:08.342 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:08.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:08.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:08.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:08.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:08.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:08.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:08.640 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:08.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:08.712 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T08:26:08.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:26:08.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:08.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:08.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:08.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:08.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:08.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:08.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:09.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:09.066 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:09.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:09.207 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:09.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:09.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.347 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:09.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:09.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:09.487 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.488 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:09.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:09.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:09.666 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:09 smithi173 ceph-mon[33187]: pgmap v719: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:26:09.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:158175 10.6 e324/287 10:6f3d6c13:::benchmark_data_smithi079_172150_object25643:head v 324'9790, mlcod=324'9788)", 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:09.819244+0000", 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.001400009, 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001423173, 2024-09-17T08:26:09.822 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819244+0000", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819244+0000", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:09.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819243+0000", 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819248+0000", 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7990000000000001e-06 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819254+0000", 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1129999999999997e-06 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819258+0000", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1659999999999996e-06 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819296+0000", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7496999999999997e-05 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:09.819316+0000", 2024-09-17T08:26:09.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0160999999999999e-05 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2024-09-17T08:26:09.826 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:09.851 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:09 smithi079 ceph-mon[40649]: pgmap v719: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:26:09.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:09 smithi079 ceph-mon[29565]: pgmap v719: 148 pgs: 148 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:26:09.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:09.902 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:09.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:09.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:10.004 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:10.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:10.143 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:10.175 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:10.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:10.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:10.409 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:10.434 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:10.537 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:10.570 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:10.605 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:26:10.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:10.700 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:10.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:10.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:10.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:10.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: pgmap v720: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2694759338' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: pgmap v720: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2694759338' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:10 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: pgmap v720: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2694759338' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.956 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:10 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:26:10.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:10.974 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:10.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:10.990 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:10.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:26:11.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:11.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.117 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:11.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:11.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:11.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:11.315 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:11.354 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:11.390 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:11.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:11.492 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:11.526 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:11.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:11.628 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:11.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:11.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:11 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:11 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:26:11.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:11 smithi173 ceph-mon[33187]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T08:26:11.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:11.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.804 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:11.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:11.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:11.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:26:11.832 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[29565]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T08:26:11.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:11.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:26:11.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:11 smithi079 ceph-mon[40649]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T08:26:11.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:11.939 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:11.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:11.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:12.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:12.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:12.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:12.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:12.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:12.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:12.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:12.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.367 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:12.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:12.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:12.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:12.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:12.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:12.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.680 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:12.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:12.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:12.814 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:12.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: pgmap v723: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:26:12.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: pgmap v723: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:26:12.875 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T08:26:12.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:26:12.876 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:12 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:26:12.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:12.948 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:12.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:12.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:12.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:26:12.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: pgmap v723: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:26:12.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:12 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:26:13.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:13.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.083 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:13.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:13.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:13.204 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:13.218 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:13.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:13.255 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:13.256 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:26:13.256 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:26:13.256 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:26:13.256 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:13.320 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:13.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:13.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:13.459 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:13.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:13.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:13.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:26:13.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:13.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:13.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:13.660 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:26:13.660 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:26:13.660 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:26:13.660 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:13.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:26:13.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:26:13.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:26:13.661 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:13.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:13.772 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:13.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:13.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:13.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:13.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:13.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:13.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:13 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:26:13.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:13 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:26:13.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:13 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:26:13.944 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:13.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:26:14.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:14.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.065 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:14.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:14.094 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:26:14.094 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:26:14.128 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:26:14.128 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:26:14.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:14.174 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:26:14.174 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:26:14.212 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:14.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:14.216 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:26:14.216 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:26:14.264 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:14.265 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:26:14.265 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:26:14.295 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:26:14.295 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:26:14.314 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:14.324 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:26:14.325 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:26:14.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.350 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:14.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:14.358 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:26:14.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:26:14.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:14.476 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:14.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:14.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:14.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.619 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:14.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:14.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:26:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: pgmap v725: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:26:14.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:26:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:26:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:26:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:26:14.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:14 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1370918001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:14.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:14.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:14.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:26:14.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: pgmap v725: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:26:14.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:26:14.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:26:14.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:26:14.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:26:14.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1370918001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:14.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:26:14.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: pgmap v725: 148 pgs: 148 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:26:14.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:26:14.806 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:26:14.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:26:14.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:26:14.807 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:14 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1370918001' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:26:14.834 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:26:14.888 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:14.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:14.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:14.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:15.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:15.055 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:15.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:15.184 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:15.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:15.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.325 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:15.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:15.465 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:15.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:15.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:15.594 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:15.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:15.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:15.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:15.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.859 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:15.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:15.946 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:15 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:26:15.946 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:15 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:26:15.946 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:15 smithi173 ceph-mon[33187]: 10.5 scrub starts 2024-09-17T08:26:15.946 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:15 smithi173 ceph-mon[33187]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:15.959 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:15.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:15.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:15.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:16.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:16.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:16.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:26:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:26:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[40649]: 10.5 scrub starts 2024-09-17T08:26:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[40649]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:16.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:26:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:26:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[29565]: 10.5 scrub starts 2024-09-17T08:26:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:15 smithi079 ceph-mon[29565]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:16.219 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:16.252 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:16.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:16.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:16.386 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:16.397 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:16.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:16.499 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:16.521 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:16.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.522 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:16.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:16.652 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:16.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:16.754 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:16.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:16.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:16.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:16.911 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:16.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:16.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:16 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:26:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:16 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:26:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:16 smithi173 ceph-mon[33187]: pgmap v726: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:16 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:26:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:16 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:26:17.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:17.045 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:17.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:17.073 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:26:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:26:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:26:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[40649]: pgmap v726: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:26:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:26:17.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:26:17.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:26:17.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[29565]: pgmap v726: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:17.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:26:17.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:16 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:26:17.148 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:17.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:26:17.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:17.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:17.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:17.369 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:17.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:17.400 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:17.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:17.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:17.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:17.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:17.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:17.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:17.744 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:17.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:17.878 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: 10.9 scrub starts 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:26:17.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:17 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:17.908 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:17.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:17.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:18.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:18.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:18.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: 10.9 scrub starts 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: 10.9 scrub starts 2024-09-17T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:26:18.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:17 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:18.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:18.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:18.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:18.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:18.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:18.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:18.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:18.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.463 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:18.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:18.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:18.594 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:18.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:18.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:18.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:18.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:18.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:18.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:18.866 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:18.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:18.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: 10.5 scrub ok 2024-09-17T08:26:18.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:26:18.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:26:18.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: pgmap v727: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 641 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:18.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:26:18.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:18 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:26:18.967 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:19.005 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:19.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:19.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:19.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: 10.5 scrub ok 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: pgmap v727: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 641 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: 10.5 scrub ok 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:26:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: pgmap v727: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 641 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:19.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:26:19.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:18 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:26:19.143 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:19.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:19.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:19.282 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:19.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:19.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:19.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:19.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:19.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:19.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:19.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:19.629 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:19.629 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:19.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:19.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:19.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:19.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:19.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:19.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:19.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:19.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:19.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:19 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:26:19.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:19 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:26:19.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:19 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:26:19.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:19 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:26:20.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:20.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:26:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:26:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:26:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:26:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:26:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:26:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:26:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:19 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:26:20.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:20.216 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:20.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:20.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:20.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.354 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:20.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:20.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:20.486 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:20.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:20.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:20.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:20.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:20.630 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:20.662 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":725,"stamp":"2024-09-17T08:26:13.570140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6891837742,"num_objects":105349,"num_object_clones":0,"num_object_copies":316047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105349,"num_whiteouts":0,"num_read":1980,"num_read_kb":1826,"num_write":160802,"num_write_kb":8484316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":1775729,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88277,"ondisk_log_size":88277,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":24282052,"kb_used_data":20165936,"kb_used_omap":15,"kb_used_meta":4115952,"kb_avail":725679164,"statfs":{"total":767960285184,"available":743095463936,"internally_reserved":0,"allocated":20649918464,"data_stored":20647293150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":4214735637},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,8,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-503840768,"num_objects":-7688,"num_object_clones":0,"num_object_copies":-23064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7688,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001840"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":334,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520857+0000","last_change":"2024-09-17T08:24:22.829264+0000","last_active":"2024-09-17T08:26:04.520857+0000","last_peered":"2024-09-17T08:26:04.520857+0000","last_clean":"2024-09-17T08:26:04.520857+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:26:04.520857+0000","last_undegraded":"2024-09-17T08:26:04.520857+0000","last_fullsized":"2024-09-17T08:26:04.520857+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:22.829138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:24:22.829138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002109297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":355,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608136+0000","last_change":"2024-09-17T08:23:33.814550+0000","last_active":"2024-09-17T08:26:10.608136+0000","last_peered":"2024-09-17T08:26:10.608136+0000","last_clean":"2024-09-17T08:26:10.608136+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:26:10.608136+0000","last_undegraded":"2024-09-17T08:26:10.608136+0000","last_fullsized":"2024-09-17T08:26:10.608136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:33.814411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:23:33.814411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040554900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":400,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520519+0000","last_change":"2024-09-17T08:23:28.822917+0000","last_active":"2024-09-17T08:26:04.520519+0000","last_peered":"2024-09-17T08:26:04.520519+0000","last_clean":"2024-09-17T08:26:04.520519+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:26:04.520519+0000","last_undegraded":"2024-09-17T08:26:04.520519+0000","last_fullsized":"2024-09-17T08:26:04.520519+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:23:28.822775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.822775+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081422939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":357,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607200+0000","last_change":"2024-09-17T08:23:33.566799+0000","last_active":"2024-09-17T08:26:10.607200+0000","last_peered":"2024-09-17T08:26:10.607200+0000","last_clean":"2024-09-17T08:26:10.607200+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:26:10.607200+0000","last_undegraded":"2024-09-17T08:26:10.607200+0000","last_fullsized":"2024-09-17T08:26:10.607200+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:33.566719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:33.566719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016610028999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":342,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520160+0000","last_change":"2024-09-17T08:25:12.813837+0000","last_active":"2024-09-17T08:26:04.520160+0000","last_peered":"2024-09-17T08:26:04.520160+0000","last_clean":"2024-09-17T08:26:04.520160+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:26:04.520160+0000","last_undegraded":"2024-09-17T08:26:04.520160+0000","last_fullsized":"2024-09-17T08:26:04.520160+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.813704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.813704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.648729+0000","scrub_duration":0.0071490010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":470,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607065+0000","last_change":"2024-09-17T08:24:29.572754+0000","last_active":"2024-09-17T08:26:10.607065+0000","last_peered":"2024-09-17T08:26:10.607065+0000","last_clean":"2024-09-17T08:26:10.607065+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:26:10.607065+0000","last_undegraded":"2024-09-17T08:26:10.607065+0000","last_fullsized":"2024-09-17T08:26:10.607065+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:29.572400+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.572400+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041638179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.506848+0000","last_change":"2024-09-17T08:25:29.240062+0000","last_active":"2024-09-17T08:26:04.506848+0000","last_peered":"2024-09-17T08:26:04.506848+0000","last_clean":"2024-09-17T08:26:04.506848+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:26:04.506848+0000","last_undegraded":"2024-09-17T08:26:04.506848+0000","last_fullsized":"2024-09-17T08:26:04.506848+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.239407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.239407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.084448+0000","scrub_duration":0.0062858009999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":366,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607127+0000","last_change":"2024-09-17T08:25:28.285107+0000","last_active":"2024-09-17T08:26:10.607127+0000","last_peered":"2024-09-17T08:26:10.607127+0000","last_clean":"2024-09-17T08:26:10.607127+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:26:10.607127+0000","last_undegraded":"2024-09-17T08:26:10.607127+0000","last_fullsized":"2024-09-17T08:26:10.607127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.284981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:28.287282+0000","scrub_duration":0.0042235850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":353,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607020+0000","last_change":"2024-09-17T08:25:14.860865+0000","last_active":"2024-09-17T08:26:10.607020+0000","last_peered":"2024-09-17T08:26:10.607020+0000","last_clean":"2024-09-17T08:26:10.607020+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:26:10.607020+0000","last_undegraded":"2024-09-17T08:26:10.607020+0000","last_fullsized":"2024-09-17T08:26:10.607020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.860746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.860746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.290009+0000","scrub_duration":0.0074367399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":386,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607052+0000","last_change":"2024-09-17T08:24:16.615788+0000","last_active":"2024-09-17T08:26:10.607052+0000","last_peered":"2024-09-17T08:26:10.607052+0000","last_clean":"2024-09-17T08:26:10.607052+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:26:10.607052+0000","last_undegraded":"2024-09-17T08:26:10.607052+0000","last_fullsized":"2024-09-17T08:26:10.607052+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:24:16.615683+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:24:16.615683+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001772476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":378,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608242+0000","last_change":"2024-09-17T08:24:32.685119+0000","last_active":"2024-09-17T08:26:10.608242+0000","last_peered":"2024-09-17T08:26:10.608242+0000","last_clean":"2024-09-17T08:26:10.608242+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:26:10.608242+0000","last_undegraded":"2024-09-17T08:26:10.608242+0000","last_fullsized":"2024-09-17T08:26:10.608242+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:32.684474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.684474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038163649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.506714+0000","last_change":"2024-09-17T08:24:25.823602+0000","last_active":"2024-09-17T08:26:04.506714+0000","last_peered":"2024-09-17T08:26:04.506714+0000","last_clean":"2024-09-17T08:26:04.506714+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:26:04.506714+0000","last_undegraded":"2024-09-17T08:26:04.506714+0000","last_fullsized":"2024-09-17T08:26:04.506714+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:25.823472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.823472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":379,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.668330+0000","last_change":"2024-09-17T08:26:11.668739+0000","last_active":"2024-09-17T08:26:11.668330+0000","last_peered":"2024-09-17T08:26:11.668330+0000","last_clean":"2024-09-17T08:26:11.668330+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:26:11.668330+0000","last_undegraded":"2024-09-17T08:26:11.668330+0000","last_fullsized":"2024-09-17T08:26:11.668330+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.668207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.668207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.803189+0000","scrub_duration":0.0090449410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":381,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607452+0000","last_change":"2024-09-17T08:25:22.208990+0000","last_active":"2024-09-17T08:26:10.607452+0000","last_peered":"2024-09-17T08:26:10.607452+0000","last_clean":"2024-09-17T08:26:10.607452+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:26:10.607452+0000","last_undegraded":"2024-09-17T08:26:10.607452+0000","last_fullsized":"2024-09-17T08:26:10.607452+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:25:22.208698+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.208698+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:22.812079+0000","scrub_duration":0.0038528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":253,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607556+0000","last_change":"2024-09-17T08:24:27.797180+0000","last_active":"2024-09-17T08:26:10.607556+0000","last_peered":"2024-09-17T08:26:10.607556+0000","last_clean":"2024-09-17T08:26:10.607556+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:26:10.607556+0000","last_undegraded":"2024-09-17T08:26:10.607556+0000","last_fullsized":"2024-09-17T08:26:10.607556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:27.796880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:27.796880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049344319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":366,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607155+0000","last_change":"2024-09-17T08:24:20.752436+0000","last_active":"2024-09-17T08:26:10.607155+0000","last_peered":"2024-09-17T08:26:10.607155+0000","last_clean":"2024-09-17T08:26:10.607155+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:26:10.607155+0000","last_undegraded":"2024-09-17T08:26:10.607155+0000","last_fullsized":"2024-09-17T08:26:10.607155+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:20.752313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:20.752313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027283759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"326'10042","reported_seq":10242,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.637111+0000","last_change":"2024-09-17T08:25:28.636224+0000","last_active":"2024-09-17T08:26:11.637111+0000","last_peered":"2024-09-17T08:26:11.637111+0000","last_clean":"2024-09-17T08:26:11.637111+0000","last_became_active":"2024-09-17T08:24:04.453791+0000","last_became_peered":"2024-09-17T08:24:04.453791+0000","last_unstale":"2024-09-17T08:26:11.637111+0000","last_undegraded":"2024-09-17T08:26:11.637111+0000","last_fullsized":"2024-09-17T08:26:11.637111+0000","mapping_epoch":287,"log_start":"298'3600","ondisk_log_start":"298'3600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6965","last_scrub_stamp":"2024-09-17T08:25:28.635952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.635952+0000","objects_scrubbed":6875,"log_size":6442,"log_dups_size":0,"ondisk_log_size":6442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:52.900306+0000","scrub_duration":2.5130719930000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435945472,"num_objects":6652,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":534208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":477,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608196+0000","last_change":"2024-09-17T08:24:17.608622+0000","last_active":"2024-09-17T08:26:10.608196+0000","last_peered":"2024-09-17T08:26:10.608196+0000","last_clean":"2024-09-17T08:26:10.608196+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:26:10.608196+0000","last_undegraded":"2024-09-17T08:26:10.608196+0000","last_fullsized":"2024-09-17T08:26:10.608196+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:24:17.608509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.608509+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025730330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":365,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606691+0000","last_change":"2024-09-17T08:25:21.870284+0000","last_active":"2024-09-17T08:26:10.606691+0000","last_peered":"2024-09-17T08:26:10.606691+0000","last_clean":"2024-09-17T08:26:10.606691+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:26:10.606691+0000","last_undegraded":"2024-09-17T08:26:10.606691+0000","last_fullsized":"2024-09-17T08:26:10.606691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:21.870174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:21.870174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.767809+0000","scrub_duration":0.0028963090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.509662+0000","last_change":"2024-09-17T08:23:24.040027+0000","last_active":"2024-09-17T08:26:04.509662+0000","last_peered":"2024-09-17T08:26:04.509662+0000","last_clean":"2024-09-17T08:26:04.509662+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:26:04.509662+0000","last_undegraded":"2024-09-17T08:26:04.509662+0000","last_fullsized":"2024-09-17T08:26:04.509662+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:24.039776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:24.039776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052197119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":333,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599994+0000","last_change":"2024-09-17T08:25:22.077723+0000","last_active":"2024-09-17T08:26:10.599994+0000","last_peered":"2024-09-17T08:26:10.599994+0000","last_clean":"2024-09-17T08:26:10.599994+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:26:10.599994+0000","last_undegraded":"2024-09-17T08:26:10.599994+0000","last_fullsized":"2024-09-17T08:26:10.599994+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:22.077429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.077429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.599110+0000","scrub_duration":0.0073898080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"323'9557","reported_seq":9603,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:08.814090+0000","last_change":"2024-09-17T08:24:04.455688+0000","last_active":"2024-09-17T08:26:08.814090+0000","last_peered":"2024-09-17T08:26:08.814090+0000","last_clean":"2024-09-17T08:26:08.814090+0000","last_became_active":"2024-09-17T08:24:04.455271+0000","last_became_peered":"2024-09-17T08:24:04.455271+0000","last_unstale":"2024-09-17T08:26:08.814090+0000","last_undegraded":"2024-09-17T08:26:08.814090+0000","last_fullsized":"2024-09-17T08:26:08.814090+0000","mapping_epoch":287,"log_start":"300'4100","ondisk_log_start":"300'4100","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438501376,"num_objects":6691,"num_object_clones":0,"num_object_copies":20073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":519936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":383,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607939+0000","last_change":"2024-09-17T08:25:23.209427+0000","last_active":"2024-09-17T08:26:10.607939+0000","last_peered":"2024-09-17T08:26:10.607939+0000","last_clean":"2024-09-17T08:26:10.607939+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:26:10.607939+0000","last_undegraded":"2024-09-17T08:26:10.607939+0000","last_fullsized":"2024-09-17T08:26:10.607939+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:25:23.209287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.209287+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.695831+0000","scrub_duration":0.0047713479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.514728+0000","last_change":"2024-09-17T08:25:33.132550+0000","last_active":"2024-09-17T08:26:04.514728+0000","last_peered":"2024-09-17T08:26:04.514728+0000","last_clean":"2024-09-17T08:26:04.514728+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:26:04.514728+0000","last_undegraded":"2024-09-17T08:26:04.514728+0000","last_fullsized":"2024-09-17T08:26:04.514728+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:33.131878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.131878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:58.566704+0000","scrub_duration":0.008861519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":507,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.600601+0000","last_change":"2024-09-17T08:25:11.078189+0000","last_active":"2024-09-17T08:26:10.600601+0000","last_peered":"2024-09-17T08:26:10.600601+0000","last_clean":"2024-09-17T08:26:10.600601+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:26:10.600601+0000","last_undegraded":"2024-09-17T08:26:10.600601+0000","last_fullsized":"2024-09-17T08:26:10.600601+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:11.078003+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.078003+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:39.754804+0000","scrub_duration":0.0057575789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":361,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608524+0000","last_change":"2024-09-17T08:23:25.473468+0000","last_active":"2024-09-17T08:26:10.608524+0000","last_peered":"2024-09-17T08:26:10.608524+0000","last_clean":"2024-09-17T08:26:10.608524+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:26:10.608524+0000","last_undegraded":"2024-09-17T08:26:10.608524+0000","last_fullsized":"2024-09-17T08:26:10.608524+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.473322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.473322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005682973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"326'10299","reported_seq":10348,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.690720+0000","last_change":"2024-09-17T08:24:04.459231+0000","last_active":"2024-09-17T08:26:12.690720+0000","last_peered":"2024-09-17T08:26:12.690720+0000","last_clean":"2024-09-17T08:26:12.690720+0000","last_became_active":"2024-09-17T08:24:04.458930+0000","last_became_peered":"2024-09-17T08:24:04.458930+0000","last_unstale":"2024-09-17T08:26:12.690720+0000","last_undegraded":"2024-09-17T08:26:12.690720+0000","last_fullsized":"2024-09-17T08:26:12.690720+0000","mapping_epoch":287,"log_start":"305'5300","ondisk_log_start":"305'5300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429195264,"num_objects":6549,"num_object_clones":0,"num_object_copies":19647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10299,"num_write_kb":539136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":235,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599176+0000","last_change":"2024-09-17T08:25:39.196006+0000","last_active":"2024-09-17T08:26:10.599176+0000","last_peered":"2024-09-17T08:26:10.599176+0000","last_clean":"2024-09-17T08:26:10.599176+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:26:10.599176+0000","last_undegraded":"2024-09-17T08:26:10.599176+0000","last_fullsized":"2024-09-17T08:26:10.599176+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:39.195587+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.195587+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:00.706411+0000","scrub_duration":0.0050104839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.506785+0000","last_change":"2024-09-17T08:25:12.167258+0000","last_active":"2024-09-17T08:26:04.506785+0000","last_peered":"2024-09-17T08:26:04.506785+0000","last_clean":"2024-09-17T08:26:04.506785+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:26:04.506785+0000","last_undegraded":"2024-09-17T08:26:04.506785+0000","last_fullsized":"2024-09-17T08:26:04.506785+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.167132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.167132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:17.266376+0000","scrub_duration":0.0020782719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":544,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608651+0000","last_change":"2024-09-17T08:24:14.547197+0000","last_active":"2024-09-17T08:26:10.608651+0000","last_peered":"2024-09-17T08:26:10.608651+0000","last_clean":"2024-09-17T08:26:10.608651+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:26:10.608651+0000","last_undegraded":"2024-09-17T08:26:10.608651+0000","last_fullsized":"2024-09-17T08:26:10.608651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:24:14.547105+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:24:14.547105+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013424540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510005+0000","last_change":"2024-09-17T08:25:28.215581+0000","last_active":"2024-09-17T08:26:04.510005+0000","last_peered":"2024-09-17T08:26:04.510005+0000","last_clean":"2024-09-17T08:26:04.510005+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:26:04.510005+0000","last_undegraded":"2024-09-17T08:26:04.510005+0000","last_fullsized":"2024-09-17T08:26:04.510005+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.215457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.215457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.036848+0000","scrub_duration":0.0070925789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"323'9840","reported_seq":9886,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:08.802102+0000","last_change":"2024-09-17T08:24:04.455364+0000","last_active":"2024-09-17T08:26:08.802102+0000","last_peered":"2024-09-17T08:26:08.802102+0000","last_clean":"2024-09-17T08:26:08.802102+0000","last_became_active":"2024-09-17T08:24:04.455098+0000","last_became_peered":"2024-09-17T08:24:04.455098+0000","last_unstale":"2024-09-17T08:26:08.802102+0000","last_undegraded":"2024-09-17T08:26:08.802102+0000","last_fullsized":"2024-09-17T08:26:08.802102+0000","mapping_epoch":287,"log_start":"300'4300","ondisk_log_start":"300'4300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5540,"log_dups_size":0,"ondisk_log_size":5540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442368000,"num_objects":6750,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9840,"num_write_kb":530880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":421,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608720+0000","last_change":"2024-09-17T08:24:25.581898+0000","last_active":"2024-09-17T08:26:10.608720+0000","last_peered":"2024-09-17T08:26:10.608720+0000","last_clean":"2024-09-17T08:26:10.608720+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:26:10.608720+0000","last_undegraded":"2024-09-17T08:26:10.608720+0000","last_fullsized":"2024-09-17T08:26:10.608720+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:24:25.581740+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.581740+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004957894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":372,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.687114+0000","last_change":"2024-09-17T08:26:12.688090+0000","last_active":"2024-09-17T08:26:12.687114+0000","last_peered":"2024-09-17T08:26:12.687114+0000","last_clean":"2024-09-17T08:26:12.687114+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:26:12.687114+0000","last_undegraded":"2024-09-17T08:26:12.687114+0000","last_fullsized":"2024-09-17T08:26:12.687114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:12.687091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.687091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.608122+0000","scrub_duration":0.006317067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":243,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607676+0000","last_change":"2024-09-17T08:24:30.522675+0000","last_active":"2024-09-17T08:26:10.607676+0000","last_peered":"2024-09-17T08:26:10.607676+0000","last_clean":"2024-09-17T08:26:10.607676+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:26:10.607676+0000","last_undegraded":"2024-09-17T08:26:10.607676+0000","last_fullsized":"2024-09-17T08:26:10.607676+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.522581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001427191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":335,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520173+0000","last_change":"2024-09-17T08:25:40.909763+0000","last_active":"2024-09-17T08:26:04.520173+0000","last_peered":"2024-09-17T08:26:04.520173+0000","last_clean":"2024-09-17T08:26:04.520173+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:26:04.520173+0000","last_undegraded":"2024-09-17T08:26:04.520173+0000","last_fullsized":"2024-09-17T08:26:04.520173+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.909493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.909493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:51.177881+0000","scrub_duration":0.014995481999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"323'9796","reported_seq":9841,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:08.639347+0000","last_change":"2024-09-17T08:24:04.454563+0000","last_active":"2024-09-17T08:26:08.639347+0000","last_peered":"2024-09-17T08:26:08.639347+0000","last_clean":"2024-09-17T08:26:08.639347+0000","last_became_active":"2024-09-17T08:24:04.454206+0000","last_became_peered":"2024-09-17T08:24:04.454206+0000","last_unstale":"2024-09-17T08:26:08.639347+0000","last_undegraded":"2024-09-17T08:26:08.639347+0000","last_fullsized":"2024-09-17T08:26:08.639347+0000","mapping_epoch":287,"log_start":"302'4600","ondisk_log_start":"302'4600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5196,"log_dups_size":0,"ondisk_log_size":5196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439746560,"num_objects":6710,"num_object_clones":0,"num_object_copies":20130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9796,"num_write_kb":528192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":384,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610575+0000","last_change":"2024-09-17T08:25:12.405766+0000","last_active":"2024-09-17T08:26:10.610575+0000","last_peered":"2024-09-17T08:26:10.610575+0000","last_clean":"2024-09-17T08:26:10.610575+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:26:10.610575+0000","last_undegraded":"2024-09-17T08:26:10.610575+0000","last_fullsized":"2024-09-17T08:26:10.610575+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:25:12.405574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.405574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:21.123338+0000","scrub_duration":0.00623603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":237,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607730+0000","last_change":"2024-09-17T08:24:31.660718+0000","last_active":"2024-09-17T08:26:10.607730+0000","last_peered":"2024-09-17T08:26:10.607730+0000","last_clean":"2024-09-17T08:26:10.607730+0000","last_became_active":"2024-09-17T08:23:18.669266+0000","last_became_peered":"2024-09-17T08:23:18.669266+0000","last_unstale":"2024-09-17T08:26:10.607730+0000","last_undegraded":"2024-09-17T08:26:10.607730+0000","last_fullsized":"2024-09-17T08:26:10.607730+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:24:31.660592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00371011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":573,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:06.519207+0000","last_change":"2024-09-17T08:24:12.872168+0000","last_active":"2024-09-17T08:26:06.519207+0000","last_peered":"2024-09-17T08:26:06.519207+0000","last_clean":"2024-09-17T08:26:06.519207+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:26:06.519207+0000","last_undegraded":"2024-09-17T08:26:06.519207+0000","last_fullsized":"2024-09-17T08:26:06.519207+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:24:12.872090+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.872090+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606483+0000","last_change":"2024-09-17T08:25:29.822800+0000","last_active":"2024-09-17T08:26:10.606483+0000","last_peered":"2024-09-17T08:26:10.606483+0000","last_clean":"2024-09-17T08:26:10.606483+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:26:10.606483+0000","last_undegraded":"2024-09-17T08:26:10.606483+0000","last_fullsized":"2024-09-17T08:26:10.606483+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.822689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.822689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.095715+0000","scrub_duration":0.0085561849999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"326'10154","reported_seq":10202,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.678240+0000","last_change":"2024-09-17T08:24:04.453118+0000","last_active":"2024-09-17T08:26:12.678240+0000","last_peered":"2024-09-17T08:26:12.678240+0000","last_clean":"2024-09-17T08:26:12.678240+0000","last_became_active":"2024-09-17T08:24:04.452780+0000","last_became_peered":"2024-09-17T08:24:04.452780+0000","last_unstale":"2024-09-17T08:26:12.678240+0000","last_undegraded":"2024-09-17T08:26:12.678240+0000","last_fullsized":"2024-09-17T08:26:12.678240+0000","mapping_epoch":287,"log_start":"305'5200","ondisk_log_start":"305'5200","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428736512,"num_objects":6542,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10154,"num_write_kb":534272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":390,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.519324+0000","last_change":"2024-09-17T08:24:03.927221+0000","last_active":"2024-09-17T08:26:04.519324+0000","last_peered":"2024-09-17T08:26:04.519324+0000","last_clean":"2024-09-17T08:26:04.519324+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:26:04.519324+0000","last_undegraded":"2024-09-17T08:26:04.519324+0000","last_fullsized":"2024-09-17T08:26:04.519324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:24:03.927109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.927109+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026636759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510653+0000","last_change":"2024-09-17T08:25:31.165549+0000","last_active":"2024-09-17T08:26:04.510653+0000","last_peered":"2024-09-17T08:26:04.510653+0000","last_clean":"2024-09-17T08:26:04.510653+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:26:04.510653+0000","last_undegraded":"2024-09-17T08:26:04.510653+0000","last_fullsized":"2024-09-17T08:26:04.510653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:31.165432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:25:31.165432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:36.847226+0000","scrub_duration":0.001729558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":340,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610442+0000","last_change":"2024-09-17T08:23:32.530792+0000","last_active":"2024-09-17T08:26:10.610442+0000","last_peered":"2024-09-17T08:26:10.610442+0000","last_clean":"2024-09-17T08:26:10.610442+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:26:10.610442+0000","last_undegraded":"2024-09-17T08:26:10.610442+0000","last_fullsized":"2024-09-17T08:26:10.610442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:32.530685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:23:32.530685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016463910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607373+0000","last_change":"2024-09-17T08:25:39.349334+0000","last_active":"2024-09-17T08:26:10.607373+0000","last_peered":"2024-09-17T08:26:10.607373+0000","last_clean":"2024-09-17T08:26:10.607373+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:26:10.607373+0000","last_undegraded":"2024-09-17T08:26:10.607373+0000","last_fullsized":"2024-09-17T08:26:10.607373+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:39.349120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.349120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.525815+0000","scrub_duration":0.005278652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"326'10063","reported_seq":10112,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.685664+0000","last_change":"2024-09-17T08:24:04.456347+0000","last_active":"2024-09-17T08:26:12.685664+0000","last_peered":"2024-09-17T08:26:12.685664+0000","last_clean":"2024-09-17T08:26:12.685664+0000","last_became_active":"2024-09-17T08:24:04.456039+0000","last_became_peered":"2024-09-17T08:24:04.456039+0000","last_unstale":"2024-09-17T08:26:12.685664+0000","last_undegraded":"2024-09-17T08:26:12.685664+0000","last_fullsized":"2024-09-17T08:26:12.685664+0000","mapping_epoch":287,"log_start":"304'5100","ondisk_log_start":"304'5100","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4963,"log_dups_size":0,"ondisk_log_size":4963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421265432,"num_objects":6429,"num_object_clones":0,"num_object_copies":19287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10063,"num_write_kb":527681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":415,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.052389+0000","last_change":"2024-09-17T08:26:12.052601+0000","last_active":"2024-09-17T08:26:12.052389+0000","last_peered":"2024-09-17T08:26:12.052389+0000","last_clean":"2024-09-17T08:26:12.052389+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:26:12.052389+0000","last_undegraded":"2024-09-17T08:26:12.052389+0000","last_fullsized":"2024-09-17T08:26:12.052389+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:26:12.052367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.052367+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.321528+0000","scrub_duration":0.0032600939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":359,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607090+0000","last_change":"2024-09-17T08:24:23.674368+0000","last_active":"2024-09-17T08:26:10.607090+0000","last_peered":"2024-09-17T08:26:10.607090+0000","last_clean":"2024-09-17T08:26:10.607090+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:26:10.607090+0000","last_undegraded":"2024-09-17T08:26:10.607090+0000","last_fullsized":"2024-09-17T08:26:10.607090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:24:23.674249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:23.674249+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041183280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":307,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.609346+0000","last_change":"2024-09-17T08:24:15.691704+0000","last_active":"2024-09-17T08:26:10.609346+0000","last_peered":"2024-09-17T08:26:10.609346+0000","last_clean":"2024-09-17T08:26:10.609346+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:26:10.609346+0000","last_undegraded":"2024-09-17T08:26:10.609346+0000","last_fullsized":"2024-09-17T08:26:10.609346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:15.691622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.691622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034962180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":246,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.506916+0000","last_change":"2024-09-17T08:24:12.919326+0000","last_active":"2024-09-17T08:26:04.506916+0000","last_peered":"2024-09-17T08:26:04.506916+0000","last_clean":"2024-09-17T08:26:04.506916+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:26:04.506916+0000","last_undegraded":"2024-09-17T08:26:04.506916+0000","last_fullsized":"2024-09-17T08:26:04.506916+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:12.919247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.919247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013913059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"326'10186","reported_seq":10393,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.749631+0000","last_change":"2024-09-17T08:25:36.457603+0000","last_active":"2024-09-17T08:26:12.749631+0000","last_peered":"2024-09-17T08:26:12.749631+0000","last_clean":"2024-09-17T08:26:12.749631+0000","last_became_active":"2024-09-17T08:24:04.458943+0000","last_became_peered":"2024-09-17T08:24:04.458943+0000","last_unstale":"2024-09-17T08:26:12.749631+0000","last_undegraded":"2024-09-17T08:26:12.749631+0000","last_fullsized":"2024-09-17T08:26:12.749631+0000","mapping_epoch":287,"log_start":"304'5100","ondisk_log_start":"304'5100","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'7478","last_scrub_stamp":"2024-09-17T08:25:36.457326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:36.457326+0000","objects_scrubbed":7398,"log_size":5086,"log_dups_size":0,"ondisk_log_size":5086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:50.045951+0000","scrub_duration":2.6174414289999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425852928,"num_objects":6498,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":533888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":263,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.513691+0000","last_change":"2024-09-17T08:24:30.840463+0000","last_active":"2024-09-17T08:26:04.513691+0000","last_peered":"2024-09-17T08:26:04.513691+0000","last_clean":"2024-09-17T08:26:04.513691+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:26:04.513691+0000","last_undegraded":"2024-09-17T08:26:04.513691+0000","last_fullsized":"2024-09-17T08:26:04.513691+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:24:30.840289+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.840289+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056047780000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":231,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.509918+0000","last_change":"2024-09-17T08:25:14.210636+0000","last_active":"2024-09-17T08:26:04.509918+0000","last_peered":"2024-09-17T08:26:04.509918+0000","last_clean":"2024-09-17T08:26:04.509918+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:26:04.509918+0000","last_undegraded":"2024-09-17T08:26:04.509918+0000","last_fullsized":"2024-09-17T08:26:04.509918+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.210528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.210528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.818087+0000","scrub_duration":0.0019364110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":555,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607335+0000","last_change":"2024-09-17T08:23:28.852946+0000","last_active":"2024-09-17T08:26:10.607335+0000","last_peered":"2024-09-17T08:26:10.607335+0000","last_clean":"2024-09-17T08:26:10.607335+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:26:10.607335+0000","last_undegraded":"2024-09-17T08:26:10.607335+0000","last_fullsized":"2024-09-17T08:26:10.607335+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.852455+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028496810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":366,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607030+0000","last_change":"2024-09-17T08:25:37.001934+0000","last_active":"2024-09-17T08:26:10.607030+0000","last_peered":"2024-09-17T08:26:10.607030+0000","last_clean":"2024-09-17T08:26:10.607030+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:26:10.607030+0000","last_undegraded":"2024-09-17T08:26:10.607030+0000","last_fullsized":"2024-09-17T08:26:10.607030+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:37.001438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.001438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:54.154447+0000","scrub_duration":0.0093143449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"326'9991","reported_seq":10039,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.311127+0000","last_change":"2024-09-17T08:24:04.457900+0000","last_active":"2024-09-17T08:26:12.311127+0000","last_peered":"2024-09-17T08:26:12.311127+0000","last_clean":"2024-09-17T08:26:12.311127+0000","last_became_active":"2024-09-17T08:24:04.457614+0000","last_became_peered":"2024-09-17T08:24:04.457614+0000","last_unstale":"2024-09-17T08:26:12.311127+0000","last_undegraded":"2024-09-17T08:26:12.311127+0000","last_fullsized":"2024-09-17T08:26:12.311127+0000","mapping_epoch":287,"log_start":"303'4700","ondisk_log_start":"303'4700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5291,"log_dups_size":0,"ondisk_log_size":5291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418054144,"num_objects":6379,"num_object_clones":0,"num_object_copies":19137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9991,"num_write_kb":523840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":420,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:06.512190+0000","last_change":"2024-09-17T08:24:03.009933+0000","last_active":"2024-09-17T08:26:06.512190+0000","last_peered":"2024-09-17T08:26:06.512190+0000","last_clean":"2024-09-17T08:26:06.512190+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:26:06.512190+0000","last_undegraded":"2024-09-17T08:26:06.512190+0000","last_fullsized":"2024-09-17T08:26:06.512190+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:24:03.009831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.009831+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044852599999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":231,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.514563+0000","last_change":"2024-09-17T08:23:28.072844+0000","last_active":"2024-09-17T08:26:04.514563+0000","last_peered":"2024-09-17T08:26:04.514563+0000","last_clean":"2024-09-17T08:26:04.514563+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:26:04.514563+0000","last_undegraded":"2024-09-17T08:26:04.514563+0000","last_fullsized":"2024-09-17T08:26:04.514563+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:28.072608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.072608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079140149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":369,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610816+0000","last_change":"2024-09-17T08:24:17.685902+0000","last_active":"2024-09-17T08:26:10.610816+0000","last_peered":"2024-09-17T08:26:10.610816+0000","last_clean":"2024-09-17T08:26:10.610816+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:26:10.610816+0000","last_undegraded":"2024-09-17T08:26:10.610816+0000","last_fullsized":"2024-09-17T08:26:10.610816+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:17.685819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.685819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015007639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":346,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.519930+0000","last_change":"2024-09-17T08:25:29.848581+0000","last_active":"2024-09-17T08:26:04.519930+0000","last_peered":"2024-09-17T08:26:04.519930+0000","last_clean":"2024-09-17T08:26:04.519930+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:26:04.519930+0000","last_undegraded":"2024-09-17T08:26:04.519930+0000","last_fullsized":"2024-09-17T08:26:04.519930+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.848449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.848449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.490576+0000","scrub_duration":0.0072796190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"326'10007","reported_seq":10054,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.638223+0000","last_change":"2024-09-17T08:24:04.454052+0000","last_active":"2024-09-17T08:26:11.638223+0000","last_peered":"2024-09-17T08:26:11.638223+0000","last_clean":"2024-09-17T08:26:11.638223+0000","last_became_active":"2024-09-17T08:24:04.453753+0000","last_became_peered":"2024-09-17T08:24:04.453753+0000","last_unstale":"2024-09-17T08:26:11.638223+0000","last_undegraded":"2024-09-17T08:26:11.638223+0000","last_fullsized":"2024-09-17T08:26:11.638223+0000","mapping_epoch":287,"log_start":"299'3600","ondisk_log_start":"299'3600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":6407,"log_dups_size":0,"ondisk_log_size":6407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425656320,"num_objects":6495,"num_object_clones":0,"num_object_copies":19485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":528064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":248,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510938+0000","last_change":"2024-09-17T08:25:19.161609+0000","last_active":"2024-09-17T08:26:04.510938+0000","last_peered":"2024-09-17T08:26:04.510938+0000","last_clean":"2024-09-17T08:26:04.510938+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:26:04.510938+0000","last_undegraded":"2024-09-17T08:26:04.510938+0000","last_fullsized":"2024-09-17T08:26:04.510938+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:25:19.161454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.161454+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:24.995624+0000","scrub_duration":0.015453119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":887,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:07.545396+0000","last_change":"2024-09-17T08:24:11.910215+0000","last_active":"2024-09-17T08:26:07.545396+0000","last_peered":"2024-09-17T08:26:07.545396+0000","last_clean":"2024-09-17T08:26:07.545396+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:26:07.545396+0000","last_undegraded":"2024-09-17T08:26:07.545396+0000","last_fullsized":"2024-09-17T08:26:07.545396+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:24:11.910131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:11.910131+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001356576,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":636,"num_read_kb":631,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":328,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.601443+0000","last_change":"2024-09-17T08:24:09.843581+0000","last_active":"2024-09-17T08:26:10.601443+0000","last_peered":"2024-09-17T08:26:10.601443+0000","last_clean":"2024-09-17T08:26:10.601443+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:26:10.601443+0000","last_undegraded":"2024-09-17T08:26:10.601443+0000","last_fullsized":"2024-09-17T08:26:10.601443+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:09.843489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:24:09.843489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012254729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":336,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.600955+0000","last_change":"2024-09-17T08:24:21.765981+0000","last_active":"2024-09-17T08:26:10.600955+0000","last_peered":"2024-09-17T08:26:10.600955+0000","last_clean":"2024-09-17T08:26:10.600955+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:26:10.600955+0000","last_undegraded":"2024-09-17T08:26:10.600955+0000","last_fullsized":"2024-09-17T08:26:10.600955+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:21.765863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.765863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":359,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606747+0000","last_change":"2024-09-17T08:25:18.912338+0000","last_active":"2024-09-17T08:26:10.606747+0000","last_peered":"2024-09-17T08:26:10.606747+0000","last_clean":"2024-09-17T08:26:10.606747+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:26:10.606747+0000","last_undegraded":"2024-09-17T08:26:10.606747+0000","last_fullsized":"2024-09-17T08:26:10.606747+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.912202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.912202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.360980+0000","scrub_duration":0.0058719879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"326'10170","reported_seq":10219,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.326486+0000","last_change":"2024-09-17T08:24:04.459213+0000","last_active":"2024-09-17T08:26:12.326486+0000","last_peered":"2024-09-17T08:26:12.326486+0000","last_clean":"2024-09-17T08:26:12.326486+0000","last_became_active":"2024-09-17T08:24:04.458862+0000","last_became_peered":"2024-09-17T08:24:04.458862+0000","last_unstale":"2024-09-17T08:26:12.326486+0000","last_undegraded":"2024-09-17T08:26:12.326486+0000","last_fullsized":"2024-09-17T08:26:12.326486+0000","mapping_epoch":287,"log_start":"303'4900","ondisk_log_start":"303'4900","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431882240,"num_objects":6590,"num_object_clones":0,"num_object_copies":19770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10170,"num_write_kb":536320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":471,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.612285+0000","last_change":"2024-09-17T08:25:37.983837+0000","last_active":"2024-09-17T08:26:10.612285+0000","last_peered":"2024-09-17T08:26:10.612285+0000","last_clean":"2024-09-17T08:26:10.612285+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:26:10.612285+0000","last_undegraded":"2024-09-17T08:26:10.612285+0000","last_fullsized":"2024-09-17T08:26:10.612285+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:25:37.983609+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.983609+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.256761+0000","scrub_duration":0.0081599319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":373,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608133+0000","last_change":"2024-09-17T08:25:17.027645+0000","last_active":"2024-09-17T08:26:10.608133+0000","last_peered":"2024-09-17T08:26:10.608133+0000","last_clean":"2024-09-17T08:26:10.608133+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:26:10.608133+0000","last_undegraded":"2024-09-17T08:26:10.608133+0000","last_fullsized":"2024-09-17T08:26:10.608133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:25:17.027377+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.027377+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.587148+0000","scrub_duration":0.0088147999999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":384,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606741+0000","last_change":"2024-09-17T08:25:34.874991+0000","last_active":"2024-09-17T08:26:10.606741+0000","last_peered":"2024-09-17T08:26:10.606741+0000","last_clean":"2024-09-17T08:26:10.606741+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:26:10.606741+0000","last_undegraded":"2024-09-17T08:26:10.606741+0000","last_fullsized":"2024-09-17T08:26:10.606741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.874848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.874848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.051778+0000","scrub_duration":0.0082107689999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.511010+0000","last_change":"2024-09-17T08:23:25.014558+0000","last_active":"2024-09-17T08:26:04.511010+0000","last_peered":"2024-09-17T08:26:04.511010+0000","last_clean":"2024-09-17T08:26:04.511010+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:26:04.511010+0000","last_undegraded":"2024-09-17T08:26:04.511010+0000","last_fullsized":"2024-09-17T08:26:04.511010+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.014448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.014448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015526198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":231,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.509741+0000","last_change":"2024-09-17T08:24:28.859552+0000","last_active":"2024-09-17T08:26:04.509741+0000","last_peered":"2024-09-17T08:26:04.509741+0000","last_clean":"2024-09-17T08:26:04.509741+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:26:04.509741+0000","last_undegraded":"2024-09-17T08:26:04.509741+0000","last_fullsized":"2024-09-17T08:26:04.509741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:28.859427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:28.859427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051469929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"323'9763","reported_seq":9808,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:08.804847+0000","last_change":"2024-09-17T08:24:04.456167+0000","last_active":"2024-09-17T08:26:08.804847+0000","last_peered":"2024-09-17T08:26:08.804847+0000","last_clean":"2024-09-17T08:26:08.804847+0000","last_became_active":"2024-09-17T08:24:04.455726+0000","last_became_peered":"2024-09-17T08:24:04.455726+0000","last_unstale":"2024-09-17T08:26:08.804847+0000","last_undegraded":"2024-09-17T08:26:08.804847+0000","last_fullsized":"2024-09-17T08:26:08.804847+0000","mapping_epoch":287,"log_start":"302'4300","ondisk_log_start":"302'4300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5463,"log_dups_size":0,"ondisk_log_size":5463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437059584,"num_objects":6669,"num_object_clones":0,"num_object_copies":20007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9763,"num_write_kb":525824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":344,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608117+0000","last_change":"2024-09-17T08:23:20.405772+0000","last_active":"2024-09-17T08:26:10.608117+0000","last_peered":"2024-09-17T08:26:10.608117+0000","last_clean":"2024-09-17T08:26:10.608117+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:26:10.608117+0000","last_undegraded":"2024-09-17T08:26:10.608117+0000","last_fullsized":"2024-09-17T08:26:10.608117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:23:20.405620+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:23:20.405620+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051583360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":348,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.519731+0000","last_change":"2024-09-17T08:24:06.873073+0000","last_active":"2024-09-17T08:26:04.519731+0000","last_peered":"2024-09-17T08:26:04.519731+0000","last_clean":"2024-09-17T08:26:04.519731+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:26:04.519731+0000","last_undegraded":"2024-09-17T08:26:04.519731+0000","last_fullsized":"2024-09-17T08:26:04.519731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:24:06.872995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:06.872995+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019197509999999999,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598790+0000","last_change":"2024-09-17T08:25:42.221990+0000","last_active":"2024-09-17T08:26:10.598790+0000","last_peered":"2024-09-17T08:26:10.598790+0000","last_clean":"2024-09-17T08:26:10.598790+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:26:10.598790+0000","last_undegraded":"2024-09-17T08:26:10.598790+0000","last_fullsized":"2024-09-17T08:26:10.598790+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:42.221590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:25:42.221590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.243498+0000","scrub_duration":0.005773388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":616,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606887+0000","last_change":"2024-09-17T08:24:24.654508+0000","last_active":"2024-09-17T08:26:10.606887+0000","last_peered":"2024-09-17T08:26:10.606887+0000","last_clean":"2024-09-17T08:26:10.606887+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:26:10.606887+0000","last_undegraded":"2024-09-17T08:26:10.606887+0000","last_fullsized":"2024-09-17T08:26:10.606887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:24:24.654371+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014600804,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":365,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608627+0000","last_change":"2024-09-17T08:25:38.817905+0000","last_active":"2024-09-17T08:26:10.608627+0000","last_peered":"2024-09-17T08:26:10.608627+0000","last_clean":"2024-09-17T08:26:10.608627+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:26:10.608627+0000","last_undegraded":"2024-09-17T08:26:10.608627+0000","last_fullsized":"2024-09-17T08:26:10.608627+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.817546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:04.248082+0000","scrub_duration":0.002079291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":251,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521087+0000","last_change":"2024-09-17T08:25:41.900666+0000","last_active":"2024-09-17T08:26:04.521087+0000","last_peered":"2024-09-17T08:26:04.521087+0000","last_clean":"2024-09-17T08:26:04.521087+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:26:04.521087+0000","last_undegraded":"2024-09-17T08:26:04.521087+0000","last_fullsized":"2024-09-17T08:26:04.521087+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.900483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.900483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.112986+0000","scrub_duration":0.013601894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"326'10090","reported_seq":10136,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.314853+0000","last_change":"2024-09-17T08:24:04.458040+0000","last_active":"2024-09-17T08:26:12.314853+0000","last_peered":"2024-09-17T08:26:12.314853+0000","last_clean":"2024-09-17T08:26:12.314853+0000","last_became_active":"2024-09-17T08:24:04.457745+0000","last_became_peered":"2024-09-17T08:24:04.457745+0000","last_unstale":"2024-09-17T08:26:12.314853+0000","last_undegraded":"2024-09-17T08:26:12.314853+0000","last_fullsized":"2024-09-17T08:26:12.314853+0000","mapping_epoch":287,"log_start":"303'4800","ondisk_log_start":"303'4800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422969344,"num_objects":6454,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10090,"num_write_kb":529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":189,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.514644+0000","last_change":"2024-09-17T08:25:35.125962+0000","last_active":"2024-09-17T08:26:04.514644+0000","last_peered":"2024-09-17T08:26:04.514644+0000","last_clean":"2024-09-17T08:26:04.514644+0000","last_became_active":"2024-09-17T08:23:18.670977+0000","last_became_peered":"2024-09-17T08:23:18.670977+0000","last_unstale":"2024-09-17T08:26:04.514644+0000","last_undegraded":"2024-09-17T08:26:04.514644+0000","last_fullsized":"2024-09-17T08:26:04.514644+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:25:35.125100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:35.125100+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.214432+0000","scrub_duration":0.0023912690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":317,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599244+0000","last_change":"2024-09-17T08:24:07.886685+0000","last_active":"2024-09-17T08:26:10.599244+0000","last_peered":"2024-09-17T08:26:10.599244+0000","last_clean":"2024-09-17T08:26:10.599244+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:26:10.599244+0000","last_undegraded":"2024-09-17T08:26:10.599244+0000","last_fullsized":"2024-09-17T08:26:10.599244+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:07.886609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:07.886609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001519378,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":334,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598716+0000","last_change":"2024-09-17T08:25:13.057707+0000","last_active":"2024-09-17T08:26:10.598716+0000","last_peered":"2024-09-17T08:26:10.598716+0000","last_clean":"2024-09-17T08:26:10.598716+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:26:10.598716+0000","last_undegraded":"2024-09-17T08:26:10.598716+0000","last_fullsized":"2024-09-17T08:26:10.598716+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:13.057534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.057534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.726370+0000","scrub_duration":0.011530043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":352,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606806+0000","last_change":"2024-09-17T08:24:15.515453+0000","last_active":"2024-09-17T08:26:10.606806+0000","last_peered":"2024-09-17T08:26:10.606806+0000","last_clean":"2024-09-17T08:26:10.606806+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:26:10.606806+0000","last_undegraded":"2024-09-17T08:26:10.606806+0000","last_fullsized":"2024-09-17T08:26:10.606806+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:15.515366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.515366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001755695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"326'10011","reported_seq":10060,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.641810+0000","last_change":"2024-09-17T08:24:04.459145+0000","last_active":"2024-09-17T08:26:11.641810+0000","last_peered":"2024-09-17T08:26:11.641810+0000","last_clean":"2024-09-17T08:26:11.641810+0000","last_became_active":"2024-09-17T08:24:04.458851+0000","last_became_peered":"2024-09-17T08:24:04.458851+0000","last_unstale":"2024-09-17T08:26:11.641810+0000","last_undegraded":"2024-09-17T08:26:11.641810+0000","last_fullsized":"2024-09-17T08:26:11.641810+0000","mapping_epoch":287,"log_start":"298'3600","ondisk_log_start":"298'3600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432734208,"num_objects":6603,"num_object_clones":0,"num_object_copies":19809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10011,"num_write_kb":531648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":416,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607411+0000","last_change":"2024-09-17T08:25:16.078092+0000","last_active":"2024-09-17T08:26:10.607411+0000","last_peered":"2024-09-17T08:26:10.607411+0000","last_clean":"2024-09-17T08:26:10.607411+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:26:10.607411+0000","last_undegraded":"2024-09-17T08:26:10.607411+0000","last_fullsized":"2024-09-17T08:26:10.607411+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:25:16.077960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:16.077960+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.021838+0000","scrub_duration":0.021317215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.509852+0000","last_change":"2024-09-17T08:22:40.621682+0000","last_active":"2024-09-17T08:26:04.509852+0000","last_peered":"2024-09-17T08:26:04.509852+0000","last_clean":"2024-09-17T08:26:04.509852+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:26:04.509852+0000","last_undegraded":"2024-09-17T08:26:04.509852+0000","last_fullsized":"2024-09-17T08:26:04.509852+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:22:40.621579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:22:40.621579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048029379999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":337,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598563+0000","last_change":"2024-09-17T08:24:26.864669+0000","last_active":"2024-09-17T08:26:10.598563+0000","last_peered":"2024-09-17T08:26:10.598563+0000","last_clean":"2024-09-17T08:26:10.598563+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:26:10.598563+0000","last_undegraded":"2024-09-17T08:26:10.598563+0000","last_fullsized":"2024-09-17T08:26:10.598563+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:26.864558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.864558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027307770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608088+0000","last_change":"2024-09-17T08:25:40.906564+0000","last_active":"2024-09-17T08:26:10.608088+0000","last_peered":"2024-09-17T08:26:10.608088+0000","last_clean":"2024-09-17T08:26:10.608088+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:26:10.608088+0000","last_undegraded":"2024-09-17T08:26:10.608088+0000","last_fullsized":"2024-09-17T08:26:10.608088+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.906437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.906437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.936477+0000","scrub_duration":0.014127516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"326'10004","reported_seq":10053,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.328433+0000","last_change":"2024-09-17T08:24:04.458318+0000","last_active":"2024-09-17T08:26:12.328433+0000","last_peered":"2024-09-17T08:26:12.328433+0000","last_clean":"2024-09-17T08:26:12.328433+0000","last_became_active":"2024-09-17T08:24:04.458042+0000","last_became_peered":"2024-09-17T08:24:04.458042+0000","last_unstale":"2024-09-17T08:26:12.328433+0000","last_undegraded":"2024-09-17T08:26:12.328433+0000","last_fullsized":"2024-09-17T08:26:12.328433+0000","mapping_epoch":287,"log_start":"303'4700","ondisk_log_start":"303'4700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416546816,"num_objects":6356,"num_object_clones":0,"num_object_copies":19068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10004,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":408,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606486+0000","last_change":"2024-09-17T08:25:13.823972+0000","last_active":"2024-09-17T08:26:10.606486+0000","last_peered":"2024-09-17T08:26:10.606486+0000","last_clean":"2024-09-17T08:26:10.606486+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:26:10.606486+0000","last_undegraded":"2024-09-17T08:26:10.606486+0000","last_fullsized":"2024-09-17T08:26:10.606486+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:13.823819+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.823819+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.592729+0000","scrub_duration":0.0031292669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":349,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603391+0000","last_change":"2024-09-17T08:25:27.161139+0000","last_active":"2024-09-17T08:26:10.603391+0000","last_peered":"2024-09-17T08:26:10.603391+0000","last_clean":"2024-09-17T08:26:10.603391+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:26:10.603391+0000","last_undegraded":"2024-09-17T08:26:10.603391+0000","last_fullsized":"2024-09-17T08:26:10.603391+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:27.160999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:27.160999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:42.471047+0000","scrub_duration":0.0058695600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":554,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606710+0000","last_change":"2024-09-17T08:25:17.907507+0000","last_active":"2024-09-17T08:26:10.606710+0000","last_peered":"2024-09-17T08:26:10.606710+0000","last_clean":"2024-09-17T08:26:10.606710+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:26:10.606710+0000","last_undegraded":"2024-09-17T08:26:10.606710+0000","last_fullsized":"2024-09-17T08:26:10.606710+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:25:17.907373+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.907373+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.869934+0000","scrub_duration":0.001744106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":191,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.609308+0000","last_change":"2024-09-17T08:24:13.623939+0000","last_active":"2024-09-17T08:26:10.609308+0000","last_peered":"2024-09-17T08:26:10.609308+0000","last_clean":"2024-09-17T08:26:10.609308+0000","last_became_active":"2024-09-17T08:23:18.662935+0000","last_became_peered":"2024-09-17T08:23:18.662935+0000","last_unstale":"2024-09-17T08:26:10.609308+0000","last_undegraded":"2024-09-17T08:26:10.609308+0000","last_fullsized":"2024-09-17T08:26:10.609308+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.623844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.623844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001519287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"323'9934","reported_seq":9979,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:08.693385+0000","last_change":"2024-09-17T08:24:04.453548+0000","last_active":"2024-09-17T08:26:08.693385+0000","last_peered":"2024-09-17T08:26:08.693385+0000","last_clean":"2024-09-17T08:26:08.693385+0000","last_became_active":"2024-09-17T08:24:04.453268+0000","last_became_peered":"2024-09-17T08:24:04.453268+0000","last_unstale":"2024-09-17T08:26:08.693385+0000","last_undegraded":"2024-09-17T08:26:08.693385+0000","last_fullsized":"2024-09-17T08:26:08.693385+0000","mapping_epoch":287,"log_start":"302'4700","ondisk_log_start":"302'4700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5234,"log_dups_size":0,"ondisk_log_size":5234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444727296,"num_objects":6786,"num_object_clones":0,"num_object_copies":20358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":535040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":427,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610508+0000","last_change":"2024-09-17T08:25:29.258817+0000","last_active":"2024-09-17T08:26:10.610508+0000","last_peered":"2024-09-17T08:26:10.610508+0000","last_clean":"2024-09-17T08:26:10.610508+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:26:10.610508+0000","last_undegraded":"2024-09-17T08:26:10.610508+0000","last_fullsized":"2024-09-17T08:26:10.610508+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:25:29.258616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.258616+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.967934+0000","scrub_duration":0.011223878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":362,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607938+0000","last_change":"2024-09-17T08:24:32.450419+0000","last_active":"2024-09-17T08:26:10.607938+0000","last_peered":"2024-09-17T08:26:10.607938+0000","last_clean":"2024-09-17T08:26:10.607938+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:26:10.607938+0000","last_undegraded":"2024-09-17T08:26:10.607938+0000","last_fullsized":"2024-09-17T08:26:10.607938+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:24:32.450300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.450300+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058717939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":353,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599236+0000","last_change":"2024-09-17T08:25:25.116593+0000","last_active":"2024-09-17T08:26:10.599236+0000","last_peered":"2024-09-17T08:26:10.599236+0000","last_clean":"2024-09-17T08:26:10.599236+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:26:10.599236+0000","last_undegraded":"2024-09-17T08:26:10.599236+0000","last_fullsized":"2024-09-17T08:26:10.599236+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:25.116441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:25:25.116441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.015212+0000","scrub_duration":0.008148832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":157,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598649+0000","last_change":"2024-09-17T08:24:10.881507+0000","last_active":"2024-09-17T08:26:10.598649+0000","last_peered":"2024-09-17T08:26:10.598649+0000","last_clean":"2024-09-17T08:26:10.598649+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:26:10.598649+0000","last_undegraded":"2024-09-17T08:26:10.598649+0000","last_fullsized":"2024-09-17T08:26:10.598649+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.881407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.881407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014082999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":381,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.612204+0000","last_change":"2024-09-17T08:25:33.033204+0000","last_active":"2024-09-17T08:26:10.612204+0000","last_peered":"2024-09-17T08:26:10.612204+0000","last_clean":"2024-09-17T08:26:10.612204+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:26:10.612204+0000","last_undegraded":"2024-09-17T08:26:10.612204+0000","last_fullsized":"2024-09-17T08:26:10.612204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.032722+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.145927+0000","scrub_duration":0.0085113229999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":367,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598717+0000","last_change":"2024-09-17T08:25:34.280542+0000","last_active":"2024-09-17T08:26:10.598717+0000","last_peered":"2024-09-17T08:26:10.598717+0000","last_clean":"2024-09-17T08:26:10.598717+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:26:10.598717+0000","last_undegraded":"2024-09-17T08:26:10.598717+0000","last_fullsized":"2024-09-17T08:26:10.598717+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.278744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.278744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.492017+0000","scrub_duration":0.011508979000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":354,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521361+0000","last_change":"2024-09-17T08:25:32.809559+0000","last_active":"2024-09-17T08:26:04.521361+0000","last_peered":"2024-09-17T08:26:04.521361+0000","last_clean":"2024-09-17T08:26:04.521361+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:26:04.521361+0000","last_undegraded":"2024-09-17T08:26:04.521361+0000","last_fullsized":"2024-09-17T08:26:04.521361+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:32.809305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:32.809305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.753241+0000","scrub_duration":0.0024200179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":340,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.533286+0000","last_change":"2024-09-17T08:26:11.533286+0000","last_active":"2024-09-17T08:26:11.533286+0000","last_peered":"2024-09-17T08:26:11.533286+0000","last_clean":"2024-09-17T08:26:11.533286+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:26:11.533286+0000","last_undegraded":"2024-09-17T08:26:11.533286+0000","last_fullsized":"2024-09-17T08:26:11.533286+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.533066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.533066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.390854+0000","scrub_duration":0.0073762410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":251,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610638+0000","last_change":"2024-09-17T08:25:18.296524+0000","last_active":"2024-09-17T08:26:10.610638+0000","last_peered":"2024-09-17T08:26:10.610638+0000","last_clean":"2024-09-17T08:26:10.610638+0000","last_became_active":"2024-09-17T08:23:18.666495+0000","last_became_peered":"2024-09-17T08:23:18.666495+0000","last_unstale":"2024-09-17T08:26:10.610638+0000","last_undegraded":"2024-09-17T08:26:10.610638+0000","last_fullsized":"2024-09-17T08:26:10.610638+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.296432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.296432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.437556+0000","scrub_duration":0.0041815000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":390,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521466+0000","last_change":"2024-09-17T08:23:29.829264+0000","last_active":"2024-09-17T08:26:04.521466+0000","last_peered":"2024-09-17T08:26:04.521466+0000","last_clean":"2024-09-17T08:26:04.521466+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:26:04.521466+0000","last_undegraded":"2024-09-17T08:26:04.521466+0000","last_fullsized":"2024-09-17T08:26:04.521466+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:23:29.829064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:23:29.829064+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065936679999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":257,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.519860+0000","last_change":"2024-09-17T08:24:19.785545+0000","last_active":"2024-09-17T08:26:04.519860+0000","last_peered":"2024-09-17T08:26:04.519860+0000","last_clean":"2024-09-17T08:26:04.519860+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:26:04.519860+0000","last_undegraded":"2024-09-17T08:26:04.519860+0000","last_fullsized":"2024-09-17T08:26:04.519860+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:19.785421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.785421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005223391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":358,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606669+0000","last_change":"2024-09-17T08:25:20.873823+0000","last_active":"2024-09-17T08:26:10.606669+0000","last_peered":"2024-09-17T08:26:10.606669+0000","last_clean":"2024-09-17T08:26:10.606669+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:26:10.606669+0000","last_undegraded":"2024-09-17T08:26:10.606669+0000","last_fullsized":"2024-09-17T08:26:10.606669+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:20.873748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:20.873748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:27.013947+0000","scrub_duration":0.001209087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":410,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520534+0000","last_change":"2024-09-17T08:24:09.894035+0000","last_active":"2024-09-17T08:26:04.520534+0000","last_peered":"2024-09-17T08:26:04.520534+0000","last_clean":"2024-09-17T08:26:04.520534+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:26:04.520534+0000","last_undegraded":"2024-09-17T08:26:04.520534+0000","last_fullsized":"2024-09-17T08:26:04.520534+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:09.893951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:09.893951+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019165969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":286,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.519623+0000","last_change":"2024-09-17T08:23:26.810186+0000","last_active":"2024-09-17T08:26:04.519623+0000","last_peered":"2024-09-17T08:26:04.519623+0000","last_clean":"2024-09-17T08:26:04.519623+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:26:04.519623+0000","last_undegraded":"2024-09-17T08:26:04.519623+0000","last_fullsized":"2024-09-17T08:26:04.519623+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:26.810075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:23:26.810075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080033750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":364,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:06.513944+0000","last_change":"2024-09-17T08:25:11.139098+0000","last_active":"2024-09-17T08:26:06.513944+0000","last_peered":"2024-09-17T08:26:06.513944+0000","last_clean":"2024-09-17T08:26:06.513944+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:26:06.513944+0000","last_undegraded":"2024-09-17T08:26:06.513944+0000","last_fullsized":"2024-09-17T08:26:06.513944+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:25:11.138956+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.138956+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:11.683034+0000","scrub_duration":0.004930266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":348,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.611907+0000","last_change":"2024-09-17T08:24:18.708208+0000","last_active":"2024-09-17T08:26:10.611907+0000","last_peered":"2024-09-17T08:26:10.611907+0000","last_clean":"2024-09-17T08:26:10.611907+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:26:10.611907+0000","last_undegraded":"2024-09-17T08:26:10.611907+0000","last_fullsized":"2024-09-17T08:26:10.611907+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:18.708083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:18.708083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010815099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":294,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606949+0000","last_change":"2024-09-17T08:24:29.653178+0000","last_active":"2024-09-17T08:26:10.606949+0000","last_peered":"2024-09-17T08:26:10.606949+0000","last_clean":"2024-09-17T08:26:10.606949+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:26:10.606949+0000","last_undegraded":"2024-09-17T08:26:10.606949+0000","last_fullsized":"2024-09-17T08:26:10.606949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:29.653071+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.653071+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026402420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":354,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607620+0000","last_change":"2024-09-17T08:25:15.032652+0000","last_active":"2024-09-17T08:26:10.607620+0000","last_peered":"2024-09-17T08:26:10.607620+0000","last_clean":"2024-09-17T08:26:10.607620+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:26:10.607620+0000","last_undegraded":"2024-09-17T08:26:10.607620+0000","last_fullsized":"2024-09-17T08:26:10.607620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:15.032525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:25:15.032525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.914352+0000","scrub_duration":0.013962988000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":557,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-17T08:26:06.520538+0000","last_change":"2024-09-17T08:25:23.783490+0000","last_active":"2024-09-17T08:26:06.520538+0000","last_peered":"2024-09-17T08:26:06.520538+0000","last_clean":"2024-09-17T08:26:06.520538+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:26:06.520538+0000","last_undegraded":"2024-09-17T08:26:06.520538+0000","last_fullsized":"2024-09-17T08:26:06.520538+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:23.783353+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.783353+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.268781+0000","scrub_duration":0.0064248070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":342,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607606+0000","last_change":"2024-09-17T08:23:30.540527+0000","last_active":"2024-09-17T08:26:10.607606+0000","last_peered":"2024-09-17T08:26:10.607606+0000","last_clean":"2024-09-17T08:26:10.607606+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:26:10.607606+0000","last_undegraded":"2024-09-17T08:26:10.607606+0000","last_fullsized":"2024-09-17T08:26:10.607606+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:30.540408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:30.540408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023462510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":262,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.514488+0000","last_change":"2024-09-17T08:24:15.898072+0000","last_active":"2024-09-17T08:26:04.514488+0000","last_peered":"2024-09-17T08:26:04.514488+0000","last_clean":"2024-09-17T08:26:04.514488+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:26:04.514488+0000","last_undegraded":"2024-09-17T08:26:04.514488+0000","last_fullsized":"2024-09-17T08:26:04.514488+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:24:15.897984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.897984+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018990789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":229,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520614+0000","last_change":"2024-09-17T08:24:13.873686+0000","last_active":"2024-09-17T08:26:04.520614+0000","last_peered":"2024-09-17T08:26:04.520614+0000","last_clean":"2024-09-17T08:26:04.520614+0000","last_became_active":"2024-09-17T08:23:18.671502+0000","last_became_peered":"2024-09-17T08:23:18.671502+0000","last_unstale":"2024-09-17T08:26:04.520614+0000","last_undegraded":"2024-09-17T08:26:04.520614+0000","last_fullsized":"2024-09-17T08:26:04.520614+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.873601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.873601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016156409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":354,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521048+0000","last_change":"2024-09-17T08:24:10.894915+0000","last_active":"2024-09-17T08:26:04.521048+0000","last_peered":"2024-09-17T08:26:04.521048+0000","last_clean":"2024-09-17T08:26:04.521048+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:26:04.521048+0000","last_undegraded":"2024-09-17T08:26:04.521048+0000","last_fullsized":"2024-09-17T08:26:04.521048+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.894828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.894828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001103185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":366,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606950+0000","last_change":"2024-09-17T08:24:08.507584+0000","last_active":"2024-09-17T08:26:10.606950+0000","last_peered":"2024-09-17T08:26:10.606950+0000","last_clean":"2024-09-17T08:26:10.606950+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:26:10.606950+0000","last_undegraded":"2024-09-17T08:26:10.606950+0000","last_fullsized":"2024-09-17T08:26:10.606950+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:08.507505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:08.507505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014868570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":426,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607129+0000","last_change":"2024-09-17T08:25:33.258559+0000","last_active":"2024-09-17T08:26:10.607129+0000","last_peered":"2024-09-17T08:26:10.607129+0000","last_clean":"2024-09-17T08:26:10.607129+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:26:10.607129+0000","last_undegraded":"2024-09-17T08:26:10.607129+0000","last_fullsized":"2024-09-17T08:26:10.607129+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.258469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.258469+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.181972+0000","scrub_duration":0.0016288769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510195+0000","last_change":"2024-09-17T08:25:30.191078+0000","last_active":"2024-09-17T08:26:04.510195+0000","last_peered":"2024-09-17T08:26:04.510195+0000","last_clean":"2024-09-17T08:26:04.510195+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:26:04.510195+0000","last_undegraded":"2024-09-17T08:26:04.510195+0000","last_fullsized":"2024-09-17T08:26:04.510195+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:30.190970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:30.190970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.517112+0000","scrub_duration":0.0016272669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":219,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.514804+0000","last_change":"2024-09-17T08:23:27.084326+0000","last_active":"2024-09-17T08:26:04.514804+0000","last_peered":"2024-09-17T08:26:04.514804+0000","last_clean":"2024-09-17T08:26:04.514804+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:26:04.514804+0000","last_undegraded":"2024-09-17T08:26:04.514804+0000","last_fullsized":"2024-09-17T08:26:04.514804+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.084153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.084153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013939082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":334,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520248+0000","last_change":"2024-09-17T08:24:07.897133+0000","last_active":"2024-09-17T08:26:04.520248+0000","last_peered":"2024-09-17T08:26:04.520248+0000","last_clean":"2024-09-17T08:26:04.520248+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:26:04.520248+0000","last_undegraded":"2024-09-17T08:26:04.520248+0000","last_fullsized":"2024-09-17T08:26:04.520248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:07.897050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:07.897050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015097719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":66,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606903+0000","last_change":"2024-09-17T08:24:26.616991+0000","last_active":"2024-09-17T08:26:10.606903+0000","last_peered":"2024-09-17T08:26:10.606903+0000","last_clean":"2024-09-17T08:26:10.606903+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:26:10.606903+0000","last_undegraded":"2024-09-17T08:26:10.606903+0000","last_fullsized":"2024-09-17T08:26:10.606903+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:24:26.616854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.616854+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002706263,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":227,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.605356+0000","last_change":"2024-09-17T08:25:24.143568+0000","last_active":"2024-09-17T08:26:10.605356+0000","last_peered":"2024-09-17T08:26:10.605356+0000","last_clean":"2024-09-17T08:26:10.605356+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:26:10.605356+0000","last_undegraded":"2024-09-17T08:26:10.605356+0000","last_fullsized":"2024-09-17T08:26:10.605356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:24.143382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:24.143382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:26.119340+0000","scrub_duration":0.0085248119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.610383+0000","last_change":"2024-09-17T08:25:41.337008+0000","last_active":"2024-09-17T08:26:10.610383+0000","last_peered":"2024-09-17T08:26:10.610383+0000","last_clean":"2024-09-17T08:26:10.610383+0000","last_became_active":"2024-09-17T08:23:18.671271+0000","last_became_peered":"2024-09-17T08:23:18.671271+0000","last_unstale":"2024-09-17T08:26:10.610383+0000","last_undegraded":"2024-09-17T08:26:10.610383+0000","last_fullsized":"2024-09-17T08:26:10.610383+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.336771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.336771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:01.147261+0000","scrub_duration":0.004352692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":362,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608165+0000","last_change":"2024-09-17T08:25:26.873461+0000","last_active":"2024-09-17T08:26:10.608165+0000","last_peered":"2024-09-17T08:26:10.608165+0000","last_clean":"2024-09-17T08:26:10.608165+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:26:10.608165+0000","last_undegraded":"2024-09-17T08:26:10.608165+0000","last_fullsized":"2024-09-17T08:26:10.608165+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:25:26.873378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.470812+0000","scrub_duration":0.0028675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":369,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520439+0000","last_change":"2024-09-17T08:24:21.848684+0000","last_active":"2024-09-17T08:26:04.520439+0000","last_peered":"2024-09-17T08:26:04.520439+0000","last_clean":"2024-09-17T08:26:04.520439+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:26:04.520439+0000","last_undegraded":"2024-09-17T08:26:04.520439+0000","last_fullsized":"2024-09-17T08:26:04.520439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:21.848510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.848510+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030240649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":199,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599251+0000","last_change":"2024-09-17T08:24:17.791131+0000","last_active":"2024-09-17T08:26:10.599251+0000","last_peered":"2024-09-17T08:26:10.599251+0000","last_clean":"2024-09-17T08:26:10.599251+0000","last_became_active":"2024-09-17T08:23:18.669127+0000","last_became_peered":"2024-09-17T08:23:18.669127+0000","last_unstale":"2024-09-17T08:26:10.599251+0000","last_undegraded":"2024-09-17T08:26:10.599251+0000","last_fullsized":"2024-09-17T08:26:10.599251+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:24:17.791034+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.791034+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019880050000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":349,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606291+0000","last_change":"2024-09-17T08:24:22.630555+0000","last_active":"2024-09-17T08:26:10.606291+0000","last_peered":"2024-09-17T08:26:10.606291+0000","last_clean":"2024-09-17T08:26:10.606291+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:26:10.606291+0000","last_undegraded":"2024-09-17T08:26:10.606291+0000","last_fullsized":"2024-09-17T08:26:10.606291+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:22.630455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:22.630455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020116380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":334,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520926+0000","last_change":"2024-09-17T08:24:29.919412+0000","last_active":"2024-09-17T08:26:04.520926+0000","last_peered":"2024-09-17T08:26:04.520926+0000","last_clean":"2024-09-17T08:26:04.520926+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:26:04.520926+0000","last_undegraded":"2024-09-17T08:26:04.520926+0000","last_fullsized":"2024-09-17T08:26:04.520926+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:29.919294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.919294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048696069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608540+0000","last_change":"2024-09-17T08:25:22.892701+0000","last_active":"2024-09-17T08:26:10.608540+0000","last_peered":"2024-09-17T08:26:10.608540+0000","last_clean":"2024-09-17T08:26:10.608540+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:26:10.608540+0000","last_undegraded":"2024-09-17T08:26:10.608540+0000","last_fullsized":"2024-09-17T08:26:10.608540+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:25:22.892571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.892571+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:32.568284+0000","scrub_duration":0.0024767180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":254,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.160161+0000","last_change":"2024-09-17T08:26:11.160161+0000","last_active":"2024-09-17T08:26:11.160161+0000","last_peered":"2024-09-17T08:26:11.160161+0000","last_clean":"2024-09-17T08:26:11.160161+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:26:11.160161+0000","last_undegraded":"2024-09-17T08:26:11.160161+0000","last_fullsized":"2024-09-17T08:26:11.160161+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.159902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.159902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.996888+0000","scrub_duration":0.011043470999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":384,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.612123+0000","last_change":"2024-09-17T08:24:19.731683+0000","last_active":"2024-09-17T08:26:10.612123+0000","last_peered":"2024-09-17T08:26:10.612123+0000","last_clean":"2024-09-17T08:26:10.612123+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:26:10.612123+0000","last_undegraded":"2024-09-17T08:26:10.612123+0000","last_fullsized":"2024-09-17T08:26:10.612123+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:19.731549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.731549+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011256499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":468,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607858+0000","last_change":"2024-09-17T08:23:18.424749+0000","last_active":"2024-09-17T08:26:10.607858+0000","last_peered":"2024-09-17T08:26:10.607858+0000","last_clean":"2024-09-17T08:26:10.607858+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:26:10.607858+0000","last_undegraded":"2024-09-17T08:26:10.607858+0000","last_fullsized":"2024-09-17T08:26:10.607858+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:23:18.424605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:18.424605+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087369089999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.509930+0000","last_change":"2024-09-17T08:23:18.949410+0000","last_active":"2024-09-17T08:26:04.509930+0000","last_peered":"2024-09-17T08:26:04.509930+0000","last_clean":"2024-09-17T08:26:04.509930+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:26:04.509930+0000","last_undegraded":"2024-09-17T08:26:04.509930+0000","last_fullsized":"2024-09-17T08:26:04.509930+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:18.949212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:18.949212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077693420000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":340,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520742+0000","last_change":"2024-09-17T08:24:04.885841+0000","last_active":"2024-09-17T08:26:04.520742+0000","last_peered":"2024-09-17T08:26:04.520742+0000","last_clean":"2024-09-17T08:26:04.520742+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:26:04.520742+0000","last_undegraded":"2024-09-17T08:26:04.520742+0000","last_fullsized":"2024-09-17T08:26:04.520742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:04.885747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:24:04.885747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001511376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":154,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510858+0000","last_change":"2024-09-17T08:23:18.676582+0000","last_active":"2024-09-17T08:26:04.510858+0000","last_peered":"2024-09-17T08:26:04.510858+0000","last_clean":"2024-09-17T08:26:04.510858+0000","last_became_active":"2024-09-17T08:23:18.676257+0000","last_became_peered":"2024-09-17T08:23:18.676257+0000","last_unstale":"2024-09-17T08:26:04.510858+0000","last_undegraded":"2024-09-17T08:26:04.510858+0000","last_fullsized":"2024-09-17T08:26:04.510858+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:22:39.435291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:22:39.435291+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021957669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":369,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.612044+0000","last_change":"2024-09-17T08:23:27.862145+0000","last_active":"2024-09-17T08:26:10.612044+0000","last_peered":"2024-09-17T08:26:10.612044+0000","last_clean":"2024-09-17T08:26:10.612044+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:26:10.612044+0000","last_undegraded":"2024-09-17T08:26:10.612044+0000","last_fullsized":"2024-09-17T08:26:10.612044+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.862032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.862032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029577050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":339,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608585+0000","last_change":"2024-09-17T08:25:19.293077+0000","last_active":"2024-09-17T08:26:10.608585+0000","last_peered":"2024-09-17T08:26:10.608585+0000","last_clean":"2024-09-17T08:26:10.608585+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:26:10.608585+0000","last_undegraded":"2024-09-17T08:26:10.608585+0000","last_fullsized":"2024-09-17T08:26:10.608585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:19.292858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.292858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:25.321340+0000","scrub_duration":0.0013986980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":295,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607520+0000","last_change":"2024-09-17T08:23:36.477221+0000","last_active":"2024-09-17T08:26:10.607520+0000","last_peered":"2024-09-17T08:26:10.607520+0000","last_clean":"2024-09-17T08:26:10.607520+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:26:10.607520+0000","last_undegraded":"2024-09-17T08:26:10.607520+0000","last_fullsized":"2024-09-17T08:26:10.607520+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:36.477146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:23:36.477146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":460,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.600272+0000","last_change":"2024-09-17T08:24:23.759585+0000","last_active":"2024-09-17T08:26:10.600272+0000","last_peered":"2024-09-17T08:26:10.600272+0000","last_clean":"2024-09-17T08:26:10.600272+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:26:10.600272+0000","last_undegraded":"2024-09-17T08:26:10.600272+0000","last_fullsized":"2024-09-17T08:26:10.600272+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:24:23.759435+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071267730000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":230,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.510067+0000","last_change":"2024-09-17T08:25:34.089169+0000","last_active":"2024-09-17T08:26:04.510067+0000","last_peered":"2024-09-17T08:26:04.510067+0000","last_clean":"2024-09-17T08:26:04.510067+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:26:04.510067+0000","last_undegraded":"2024-09-17T08:26:04.510067+0000","last_fullsized":"2024-09-17T08:26:04.510067+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.089010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:53.698566+0000","scrub_duration":0.0037643389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":324,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521204+0000","last_change":"2024-09-17T08:23:25.793139+0000","last_active":"2024-09-17T08:26:04.521204+0000","last_peered":"2024-09-17T08:26:04.521204+0000","last_clean":"2024-09-17T08:26:04.521204+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:26:04.521204+0000","last_undegraded":"2024-09-17T08:26:04.521204+0000","last_fullsized":"2024-09-17T08:26:04.521204+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.793012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.793012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072178959999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":351,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.521210+0000","last_change":"2024-09-17T08:25:38.883969+0000","last_active":"2024-09-17T08:26:04.521210+0000","last_peered":"2024-09-17T08:26:04.521210+0000","last_clean":"2024-09-17T08:26:04.521210+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:26:04.521210+0000","last_undegraded":"2024-09-17T08:26:04.521210+0000","last_fullsized":"2024-09-17T08:26:04.521210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.883814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.883814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.222351+0000","scrub_duration":0.012313661,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":412,"reported_epoch":322,"state":"active+clean","last_fresh":"2024-09-17T08:26:04.520820+0000","last_change":"2024-09-17T08:24:02.955324+0000","last_active":"2024-09-17T08:26:04.520820+0000","last_peered":"2024-09-17T08:26:04.520820+0000","last_clean":"2024-09-17T08:26:04.520820+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:26:04.520820+0000","last_undegraded":"2024-09-17T08:26:04.520820+0000","last_fullsized":"2024-09-17T08:26:04.520820+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:24:02.955231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:02.955231+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022957339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":355,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607849+0000","last_change":"2024-09-17T08:23:31.575041+0000","last_active":"2024-09-17T08:26:10.607849+0000","last_peered":"2024-09-17T08:26:10.607849+0000","last_clean":"2024-09-17T08:26:10.607849+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:26:10.607849+0000","last_undegraded":"2024-09-17T08:26:10.607849+0000","last_fullsized":"2024-09-17T08:26:10.607849+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:31.574717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:31.574717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019245600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":6891241496,"num_objects":105153,"num_object_clones":0,"num_object_copies":315459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159907,"num_write_kb":8481857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20519596032,"data_stored":20519583816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87307,"ondisk_log_size":87307,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":636,"num_read_kb":631,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070665,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2236480,"kb_used_data":1668020,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":91508672,"statfs":{"total":95995035648,"available":93704880128,"internally_reserved":0,"allocated":1708052480,"data_stored":1707718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":582089163},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.363,"15min":1.4370000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":5.3869999999999996,"5min":13.446,"15min":62.018000000000001},"last":2.8530000000000002},{"interface":"front","average":{"1min":2.0249999999999999,"5min":1.476,"15min":1.546},"min":{"1min":0.22,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":9.4169999999999998,"5min":13.417,"15min":61.896999999999998},"last":0.317}]},{"osd":1,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.7190000000000001,"15min":1.49},"min":{"1min":0.23999999999999999,"5min":0.23899999999999999,"15min":0.23100000000000001},"max":{"1min":5.8369999999999997,"5min":44.917999999999999,"15min":62.067},"last":2.7400000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.944,"15min":1.4810000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":5.5179999999999998,"5min":44.996000000000002,"15min":62.194000000000003},"last":1.246}]},{"osd":2,"last update":"Tue Sep 17 08:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.98799999999999999,"15min":0.89800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6379999999999999,"5min":4.4169999999999998,"15min":4.4169999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.92400000000000004,"15min":0.68000000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.849,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":2.4510000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":4.8920000000000003,"5min":2.6949999999999998,"15min":1.627},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":74.376000000000005,"5min":74.376000000000005,"15min":74.376000000000005},"last":2.548},{"interface":"front","average":{"1min":4.681,"5min":2.7610000000000001,"15min":1.73},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":75.183000000000007,"5min":75.183000000000007,"15min":75.183000000000007},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.97599999999999998,"15min":0.71699999999999997},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.962,"5min":4.391,"15min":5.335},"last":1.274},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73699999999999999,"15min":0.61199999999999999},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":1.845,"5min":3.798,"15min":5.9669999999999996},"last":2.5779999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.88900000000000001,"15min":0.76600000000000001},"min":{"1min":0.27500000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.8959999999999999,"5min":8.6400000000000006,"15min":16.808},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.88300000000000001,"15min":0.753},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":1.9359999999999999,"5min":4.4889999999999999,"15min":11.842000000000001},"last":0.35099999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.95199999999999996,"15min":0.78800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":2.0680000000000001,"5min":6.6509999999999998,"15min":7.2359999999999998},"last":2.4199999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.66500000000000004},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.246},"max":{"1min":3.1819999999999999,"5min":3.3849999999999998,"15min":5.9400000000000004},"last":2.508}]}]},{"osd":6,"up_from":50,"seq":214748365006,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3387812,"kb_used_data":2923548,"kb_used_omap":1,"kb_used_meta":464254,"kb_avail":90357340,"statfs":{"total":95995035648,"available":92525916160,"internally_reserved":0,"allocated":2993713152,"data_stored":2993395228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":475396551},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":3.9220000000000002,"5min":1.6930000000000001,"15min":2.3599999999999999},"min":{"1min":0.24299999999999999,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":55.625,"5min":55.625,"15min":81.007000000000005},"last":0.76800000000000002},{"interface":"front","average":{"1min":4.274,"5min":1.819,"15min":2.4649999999999999},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":55.597000000000001,"5min":55.597000000000001,"15min":80.310000000000002},"last":1.6679999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.494,"15min":1.627},"min":{"1min":0.27900000000000003,"5min":0.26500000000000001,"15min":0.22500000000000001},"max":{"1min":3.7029999999999998,"5min":42.618000000000002,"15min":80.813000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.4690000000000001,"15min":1.6759999999999999},"min":{"1min":0.29499999999999998,"5min":0.251,"15min":0.24299999999999999},"max":{"1min":3.0670000000000002,"5min":42.790999999999997,"15min":80.281000000000006},"last":1.5740000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":1.8720000000000001,"15min":1.0389999999999999},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.9009999999999998,"5min":70.451999999999998,"15min":70.451999999999998},"last":2.8559999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.865,"15min":0.97499999999999998},"min":{"1min":0.29999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.9180000000000001,"5min":70.488,"15min":70.488},"last":2.8889999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2050000000000001,"15min":1.9399999999999999},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.1020000000000003,"5min":9.9309999999999992,"15min":80.427000000000007},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.278,"5min":1.1080000000000001,"15min":1.8700000000000001},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26000000000000001},"max":{"1min":3.0449999999999999,"5min":11.859,"15min":80.379999999999995},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66500000000000004,"15min":0.63500000000000001},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":4.6219999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.69499999999999995,"15min":0.58599999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5219999999999998,"5min":3.585,"15min":5.0330000000000004},"last":0.35299999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.84199999999999997,"15min":0.70499999999999996},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.245},"max":{"1min":3.3740000000000001,"5min":5.5659999999999998,"15min":6.1859999999999999},"last":0.37},{"interface":"front","average":{"1min":0.625,"5min":0.73099999999999998,"15min":0.61399999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.9580000000000002,"5min":3.464,"15min":4.9969999999999999},"last":0.38200000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":2.919,"5min":2.919,"15min":4.774},"last":0.314},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.70599999999999996,"15min":0.625},"min":{"1min":0.33600000000000002,"5min":0.32200000000000001,"15min":0.28999999999999998},"max":{"1min":0.77200000000000002,"5min":3.5590000000000002,"15min":5.0599999999999996},"last":0.40000000000000002}]}]},{"osd":5,"up_from":41,"seq":176093659347,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2644332,"kb_used_data":2071780,"kb_used_omap":1,"kb_used_meta":572542,"kb_avail":91100820,"statfs":{"total":95995035648,"available":93287239680,"internally_reserved":0,"allocated":2121502720,"data_stored":2121184239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":586283462},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":7.4660000000000002,"5min":3.0750000000000002,"15min":2.1040000000000001},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.20300000000000001},"max":{"1min":64.218999999999994,"5min":65.941000000000003,"15min":65.941000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":7.7880000000000003,"5min":3.2050000000000001,"15min":2.0990000000000002},"min":{"1min":0.316,"5min":0.23000000000000001,"15min":0.22800000000000001},"max":{"1min":65.405000000000001,"5min":66.135000000000005,"15min":66.135000000000005},"last":0.443}]},{"osd":1,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":2.2549999999999999,"15min":1.3759999999999999},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":3.1309999999999998,"5min":66.981999999999999,"15min":66.981999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":2.3559999999999999,"15min":1.4430000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":2.9129999999999998,"5min":66.316000000000003,"15min":66.316000000000003},"last":0.379}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":2.2280000000000002,"5min":3.0339999999999998,"15min":1.498},"min":{"1min":0.28199999999999997,"5min":0.221,"15min":0.221},"max":{"1min":22.245000000000001,"5min":131.727,"15min":131.727},"last":0.39900000000000002},{"interface":"front","average":{"1min":2.3679999999999999,"5min":3.1850000000000001,"15min":1.7989999999999999},"min":{"1min":0.313,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.146000000000001,"5min":131.50399999999999,"15min":131.50399999999999},"last":0.307}]},{"osd":3,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":2.1469999999999998,"15min":1.718},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.253},"max":{"1min":3.4260000000000002,"5min":67.676000000000002,"15min":75.286000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":2.177,"15min":1.6819999999999999},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27100000000000002},"max":{"1min":5.4870000000000001,"5min":67.691999999999993,"15min":77.206999999999994},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.72199999999999998,"15min":0.76500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":3.3460000000000001,"5min":4.7569999999999997,"15min":42.421999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.626,"15min":0.73899999999999999},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.24399999999999999},"max":{"1min":3.395,"5min":3.395,"15min":42.191000000000003},"last":0.42999999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.8120000000000003,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":4.4000000000000004,"5min":4.9189999999999996,"15min":5.4400000000000004},"last":2.9100000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:25:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.76300000000000001,"15min":0.72899999999999998},"min":{"1min":0.36299999999999999,"5min":0.29799999999999999,"15min":0.25700000000000001},"max":{"1min":1.385,"5min":4.9409999999999998,"15min":7.5270000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.82199999999999995,"15min":0.70399999999999996},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.26900000000000002},"max":{"1min":3.5019999999999998,"5min":4.8140000000000001,"15min":4.8369999999999997},"last":0.34899999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953688,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3820232,"kb_used_data":3325620,"kb_used_omap":1,"kb_used_meta":494590,"kb_avail":89924920,"statfs":{"total":95995035648,"available":92083118080,"internally_reserved":0,"allocated":3405434880,"data_stored":3405100668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":506460614},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.93700000000000006,"15min":1.0109999999999999},"min":{"1min":0.254,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":3.4849999999999999,"5min":4.5229999999999997,"15min":23.786000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":0.96699999999999997,"15min":1.056},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":3.7450000000000001,"5min":6.2939999999999996,"15min":23.245000000000001},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.2609999999999999,"15min":1.1619999999999999},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":5.5839999999999996,"5min":9.3789999999999996,"15min":23.135000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.2410000000000001,"15min":1.1699999999999999},"min":{"1min":0.39300000000000002,"5min":0.28100000000000003,"15min":0.24099999999999999},"max":{"1min":3.52,"5min":9.0719999999999992,"15min":23.434999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.78600000000000003,"15min":0.63600000000000001},"min":{"1min":0.29599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6499999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":2.3159999999999998,"5min":1.2929999999999999,"15min":1.9430000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":28.138000000000002,"5min":28.138000000000002,"15min":154.137},"last":0.41499999999999998},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.403,"15min":1.8859999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.26700000000000002},"max":{"1min":28.117000000000001,"5min":28.117000000000001,"15min":153.54400000000001},"last":0.48299999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.66700000000000004,"15min":0.61599999999999999},"min":{"1min":0.28000000000000003,"5min":0.249,"15min":0.23000000000000001},"max":{"1min":3.5880000000000001,"5min":3.5880000000000001,"15min":6.2329999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.73099999999999998,"15min":0.65800000000000003},"min":{"1min":0.29599999999999999,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":1.474,"5min":4.0229999999999997,"15min":4.5300000000000002},"last":0.439}]},{"osd":6,"last update":"Tue Sep 17 08:25:45 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.60399999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":2.5529999999999999,"5min":3.0110000000000001,"15min":4.827},"last":0.378},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":3.6659999999999999,"5min":3.6659999999999999,"15min":6.0010000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":2.1989999999999998,"5min":4.0259999999999998,"15min":4.875},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.70499999999999996},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.6499999999999999,"5min":2.7949999999999999,"15min":4.6950000000000003},"last":0.5}]}]},{"osd":3,"up_from":27,"seq":115964117211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2801824,"kb_used_data":2189080,"kb_used_omap":1,"kb_used_meta":612734,"kb_avail":90943328,"statfs":{"total":95995035648,"available":93125967872,"internally_reserved":0,"allocated":2241617920,"data_stored":2241287121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":627440072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:36 2024","interfaces":[{"interface":"back","average":{"1min":8.9649999999999999,"5min":3.7069999999999999,"15min":2.2309999999999999},"min":{"1min":0.54600000000000004,"5min":0.156,"15min":0.156},"max":{"1min":130.40299999999999,"5min":130.40299999999999,"15min":130.40299999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":8.2680000000000007,"5min":3.0899999999999999,"15min":2.0230000000000001},"min":{"1min":0.33800000000000002,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":130.70099999999999,"5min":130.70099999999999,"15min":130.70099999999999},"last":0.68999999999999995}]},{"osd":1,"last update":"Tue Sep 17 08:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":2.153,"15min":1.6919999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.186},"max":{"1min":3.5529999999999999,"5min":71.459999999999994,"15min":71.459999999999994},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.278,"5min":2.29,"15min":1.706},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.188},"max":{"1min":3.718,"5min":71.438999999999993,"15min":71.438999999999993},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.657,"15min":1.268},"min":{"1min":0.253,"5min":0.215,"15min":0.215},"max":{"1min":3.968,"5min":60.423999999999999,"15min":60.423999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.8919999999999999,"15min":1.7250000000000001},"min":{"1min":0.35299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.8959999999999999,"5min":60.368000000000002,"15min":60.368000000000002},"last":5.6070000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.306,"15min":1.3420000000000001},"min":{"1min":0.35199999999999998,"5min":0.28499999999999998,"15min":0.252},"max":{"1min":3.7589999999999999,"5min":5.5460000000000003,"15min":35.841000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.194,"15min":1.274},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":3.9889999999999999,"5min":7.5999999999999996,"15min":35.857999999999997},"last":0.45200000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:25:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.228},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.29799999999999999},"max":{"1min":3.7850000000000001,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":2.4860000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.415,"15min":1.359},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.29499999999999998},"max":{"1min":2.4649999999999999,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.35699999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:25:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.127,"15min":1.179},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":4.2409999999999997,"5min":7.6669999999999998,"15min":10.945},"last":0.5},{"interface":"front","average":{"1min":1.26,"5min":1.536,"15min":1.4430000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":4.4829999999999997,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.38400000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":2.9420000000000002,"15min":1.8169999999999999},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":4.4720000000000004,"5min":163.006,"15min":163.006},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.456,"5min":3.089,"15min":1.8420000000000001},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.28999999999999998},"max":{"1min":3.8079999999999998,"5min":162.83699999999999,"15min":162.83699999999999},"last":2.3919999999999999}]}]},{"osd":2,"up_from":227,"seq":974957576263,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2740708,"kb_used_data":2532380,"kb_used_omap":4,"kb_used_meta":208251,"kb_avail":91004444,"statfs":{"total":95995035648,"available":93188550656,"internally_reserved":0,"allocated":2593157120,"data_stored":2592838975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":213249699},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":2.2759999999999998,"5min":1.327,"15min":1.091},"min":{"1min":0.313,"5min":0.188,"15min":0.188},"max":{"1min":5.6399999999999997,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.9249999999999998,"5min":1.3280000000000001,"15min":1.419},"min":{"1min":0.23300000000000001,"5min":0.184,"15min":0.184},"max":{"1min":9.1940000000000008,"5min":9.1940000000000008,"15min":9.1940000000000008},"last":2.0550000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.8899999999999999,"15min":1.45},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.6950000000000003,"5min":41.405000000000001,"15min":41.405000000000001},"last":2.2450000000000001},{"interface":"front","average":{"1min":1.7490000000000001,"5min":1.8660000000000001,"15min":1.47},"min":{"1min":0.30499999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":5.617,"5min":41.25,"15min":41.25},"last":1.679}]},{"osd":3,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":2.5019999999999998,"5min":1.7749999999999999,"15min":1.542},"min":{"1min":0.33800000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":6.9720000000000004,"5min":44.185000000000002,"15min":44.185000000000002},"last":2.0169999999999999},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7430000000000001,"15min":1.4610000000000001},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.9109999999999996,"5min":43.994,"15min":43.994},"last":2.2200000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.0409999999999999,"15min":0.89700000000000002},"min":{"1min":0.29799999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":3.5649999999999999},{"interface":"front","average":{"1min":1.389,"5min":0.92500000000000004,"15min":0.85499999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.6749999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":4.4699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":0.97499999999999998,"15min":0.95199999999999996},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.2240000000000002,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":2.1360000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":0.995,"15min":0.94499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.9770000000000003,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":2.0819999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":0.98899999999999999,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.317,"15min":0.317},"max":{"1min":4.359,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":3.6640000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.0549999999999999,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.5739999999999998,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":2.157}]},{"osd":7,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.7240000000000002,"5min":4.7240000000000002,"15min":4.7240000000000002},"last":2.1000000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.0680000000000001,"15min":0.93300000000000005},"min":{"1min":0.433,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.2770000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":3.7000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1919464,"kb_used_data":1296864,"kb_used_omap":1,"kb_used_meta":622590,"kb_avail":91825688,"statfs":{"total":95995035648,"available":94029504512,"internally_reserved":0,"allocated":1327988736,"data_stored":1327650335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":637532614},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":3.3559999999999999,"5min":1.6719999999999999,"15min":1.6930000000000001},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.155},"max":{"1min":27.670999999999999,"5min":27.670999999999999,"15min":31.585999999999999},"last":2.0830000000000002},{"interface":"front","average":{"1min":2.6320000000000001,"5min":1.8109999999999999,"15min":1.802},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.17199999999999999},"max":{"1min":26.120999999999999,"5min":26.120999999999999,"15min":35.807000000000002},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.5649999999999999,"15min":1.05},"min":{"1min":0.33900000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":26.297999999999998,"5min":26.297999999999998,"15min":26.297999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":2.266,"5min":1.4930000000000001,"15min":1.4870000000000001},"min":{"1min":0.22900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":26.602,"5min":26.602,"15min":26.602},"last":2.3290000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.5589999999999999,"15min":1.383},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.184},"max":{"1min":2.8490000000000002,"5min":5.2629999999999999,"15min":8.0559999999999992},"last":1.367},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.7589999999999999,"15min":1.5740000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.189},"max":{"1min":2.944,"5min":22.984999999999999,"15min":22.984999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.3009999999999999,"15min":1.2769999999999999},"min":{"1min":0.35299999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":2.3199999999999998,"5min":16.884,"15min":16.884},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.1870000000000001,"15min":1.173},"min":{"1min":0.39200000000000002,"5min":0.30499999999999999,"15min":0.25700000000000001},"max":{"1min":2.2480000000000002,"5min":9.6890000000000001,"15min":10.208},"last":2.2480000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.212,"15min":1.139},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":1.8129999999999999,"5min":4.5,"15min":7.9139999999999997},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.3300000000000001,"15min":1.3320000000000001},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":3.206,"5min":16.853999999999999,"15min":16.853999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.389,"15min":1.254},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":2.7389999999999999,"5min":9.7159999999999993,"15min":10.253},"last":2.3069999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.262,"15min":1.24},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.27200000000000002},"max":{"1min":2.855,"5min":5.5469999999999997,"15min":7.9459999999999997},"last":1.341}]},{"osd":7,"last update":"Tue Sep 17 08:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.9379999999999999,"15min":1.522},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.33100000000000002},"max":{"1min":3.169,"5min":47.442,"15min":47.442},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.8100000000000001,"15min":1.419},"min":{"1min":0.442,"5min":0.39700000000000002,"15min":0.378},"max":{"1min":3.7759999999999998,"5min":47.475000000000001,"15min":47.475000000000001},"last":2.2789999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738600,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4731200,"kb_used_data":4158644,"kb_used_omap":1,"kb_used_meta":572542,"kb_avail":89013952,"statfs":{"total":95995035648,"available":91150286848,"internally_reserved":0,"allocated":4258451456,"data_stored":4258117634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":586283462},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":2.105,"15min":1.917},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":10.848000000000001,"5min":41.247,"15min":61.649000000000001},"last":1.635},{"interface":"front","average":{"1min":2.1749999999999998,"5min":2.387,"15min":2.125},"min":{"1min":0.182,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1669999999999998,"5min":40.436,"15min":61.281999999999996},"last":0.80900000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:25:33 2024","interfaces":[{"interface":"back","average":{"1min":2.1309999999999998,"5min":2.6030000000000002,"15min":1.623},"min":{"1min":0.39700000000000002,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":9.1189999999999998,"5min":99.120999999999995,"15min":99.120999999999995},"last":1.8600000000000001},{"interface":"front","average":{"1min":2.2959999999999998,"5min":2.75,"15min":1.506},"min":{"1min":0.27800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":5.274,"5min":99.256,"15min":99.256},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:25:46 2024","interfaces":[{"interface":"back","average":{"1min":2.1160000000000001,"5min":2.0299999999999998,"15min":2.069},"min":{"1min":0.39500000000000002,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":5.968,"5min":38.697000000000003,"15min":61.926000000000002},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.6659999999999999,"15min":1.819},"min":{"1min":0.35099999999999998,"5min":0.223,"15min":0.223},"max":{"1min":10.868,"5min":38.950000000000003,"15min":61.616},"last":1.671}]},{"osd":4,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.627,"15min":1.413},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26300000000000001},"max":{"1min":5.242,"5min":41.158000000000001,"15min":41.158000000000001},"last":0.755},{"interface":"front","average":{"1min":1.923,"5min":1.542,"15min":1.423},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":6.8920000000000003,"5min":38.978000000000002,"15min":38.978000000000002},"last":2.27}]},{"osd":5,"last update":"Tue Sep 17 08:25:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.454,"15min":1.2849999999999999},"min":{"1min":0.41899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":10.805999999999999,"5min":39.058,"15min":39.058},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.6839999999999999,"15min":1.4570000000000001},"min":{"1min":0.307,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.7590000000000003,"5min":43.485999999999997,"15min":43.485999999999997},"last":1.0329999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:25:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.3680000000000001,"15min":1.3009999999999999},"min":{"1min":0.377,"5min":0.317,"15min":0.27700000000000002},"max":{"1min":10.827999999999999,"5min":38.917999999999999,"15min":38.917999999999999},"last":1.71},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.7709999999999999,"15min":1.518},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.2400000000000002,"5min":41.189,"15min":41.189},"last":0.78400000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:25:12 2024","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5249999999999999,"15min":1.536},"min":{"1min":0.38,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":5.6769999999999996,"5min":39.045999999999999,"15min":39.045999999999999},"last":1.9179999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.351,"15min":1.4239999999999999},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.307},"max":{"1min":2.6099999999999999,"5min":39.082999999999998,"15min":39.082999999999998},"last":1.6919999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4241494016,"data_stored":4241489944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1317666816,"data_stored":1317666816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2589392896,"data_stored":2589392896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2222129152,"data_stored":2222129152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3380744192,"data_stored":3380740120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2106851328,"data_stored":2106851328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2957905920,"data_stored":2957901848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1703411712,"data_stored":1703411712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:26:20.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:26:20.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:20.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:20.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:20.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:20 smithi173 ceph-mon[33187]: 10.9 scrub ok 2024-09-17T08:26:20.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:20 smithi173 ceph-mon[33187]: pgmap v728: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:26:20.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:20.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:20.878 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:20.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:20.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:21.017 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:21.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:21.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:20 smithi079 ceph-mon[40649]: 10.9 scrub ok 2024-09-17T08:26:21.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:20 smithi079 ceph-mon[40649]: pgmap v728: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:26:21.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:20 smithi079 ceph-mon[29565]: 10.9 scrub ok 2024-09-17T08:26:21.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:20 smithi079 ceph-mon[29565]: pgmap v728: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-17T08:26:21.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:21.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:21.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:21.280 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:21.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:21.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:21.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:21.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.457 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:21.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:21.474 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:26:21.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:21.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:21.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:21.683 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:21.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:21 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:26:21.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:21 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:26:21.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:21 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:26:21.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:21 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:26:21.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:21.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:21.825 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:21.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:21.886 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:21.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:21.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:21.988 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:22.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.021 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:22.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:26:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:26:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:26:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:26:22.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:26:22.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:26:22.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:26:22.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:21 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:26:22.122 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:22.159 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:22.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:22.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:22.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:22.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:22.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:22.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:22.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:22.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:22.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:22.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:22.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:22.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:22.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:22.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:22.826 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:22.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:22.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:26:22.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[29565]: pgmap v729: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:26:22.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:22 smithi079 ceph-mon[40649]: pgmap v729: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:26:22.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:22.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:22.964 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:22.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:22.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:22 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:26:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:22 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:26:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:22 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:26:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:22 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:26:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:22 smithi173 ceph-mon[33187]: pgmap v729: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-17T08:26:23.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:23.099 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:23.103 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:23.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:23.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:23.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:23.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:23.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:23.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:23.285 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:23.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:23.379 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:23.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:23.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:23.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:23.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:23.731 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:23.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:23.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:23.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:23.882 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:26:23.883 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:23.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:23.888 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:23.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:23.942 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:23.942 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:23.984 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:26:23.985 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:26:23.986 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:26:23.986 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:26:23.986 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:26:23.986 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:23.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:24.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.031 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:24.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:24.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:24.165 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:24.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:24.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:24.304 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:24.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:24.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:24.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.451 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:24.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:24.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:24.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:24.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:24.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:26:24.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:24.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:24.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:24.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:24.866 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:24.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:24.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:24.881 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:24.952 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:24.969 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:26:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:24 smithi173 ceph-mon[33187]: pgmap v730: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 855 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:26:25.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:25.024 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:26:25.024 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:26:25.024 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:26:25.024 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:26:25.024 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[29565]: pgmap v730: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 855 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:26:25.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:26:25.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:26:25.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:24 smithi079 ceph-mon[40649]: pgmap v730: 148 pgs: 148 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 855 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:26:25.114 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:25.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:25.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:25.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:25.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:25.432 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.433 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:25.534 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:25.561 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:25.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:25.696 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:25.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:25.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:25.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:25.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:25 smithi173 ceph-mon[33187]: 10.a scrub starts 2024-09-17T08:26:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:25 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:26:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:25 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:26:25.976 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:25.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:25.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:26.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[29565]: 10.a scrub starts 2024-09-17T08:26:26.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:26:26.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:26:26.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[40649]: 10.a scrub starts 2024-09-17T08:26:26.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:26:26.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:25 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:26:26.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:26.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.120 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:26.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:26.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:26.253 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:26.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:26.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:26.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:26.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:26.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:26.540 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:26.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:26.643 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:26.687 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:26.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:26.727 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:26:26.747 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: pgmap v731: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: pgmap v731: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:26:26.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:26:26.749 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:26 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:26.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:26.819 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:26.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:26.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:26.963 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:26.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:26.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:26:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:26:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: pgmap v731: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-17T08:26:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:26:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:26:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:26 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:27.065 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:27.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:27.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:27.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:27.238 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:27.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:27.243 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:27.243 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:27.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:27.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:27.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:27.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.521 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:27.521 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:27.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:27.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.660 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:27.660 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:27.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:27.798 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:27.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: 10.a scrub ok 2024-09-17T08:26:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[40649]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: 10.a scrub ok 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:26:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:26:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:27.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:27 smithi079 ceph-mon[29565]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T08:26:27.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:27.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:27.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:27.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:27.950 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: 10.a scrub ok 2024-09-17T08:26:27.950 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:26:27.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:26:27.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:26:27.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:26:27.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3408552252' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:27.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:27 smithi173 ceph-mon[33187]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T08:26:27.988 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:26:28.049 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:28.098 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:28.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:28.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:28.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.246 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:28.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:28.302 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:28.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:28.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:28.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:28.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:28.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.518 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:28.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:28.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:28.664 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:28.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:28.723 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=sqlstore.transactions t=2024-09-17T08:26:28.634968307Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T08:26:28.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:28.810 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:28.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:28.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:28.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:26:28.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:26:28.932 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:26:28.932 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:26:28.957 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:26:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:26:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:28.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:28.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:28.999 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:26:28.999 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:26:28.999 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: pgmap v733: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: 2.5 deep-scrub starts 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[29565]: 2.5 deep-scrub ok 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:29.000 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:26:29.001 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:26:29.001 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: pgmap v733: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:29.001 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T08:26:29.001 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: 2.5 deep-scrub starts 2024-09-17T08:26:29.001 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:28 smithi079 ceph-mon[40649]: 2.5 deep-scrub ok 2024-09-17T08:26:29.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:26:29.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:26:29.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:29.073 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:26:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:26:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:26:29.084 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:26:29.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.102 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:29.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:29.114 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:26:29.114 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:26:29.149 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:26:29.149 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:26:29.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:29.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: pgmap v733: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: 2.5 deep-scrub starts 2024-09-17T08:26:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:28 smithi173 ceph-mon[33187]: 2.5 deep-scrub ok 2024-09-17T08:26:29.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:29.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:29.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:29.380 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:29.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:29.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:29.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:29.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:29.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:29.660 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:29.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:29.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:29.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:29.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:29.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:29 smithi173 ceph-mon[33187]: 10.8 scrub starts 2024-09-17T08:26:29.899 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3167010399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:29.899 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:29 smithi173 ceph-mon[33187]: from='client.15687 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:29.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:29.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:29.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:29.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:29.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:26:30.017 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[29565]: 10.8 scrub starts 2024-09-17T08:26:30.017 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3167010399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:30.017 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[29565]: from='client.15687 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:30.017 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[40649]: 10.8 scrub starts 2024-09-17T08:26:30.018 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3167010399' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:30.018 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:29 smithi079 ceph-mon[40649]: from='client.15687 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:30.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:30.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:30.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:30.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:181744 10.e e328/287 10:7112809c:::benchmark_data_smithi079_172150_object49212:head v 328'11408, mlcod=328'11404)", 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:30.215759+0000", 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00243349, 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024609829999999999, 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215759+0000", 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:30.220 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215759+0000", 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215759+0000", 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215765+0000", 2024-09-17T08:26:30.221 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5089999999999999e-06 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215779+0000", 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4358e-05 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215786+0000", 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6839999999999999e-06 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.222 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215828+0000", 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2199e-05 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.215849+0000", 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0981000000000001e-05 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.223 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:181743 10.0 e328/287 10:037c1a02:::benchmark_data_smithi079_172150_object49211:head v 328'11489, mlcod=328'11487)", 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:30.217511+0000", 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00068178599999999998, 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000768846, 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217511+0000", 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217511+0000", 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217510+0000", 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:30.225 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217517+0000", 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8249999999999999e-06 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217531+0000", 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3471e-05 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217537+0000", 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9199999999999998e-06 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.226 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217589+0000", 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1303e-05 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217611+0000", 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1855999999999999e-05 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.227 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:181746 10.b e328/287 10:d011f0fb:::benchmark_data_smithi079_172150_object49214:head v 328'11303, mlcod=328'11301)", 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:30.217551+0000", 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00064134700000000001, 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078132500000000003, 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217551+0000", 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.228 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217551+0000", 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:30.229 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217551+0000", 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217555+0000", 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0419999999999996e-06 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.230 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217562+0000", 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4609999999999996e-06 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217566+0000", 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.583e-06 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.231 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217642+0000", 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5921999999999996e-05 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:30.217662+0000", 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9644999999999999e-05 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:26:30.232 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 3 2024-09-17T08:26:30.233 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:30.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:30.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:30.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:30.373 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:30.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:30.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:30.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:30.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:30.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:30.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:30.704 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":734,"stamp":"2024-09-17T08:26:27.729456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5792405806,"num_objects":88573,"num_object_clones":0,"num_object_copies":265719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88573,"num_whiteouts":0,"num_read":1995,"num_read_kb":1841,"num_write":177578,"num_write_kb":8484316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":1775729,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88353,"ondisk_log_size":88353,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21155604,"kb_used_data":16956192,"kb_used_omap":15,"kb_used_meta":4198768,"kb_avail":728805612,"statfs":{"total":767960285184,"available":746296946688,"internally_reserved":0,"allocated":17363140608,"data_stored":17360630202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":4299539221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-600965120,"num_objects":-9170,"num_object_clones":0,"num_object_copies":-27510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9170,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9170,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.156768"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":337,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603056+0000","last_change":"2024-09-17T08:24:22.829264+0000","last_active":"2024-09-17T08:26:10.603056+0000","last_peered":"2024-09-17T08:26:10.603056+0000","last_clean":"2024-09-17T08:26:10.603056+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:26:10.603056+0000","last_undegraded":"2024-09-17T08:26:10.603056+0000","last_fullsized":"2024-09-17T08:26:10.603056+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:22.829138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:24:22.829138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002109297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.241237+0000","last_change":"2024-09-17T08:26:23.241237+0000","last_active":"2024-09-17T08:26:23.241237+0000","last_peered":"2024-09-17T08:26:23.241237+0000","last_clean":"2024-09-17T08:26:23.241237+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:26:23.241237+0000","last_undegraded":"2024-09-17T08:26:23.241237+0000","last_fullsized":"2024-09-17T08:26:23.241237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.241125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.241125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:49.606056+0000","scrub_duration":0.002185354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":414,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:21.027111+0000","last_change":"2024-09-17T08:26:21.027223+0000","last_active":"2024-09-17T08:26:21.027111+0000","last_peered":"2024-09-17T08:26:21.027111+0000","last_clean":"2024-09-17T08:26:21.027111+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:26:21.027111+0000","last_undegraded":"2024-09-17T08:26:21.027111+0000","last_fullsized":"2024-09-17T08:26:21.027111+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:26:21.027087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.027087+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:45.075711+0000","scrub_duration":0.004246901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":358,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610773+0000","last_change":"2024-09-17T08:23:33.566799+0000","last_active":"2024-09-17T08:26:11.610773+0000","last_peered":"2024-09-17T08:26:11.610773+0000","last_clean":"2024-09-17T08:26:11.610773+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:26:11.610773+0000","last_undegraded":"2024-09-17T08:26:11.610773+0000","last_fullsized":"2024-09-17T08:26:11.610773+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:33.566719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:33.566719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016610028999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":345,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606225+0000","last_change":"2024-09-17T08:25:12.813837+0000","last_active":"2024-09-17T08:26:10.606225+0000","last_peered":"2024-09-17T08:26:10.606225+0000","last_clean":"2024-09-17T08:26:10.606225+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:26:10.606225+0000","last_undegraded":"2024-09-17T08:26:10.606225+0000","last_fullsized":"2024-09-17T08:26:10.606225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.813704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.813704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.648729+0000","scrub_duration":0.0071490010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":471,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610786+0000","last_change":"2024-09-17T08:24:29.572754+0000","last_active":"2024-09-17T08:26:11.610786+0000","last_peered":"2024-09-17T08:26:11.610786+0000","last_clean":"2024-09-17T08:26:11.610786+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:26:11.610786+0000","last_undegraded":"2024-09-17T08:26:11.610786+0000","last_fullsized":"2024-09-17T08:26:11.610786+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:29.572400+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.572400+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041638179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.605303+0000","last_change":"2024-09-17T08:25:29.240062+0000","last_active":"2024-09-17T08:26:10.605303+0000","last_peered":"2024-09-17T08:26:10.605303+0000","last_clean":"2024-09-17T08:26:10.605303+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:26:10.605303+0000","last_undegraded":"2024-09-17T08:26:10.605303+0000","last_fullsized":"2024-09-17T08:26:10.605303+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.239407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.239407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.084448+0000","scrub_duration":0.0062858009999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610626+0000","last_change":"2024-09-17T08:25:28.285107+0000","last_active":"2024-09-17T08:26:11.610626+0000","last_peered":"2024-09-17T08:26:11.610626+0000","last_clean":"2024-09-17T08:26:11.610626+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:26:11.610626+0000","last_undegraded":"2024-09-17T08:26:11.610626+0000","last_fullsized":"2024-09-17T08:26:11.610626+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.284981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:28.287282+0000","scrub_duration":0.0042235850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":353,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607020+0000","last_change":"2024-09-17T08:25:14.860865+0000","last_active":"2024-09-17T08:26:10.607020+0000","last_peered":"2024-09-17T08:26:10.607020+0000","last_clean":"2024-09-17T08:26:10.607020+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:26:10.607020+0000","last_undegraded":"2024-09-17T08:26:10.607020+0000","last_fullsized":"2024-09-17T08:26:10.607020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.860746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.860746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.290009+0000","scrub_duration":0.0074367399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":387,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.611308+0000","last_change":"2024-09-17T08:24:16.615788+0000","last_active":"2024-09-17T08:26:11.611308+0000","last_peered":"2024-09-17T08:26:11.611308+0000","last_clean":"2024-09-17T08:26:11.611308+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:26:11.611308+0000","last_undegraded":"2024-09-17T08:26:11.611308+0000","last_fullsized":"2024-09-17T08:26:11.611308+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:24:16.615683+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:24:16.615683+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001772476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":378,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608242+0000","last_change":"2024-09-17T08:24:32.685119+0000","last_active":"2024-09-17T08:26:10.608242+0000","last_peered":"2024-09-17T08:26:10.608242+0000","last_clean":"2024-09-17T08:26:10.608242+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:26:10.608242+0000","last_undegraded":"2024-09-17T08:26:10.608242+0000","last_fullsized":"2024-09-17T08:26:10.608242+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:32.684474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.684474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038163649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603976+0000","last_change":"2024-09-17T08:24:25.823602+0000","last_active":"2024-09-17T08:26:10.603976+0000","last_peered":"2024-09-17T08:26:10.603976+0000","last_clean":"2024-09-17T08:26:10.603976+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:26:10.603976+0000","last_undegraded":"2024-09-17T08:26:10.603976+0000","last_fullsized":"2024-09-17T08:26:10.603976+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:25.823472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.823472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":379,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.668330+0000","last_change":"2024-09-17T08:26:11.668739+0000","last_active":"2024-09-17T08:26:11.668330+0000","last_peered":"2024-09-17T08:26:11.668330+0000","last_clean":"2024-09-17T08:26:11.668330+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:26:11.668330+0000","last_undegraded":"2024-09-17T08:26:11.668330+0000","last_fullsized":"2024-09-17T08:26:11.668330+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.668207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.668207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.803189+0000","scrub_duration":0.0090449410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":382,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610849+0000","last_change":"2024-09-17T08:25:22.208990+0000","last_active":"2024-09-17T08:26:11.610849+0000","last_peered":"2024-09-17T08:26:11.610849+0000","last_clean":"2024-09-17T08:26:11.610849+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:26:11.610849+0000","last_undegraded":"2024-09-17T08:26:11.610849+0000","last_fullsized":"2024-09-17T08:26:11.610849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:25:22.208698+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.208698+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":254,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.620632+0000","last_change":"2024-09-17T08:24:27.797180+0000","last_active":"2024-09-17T08:26:11.620632+0000","last_peered":"2024-09-17T08:26:11.620632+0000","last_clean":"2024-09-17T08:26:11.620632+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:26:11.620632+0000","last_undegraded":"2024-09-17T08:26:11.620632+0000","last_fullsized":"2024-09-17T08:26:11.620632+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:27.796880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:27.796880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049344319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.618426+0000","last_change":"2024-09-17T08:24:20.752436+0000","last_active":"2024-09-17T08:26:11.618426+0000","last_peered":"2024-09-17T08:26:11.618426+0000","last_clean":"2024-09-17T08:26:11.618426+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:26:11.618426+0000","last_undegraded":"2024-09-17T08:26:11.618426+0000","last_fullsized":"2024-09-17T08:26:11.618426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:20.752313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:20.752313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027283759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"326'11164","reported_seq":11363,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.667531+0000","last_change":"2024-09-17T08:25:28.636224+0000","last_active":"2024-09-17T08:26:26.667531+0000","last_peered":"2024-09-17T08:26:26.667531+0000","last_clean":"2024-09-17T08:26:26.667531+0000","last_became_active":"2024-09-17T08:24:04.453791+0000","last_became_peered":"2024-09-17T08:24:04.453791+0000","last_unstale":"2024-09-17T08:26:26.667531+0000","last_undegraded":"2024-09-17T08:26:26.667531+0000","last_fullsized":"2024-09-17T08:26:26.667531+0000","mapping_epoch":287,"log_start":"303'4700","ondisk_log_start":"303'4700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6965","last_scrub_stamp":"2024-09-17T08:25:28.635952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.635952+0000","objects_scrubbed":6875,"log_size":6464,"log_dups_size":0,"ondisk_log_size":6464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:52.900306+0000","scrub_duration":2.5130719930000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362414080,"num_objects":5530,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11164,"num_write_kb":534208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":478,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.611608+0000","last_change":"2024-09-17T08:24:17.608622+0000","last_active":"2024-09-17T08:26:11.611608+0000","last_peered":"2024-09-17T08:26:11.611608+0000","last_clean":"2024-09-17T08:26:11.611608+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:26:11.611608+0000","last_undegraded":"2024-09-17T08:26:11.611608+0000","last_fullsized":"2024-09-17T08:26:11.611608+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:24:17.608509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.608509+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025730330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":365,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606691+0000","last_change":"2024-09-17T08:25:21.870284+0000","last_active":"2024-09-17T08:26:10.606691+0000","last_peered":"2024-09-17T08:26:10.606691+0000","last_clean":"2024-09-17T08:26:10.606691+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:26:10.606691+0000","last_undegraded":"2024-09-17T08:26:10.606691+0000","last_fullsized":"2024-09-17T08:26:10.606691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:21.870174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:21.870174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.767809+0000","scrub_duration":0.0028963090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":244,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.325829+0000","last_change":"2024-09-17T08:26:22.325911+0000","last_active":"2024-09-17T08:26:22.325829+0000","last_peered":"2024-09-17T08:26:22.325829+0000","last_clean":"2024-09-17T08:26:22.325829+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:26:22.325829+0000","last_undegraded":"2024-09-17T08:26:22.325829+0000","last_fullsized":"2024-09-17T08:26:22.325829+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:22.325816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.325816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.578172+0000","scrub_duration":0.0015290900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":333,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599994+0000","last_change":"2024-09-17T08:25:22.077723+0000","last_active":"2024-09-17T08:26:10.599994+0000","last_peered":"2024-09-17T08:26:10.599994+0000","last_clean":"2024-09-17T08:26:10.599994+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:26:10.599994+0000","last_undegraded":"2024-09-17T08:26:10.599994+0000","last_fullsized":"2024-09-17T08:26:10.599994+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:22.077429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.077429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.599110+0000","scrub_duration":0.0073898080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"326'10655","reported_seq":10704,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.810468+0000","last_change":"2024-09-17T08:24:04.455688+0000","last_active":"2024-09-17T08:26:23.810468+0000","last_peered":"2024-09-17T08:26:23.810468+0000","last_clean":"2024-09-17T08:26:23.810468+0000","last_became_active":"2024-09-17T08:24:04.455271+0000","last_became_peered":"2024-09-17T08:24:04.455271+0000","last_unstale":"2024-09-17T08:26:23.810468+0000","last_undegraded":"2024-09-17T08:26:23.810468+0000","last_fullsized":"2024-09-17T08:26:23.810468+0000","mapping_epoch":287,"log_start":"306'5200","ondisk_log_start":"306'5200","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366542848,"num_objects":5593,"num_object_clones":0,"num_object_copies":16779,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10655,"num_write_kb":519936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":384,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610338+0000","last_change":"2024-09-17T08:25:23.209427+0000","last_active":"2024-09-17T08:26:11.610338+0000","last_peered":"2024-09-17T08:26:11.610338+0000","last_clean":"2024-09-17T08:26:11.610338+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:26:11.610338+0000","last_undegraded":"2024-09-17T08:26:11.610338+0000","last_fullsized":"2024-09-17T08:26:11.610338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:25:23.209287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.209287+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.695831+0000","scrub_duration":0.0047713479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":234,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.611671+0000","last_change":"2024-09-17T08:25:33.132550+0000","last_active":"2024-09-17T08:26:11.611671+0000","last_peered":"2024-09-17T08:26:11.611671+0000","last_clean":"2024-09-17T08:26:11.611671+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:26:11.611671+0000","last_undegraded":"2024-09-17T08:26:11.611671+0000","last_fullsized":"2024-09-17T08:26:11.611671+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:33.131878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.131878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:58.566704+0000","scrub_duration":0.008861519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":510,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:18.037292+0000","last_change":"2024-09-17T08:25:11.078189+0000","last_active":"2024-09-17T08:26:18.037292+0000","last_peered":"2024-09-17T08:26:18.037292+0000","last_clean":"2024-09-17T08:26:18.037292+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:26:18.037292+0000","last_undegraded":"2024-09-17T08:26:18.037292+0000","last_fullsized":"2024-09-17T08:26:18.037292+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:11.078003+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.078003+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:39.754804+0000","scrub_duration":0.0057575789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":362,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610367+0000","last_change":"2024-09-17T08:23:25.473468+0000","last_active":"2024-09-17T08:26:11.610367+0000","last_peered":"2024-09-17T08:26:11.610367+0000","last_clean":"2024-09-17T08:26:11.610367+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:26:11.610367+0000","last_undegraded":"2024-09-17T08:26:11.610367+0000","last_fullsized":"2024-09-17T08:26:11.610367+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:25.473322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:23:25.473322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005682973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"327'11386","reported_seq":11436,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.713219+0000","last_change":"2024-09-17T08:24:04.459231+0000","last_active":"2024-09-17T08:26:27.713219+0000","last_peered":"2024-09-17T08:26:27.713219+0000","last_clean":"2024-09-17T08:26:27.713219+0000","last_became_active":"2024-09-17T08:24:04.458930+0000","last_became_peered":"2024-09-17T08:24:04.458930+0000","last_unstale":"2024-09-17T08:26:27.713219+0000","last_undegraded":"2024-09-17T08:26:27.713219+0000","last_fullsized":"2024-09-17T08:26:27.713219+0000","mapping_epoch":287,"log_start":"310'6400","ondisk_log_start":"310'6400","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4986,"log_dups_size":0,"ondisk_log_size":4986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357957632,"num_objects":5462,"num_object_clones":0,"num_object_copies":16386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11386,"num_write_kb":539136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":235,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599176+0000","last_change":"2024-09-17T08:25:39.196006+0000","last_active":"2024-09-17T08:26:10.599176+0000","last_peered":"2024-09-17T08:26:10.599176+0000","last_clean":"2024-09-17T08:26:10.599176+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:26:10.599176+0000","last_undegraded":"2024-09-17T08:26:10.599176+0000","last_fullsized":"2024-09-17T08:26:10.599176+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:39.195587+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.195587+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:00.706411+0000","scrub_duration":0.0050104839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":234,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609672+0000","last_change":"2024-09-17T08:25:12.167258+0000","last_active":"2024-09-17T08:26:11.609672+0000","last_peered":"2024-09-17T08:26:11.609672+0000","last_clean":"2024-09-17T08:26:11.609672+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:26:11.609672+0000","last_undegraded":"2024-09-17T08:26:11.609672+0000","last_fullsized":"2024-09-17T08:26:11.609672+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.167132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.167132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020782719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":548,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.028396+0000","last_change":"2024-09-17T08:24:14.547197+0000","last_active":"2024-09-17T08:26:23.028396+0000","last_peered":"2024-09-17T08:26:23.028396+0000","last_clean":"2024-09-17T08:26:23.028396+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:26:23.028396+0000","last_undegraded":"2024-09-17T08:26:23.028396+0000","last_fullsized":"2024-09-17T08:26:23.028396+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:24:14.547105+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:24:14.547105+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013424540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604722+0000","last_change":"2024-09-17T08:25:28.215581+0000","last_active":"2024-09-17T08:26:10.604722+0000","last_peered":"2024-09-17T08:26:10.604722+0000","last_clean":"2024-09-17T08:26:10.604722+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:26:10.604722+0000","last_undegraded":"2024-09-17T08:26:10.604722+0000","last_fullsized":"2024-09-17T08:26:10.604722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.215457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.215457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.036848+0000","scrub_duration":0.0070925789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"326'10981","reported_seq":11030,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.808617+0000","last_change":"2024-09-17T08:24:04.455364+0000","last_active":"2024-09-17T08:26:23.808617+0000","last_peered":"2024-09-17T08:26:23.808617+0000","last_clean":"2024-09-17T08:26:23.808617+0000","last_became_active":"2024-09-17T08:24:04.455098+0000","last_became_peered":"2024-09-17T08:24:04.455098+0000","last_unstale":"2024-09-17T08:26:23.808617+0000","last_undegraded":"2024-09-17T08:26:23.808617+0000","last_fullsized":"2024-09-17T08:26:23.808617+0000","mapping_epoch":287,"log_start":"307'5500","ondisk_log_start":"307'5500","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367591424,"num_objects":5609,"num_object_clones":0,"num_object_copies":16827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10981,"num_write_kb":530880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":422,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610384+0000","last_change":"2024-09-17T08:24:25.581898+0000","last_active":"2024-09-17T08:26:11.610384+0000","last_peered":"2024-09-17T08:26:11.610384+0000","last_clean":"2024-09-17T08:26:11.610384+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:26:11.610384+0000","last_undegraded":"2024-09-17T08:26:11.610384+0000","last_fullsized":"2024-09-17T08:26:11.610384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:24:25.581740+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.581740+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004957894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":372,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.687114+0000","last_change":"2024-09-17T08:26:12.688090+0000","last_active":"2024-09-17T08:26:12.687114+0000","last_peered":"2024-09-17T08:26:12.687114+0000","last_clean":"2024-09-17T08:26:12.687114+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:26:12.687114+0000","last_undegraded":"2024-09-17T08:26:12.687114+0000","last_fullsized":"2024-09-17T08:26:12.687114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:12.687091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.687091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.608122+0000","scrub_duration":0.006317067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":245,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:17.696982+0000","last_change":"2024-09-17T08:24:30.522675+0000","last_active":"2024-09-17T08:26:17.696982+0000","last_peered":"2024-09-17T08:26:17.696982+0000","last_clean":"2024-09-17T08:26:17.696982+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:26:17.696982+0000","last_undegraded":"2024-09-17T08:26:17.696982+0000","last_fullsized":"2024-09-17T08:26:17.696982+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.522581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001427191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":338,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604474+0000","last_change":"2024-09-17T08:25:40.909763+0000","last_active":"2024-09-17T08:26:10.604474+0000","last_peered":"2024-09-17T08:26:10.604474+0000","last_clean":"2024-09-17T08:26:10.604474+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:26:10.604474+0000","last_undegraded":"2024-09-17T08:26:10.604474+0000","last_fullsized":"2024-09-17T08:26:10.604474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.909493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.909493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:51.177881+0000","scrub_duration":0.014995481999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"326'10854","reported_seq":10903,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.719525+0000","last_change":"2024-09-17T08:24:04.454563+0000","last_active":"2024-09-17T08:26:23.719525+0000","last_peered":"2024-09-17T08:26:23.719525+0000","last_clean":"2024-09-17T08:26:23.719525+0000","last_became_active":"2024-09-17T08:24:04.454206+0000","last_became_peered":"2024-09-17T08:24:04.454206+0000","last_unstale":"2024-09-17T08:26:23.719525+0000","last_undegraded":"2024-09-17T08:26:23.719525+0000","last_fullsized":"2024-09-17T08:26:23.719525+0000","mapping_epoch":287,"log_start":"307'5600","ondisk_log_start":"307'5600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5254,"log_dups_size":0,"ondisk_log_size":5254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370409472,"num_objects":5652,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":528192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":385,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609785+0000","last_change":"2024-09-17T08:25:12.405766+0000","last_active":"2024-09-17T08:26:11.609785+0000","last_peered":"2024-09-17T08:26:11.609785+0000","last_clean":"2024-09-17T08:26:11.609785+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:26:11.609785+0000","last_undegraded":"2024-09-17T08:26:11.609785+0000","last_fullsized":"2024-09-17T08:26:11.609785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:25:12.405574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.405574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00623603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":237,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607730+0000","last_change":"2024-09-17T08:24:31.660718+0000","last_active":"2024-09-17T08:26:10.607730+0000","last_peered":"2024-09-17T08:26:10.607730+0000","last_clean":"2024-09-17T08:26:10.607730+0000","last_became_active":"2024-09-17T08:23:18.669266+0000","last_became_peered":"2024-09-17T08:23:18.669266+0000","last_unstale":"2024-09-17T08:26:10.607730+0000","last_undegraded":"2024-09-17T08:26:10.607730+0000","last_fullsized":"2024-09-17T08:26:10.607730+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:24:31.660592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00371011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":579,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:18.027772+0000","last_change":"2024-09-17T08:24:12.872168+0000","last_active":"2024-09-17T08:26:18.027772+0000","last_peered":"2024-09-17T08:26:18.027772+0000","last_clean":"2024-09-17T08:26:18.027772+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:26:18.027772+0000","last_undegraded":"2024-09-17T08:26:18.027772+0000","last_fullsized":"2024-09-17T08:26:18.027772+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:24:12.872090+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.872090+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606483+0000","last_change":"2024-09-17T08:25:29.822800+0000","last_active":"2024-09-17T08:26:10.606483+0000","last_peered":"2024-09-17T08:26:10.606483+0000","last_clean":"2024-09-17T08:26:10.606483+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:26:10.606483+0000","last_undegraded":"2024-09-17T08:26:10.606483+0000","last_fullsized":"2024-09-17T08:26:10.606483+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.822689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.822689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.095715+0000","scrub_duration":0.0085561849999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"327'11222","reported_seq":11272,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.697286+0000","last_change":"2024-09-17T08:24:04.453118+0000","last_active":"2024-09-17T08:26:27.697286+0000","last_peered":"2024-09-17T08:26:27.697286+0000","last_clean":"2024-09-17T08:26:27.697286+0000","last_became_active":"2024-09-17T08:24:04.452780+0000","last_became_peered":"2024-09-17T08:24:04.452780+0000","last_unstale":"2024-09-17T08:26:27.697286+0000","last_undegraded":"2024-09-17T08:26:27.697286+0000","last_fullsized":"2024-09-17T08:26:27.697286+0000","mapping_epoch":287,"log_start":"310'6300","ondisk_log_start":"310'6300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4922,"log_dups_size":0,"ondisk_log_size":4922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358744064,"num_objects":5474,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11222,"num_write_kb":534272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":393,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.602991+0000","last_change":"2024-09-17T08:24:03.927221+0000","last_active":"2024-09-17T08:26:10.602991+0000","last_peered":"2024-09-17T08:26:10.602991+0000","last_clean":"2024-09-17T08:26:10.602991+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:26:10.602991+0000","last_undegraded":"2024-09-17T08:26:10.602991+0000","last_fullsized":"2024-09-17T08:26:10.602991+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:24:03.927109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.927109+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026636759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":234,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.612531+0000","last_change":"2024-09-17T08:25:31.165549+0000","last_active":"2024-09-17T08:26:11.612531+0000","last_peered":"2024-09-17T08:26:11.612531+0000","last_clean":"2024-09-17T08:26:11.612531+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:26:11.612531+0000","last_undegraded":"2024-09-17T08:26:11.612531+0000","last_fullsized":"2024-09-17T08:26:11.612531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:31.165432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:25:31.165432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:36.847226+0000","scrub_duration":0.001729558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":352,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.149322+0000","last_change":"2024-09-17T08:26:23.149322+0000","last_active":"2024-09-17T08:26:23.149322+0000","last_peered":"2024-09-17T08:26:23.149322+0000","last_clean":"2024-09-17T08:26:23.149322+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:26:23.149322+0000","last_undegraded":"2024-09-17T08:26:23.149322+0000","last_fullsized":"2024-09-17T08:26:23.149322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.149070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.149070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.469461+0000","scrub_duration":0.0050165310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":365,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610773+0000","last_change":"2024-09-17T08:25:39.349334+0000","last_active":"2024-09-17T08:26:11.610773+0000","last_peered":"2024-09-17T08:26:11.610773+0000","last_clean":"2024-09-17T08:26:11.610773+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:26:11.610773+0000","last_undegraded":"2024-09-17T08:26:11.610773+0000","last_fullsized":"2024-09-17T08:26:11.610773+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:39.349120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.349120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.525815+0000","scrub_duration":0.005278652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"327'11131","reported_seq":11182,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.694986+0000","last_change":"2024-09-17T08:24:04.456347+0000","last_active":"2024-09-17T08:26:27.694986+0000","last_peered":"2024-09-17T08:26:27.694986+0000","last_clean":"2024-09-17T08:26:27.694986+0000","last_became_active":"2024-09-17T08:24:04.456039+0000","last_became_peered":"2024-09-17T08:24:04.456039+0000","last_unstale":"2024-09-17T08:26:27.694986+0000","last_undegraded":"2024-09-17T08:26:27.694986+0000","last_fullsized":"2024-09-17T08:26:27.694986+0000","mapping_epoch":287,"log_start":"310'6200","ondisk_log_start":"310'6200","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351272984,"num_objects":5361,"num_object_clones":0,"num_object_copies":16083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11131,"num_write_kb":527681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":416,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:12.052601+0000","last_change":"2024-09-17T08:26:12.052601+0000","last_active":"2024-09-17T08:26:12.052601+0000","last_peered":"2024-09-17T08:26:12.052601+0000","last_clean":"2024-09-17T08:26:12.052601+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:26:12.052601+0000","last_undegraded":"2024-09-17T08:26:12.052601+0000","last_fullsized":"2024-09-17T08:26:12.052601+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:26:12.052367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.052367+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.321528+0000","scrub_duration":0.0032600939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"65'1","reported_seq":359,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.607090+0000","last_change":"2024-09-17T08:24:23.674368+0000","last_active":"2024-09-17T08:26:10.607090+0000","last_peered":"2024-09-17T08:26:10.607090+0000","last_clean":"2024-09-17T08:26:10.607090+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:26:10.607090+0000","last_undegraded":"2024-09-17T08:26:10.607090+0000","last_fullsized":"2024-09-17T08:26:10.607090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'1","last_scrub_stamp":"2024-09-17T08:24:23.674249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:23.674249+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041183280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":308,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.621248+0000","last_change":"2024-09-17T08:24:15.691704+0000","last_active":"2024-09-17T08:26:11.621248+0000","last_peered":"2024-09-17T08:26:11.621248+0000","last_clean":"2024-09-17T08:26:11.621248+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:26:11.621248+0000","last_undegraded":"2024-09-17T08:26:11.621248+0000","last_fullsized":"2024-09-17T08:26:11.621248+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:15.691622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.691622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034962180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":249,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.605270+0000","last_change":"2024-09-17T08:24:12.919326+0000","last_active":"2024-09-17T08:26:10.605270+0000","last_peered":"2024-09-17T08:26:10.605270+0000","last_clean":"2024-09-17T08:26:10.605270+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:26:10.605270+0000","last_undegraded":"2024-09-17T08:26:10.605270+0000","last_fullsized":"2024-09-17T08:26:10.605270+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:12.919247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.919247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013913059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"326'10871","reported_seq":11078,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.819782+0000","last_change":"2024-09-17T08:25:36.457603+0000","last_active":"2024-09-17T08:26:22.819782+0000","last_peered":"2024-09-17T08:26:22.819782+0000","last_clean":"2024-09-17T08:26:22.819782+0000","last_became_active":"2024-09-17T08:24:04.458943+0000","last_became_peered":"2024-09-17T08:24:04.458943+0000","last_unstale":"2024-09-17T08:26:22.819782+0000","last_undegraded":"2024-09-17T08:26:22.819782+0000","last_fullsized":"2024-09-17T08:26:22.819782+0000","mapping_epoch":287,"log_start":"308'5800","ondisk_log_start":"308'5800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'7478","last_scrub_stamp":"2024-09-17T08:25:36.457326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:36.457326+0000","objects_scrubbed":7398,"log_size":5071,"log_dups_size":0,"ondisk_log_size":5071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:50.045951+0000","scrub_duration":2.6174414289999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380960768,"num_objects":5813,"num_object_clones":0,"num_object_copies":17439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":533888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":266,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603842+0000","last_change":"2024-09-17T08:24:30.840463+0000","last_active":"2024-09-17T08:26:10.603842+0000","last_peered":"2024-09-17T08:26:10.603842+0000","last_clean":"2024-09-17T08:26:10.603842+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:26:10.603842+0000","last_undegraded":"2024-09-17T08:26:10.603842+0000","last_fullsized":"2024-09-17T08:26:10.603842+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:24:30.840289+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.840289+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056047780000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604265+0000","last_change":"2024-09-17T08:25:14.210636+0000","last_active":"2024-09-17T08:26:10.604265+0000","last_peered":"2024-09-17T08:26:10.604265+0000","last_clean":"2024-09-17T08:26:10.604265+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:26:10.604265+0000","last_undegraded":"2024-09-17T08:26:10.604265+0000","last_fullsized":"2024-09-17T08:26:10.604265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.210528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.210528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.818087+0000","scrub_duration":0.0019364110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":570,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.029497+0000","last_change":"2024-09-17T08:26:22.193822+0000","last_active":"2024-09-17T08:26:23.029497+0000","last_peered":"2024-09-17T08:26:23.029497+0000","last_clean":"2024-09-17T08:26:23.029497+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:26:23.029497+0000","last_undegraded":"2024-09-17T08:26:23.029497+0000","last_fullsized":"2024-09-17T08:26:23.029497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:26:22.193701+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.193701+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.638128+0000","scrub_duration":0.0030149489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.618314+0000","last_change":"2024-09-17T08:25:37.001934+0000","last_active":"2024-09-17T08:26:11.618314+0000","last_peered":"2024-09-17T08:26:11.618314+0000","last_clean":"2024-09-17T08:26:11.618314+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:26:11.618314+0000","last_undegraded":"2024-09-17T08:26:11.618314+0000","last_fullsized":"2024-09-17T08:26:11.618314+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:37.001438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.001438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:54.154447+0000","scrub_duration":0.0093143449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"327'10991","reported_seq":11162,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.329463+0000","last_change":"2024-09-17T08:26:25.894830+0000","last_active":"2024-09-17T08:26:27.329463+0000","last_peered":"2024-09-17T08:26:27.329463+0000","last_clean":"2024-09-17T08:26:27.329463+0000","last_became_active":"2024-09-17T08:24:04.457614+0000","last_became_peered":"2024-09-17T08:24:04.457614+0000","last_unstale":"2024-09-17T08:26:27.329463+0000","last_undegraded":"2024-09-17T08:26:27.329463+0000","last_fullsized":"2024-09-17T08:26:27.329463+0000","mapping_epoch":287,"log_start":"308'5700","ondisk_log_start":"308'5700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10898","last_scrub_stamp":"2024-09-17T08:26:25.894587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.894587+0000","objects_scrubbed":5529,"log_size":5291,"log_dups_size":0,"ondisk_log_size":5291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.232827+0000","scrub_duration":1.670279176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352518144,"num_objects":5379,"num_object_clones":0,"num_object_copies":16137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10991,"num_write_kb":523840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":426,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:18.029727+0000","last_change":"2024-09-17T08:24:03.009933+0000","last_active":"2024-09-17T08:26:18.029727+0000","last_peered":"2024-09-17T08:26:18.029727+0000","last_clean":"2024-09-17T08:26:18.029727+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:26:18.029727+0000","last_undegraded":"2024-09-17T08:26:18.029727+0000","last_fullsized":"2024-09-17T08:26:18.029727+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:24:03.009831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.009831+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044852599999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":234,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604000+0000","last_change":"2024-09-17T08:23:28.072844+0000","last_active":"2024-09-17T08:26:10.604000+0000","last_peered":"2024-09-17T08:26:10.604000+0000","last_clean":"2024-09-17T08:26:10.604000+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:26:10.604000+0000","last_undegraded":"2024-09-17T08:26:10.604000+0000","last_fullsized":"2024-09-17T08:26:10.604000+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:28.072608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:23:28.072608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079140149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":370,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.623056+0000","last_change":"2024-09-17T08:24:17.685902+0000","last_active":"2024-09-17T08:26:11.623056+0000","last_peered":"2024-09-17T08:26:11.623056+0000","last_clean":"2024-09-17T08:26:11.623056+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:26:11.623056+0000","last_undegraded":"2024-09-17T08:26:11.623056+0000","last_fullsized":"2024-09-17T08:26:11.623056+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:17.685819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.685819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015007639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":349,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604219+0000","last_change":"2024-09-17T08:25:29.848581+0000","last_active":"2024-09-17T08:26:10.604219+0000","last_peered":"2024-09-17T08:26:10.604219+0000","last_clean":"2024-09-17T08:26:10.604219+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:26:10.604219+0000","last_undegraded":"2024-09-17T08:26:10.604219+0000","last_fullsized":"2024-09-17T08:26:10.604219+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.848449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.848449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.490576+0000","scrub_duration":0.0072796190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"326'11017","reported_seq":11203,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.662997+0000","last_change":"2024-09-17T08:26:18.800526+0000","last_active":"2024-09-17T08:26:26.662997+0000","last_peered":"2024-09-17T08:26:26.662997+0000","last_clean":"2024-09-17T08:26:26.662997+0000","last_became_active":"2024-09-17T08:24:04.453753+0000","last_became_peered":"2024-09-17T08:24:04.453753+0000","last_unstale":"2024-09-17T08:26:26.662997+0000","last_undegraded":"2024-09-17T08:26:26.662997+0000","last_fullsized":"2024-09-17T08:26:26.662997+0000","mapping_epoch":287,"log_start":"303'4600","ondisk_log_start":"303'4600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10467","last_scrub_stamp":"2024-09-17T08:26:18.800219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.800219+0000","objects_scrubbed":6104,"log_size":6417,"log_dups_size":0,"ondisk_log_size":6417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.431424+0000","scrub_duration":2.2200420680000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359464960,"num_objects":5485,"num_object_clones":0,"num_object_copies":16455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11017,"num_write_kb":528064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":252,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.612163+0000","last_change":"2024-09-17T08:25:19.161609+0000","last_active":"2024-09-17T08:26:11.612163+0000","last_peered":"2024-09-17T08:26:11.612163+0000","last_clean":"2024-09-17T08:26:11.612163+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:26:11.612163+0000","last_undegraded":"2024-09-17T08:26:11.612163+0000","last_fullsized":"2024-09-17T08:26:11.612163+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:25:19.161454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.161454+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:24.995624+0000","scrub_duration":0.015453119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":905,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.587224+0000","last_change":"2024-09-17T08:24:11.910215+0000","last_active":"2024-09-17T08:26:22.587224+0000","last_peered":"2024-09-17T08:26:22.587224+0000","last_clean":"2024-09-17T08:26:22.587224+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:26:22.587224+0000","last_undegraded":"2024-09-17T08:26:22.587224+0000","last_fullsized":"2024-09-17T08:26:22.587224+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:24:11.910131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:11.910131+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001356576,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":651,"num_read_kb":646,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":339,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:15.566110+0000","last_change":"2024-09-17T08:26:15.566206+0000","last_active":"2024-09-17T08:26:15.566110+0000","last_peered":"2024-09-17T08:26:15.566110+0000","last_clean":"2024-09-17T08:26:15.566110+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:26:15.566110+0000","last_undegraded":"2024-09-17T08:26:15.566110+0000","last_fullsized":"2024-09-17T08:26:15.566110+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.566090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.566090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:37.464727+0000","scrub_duration":0.0094002110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":336,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.600955+0000","last_change":"2024-09-17T08:24:21.765981+0000","last_active":"2024-09-17T08:26:10.600955+0000","last_peered":"2024-09-17T08:26:10.600955+0000","last_clean":"2024-09-17T08:26:10.600955+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:26:10.600955+0000","last_undegraded":"2024-09-17T08:26:10.600955+0000","last_fullsized":"2024-09-17T08:26:10.600955+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:21.765863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.765863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":359,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606747+0000","last_change":"2024-09-17T08:25:18.912338+0000","last_active":"2024-09-17T08:26:10.606747+0000","last_peered":"2024-09-17T08:26:10.606747+0000","last_clean":"2024-09-17T08:26:10.606747+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:26:10.606747+0000","last_undegraded":"2024-09-17T08:26:10.606747+0000","last_fullsized":"2024-09-17T08:26:10.606747+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.912202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.912202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.360980+0000","scrub_duration":0.0058719879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"327'11224","reported_seq":11274,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.311118+0000","last_change":"2024-09-17T08:24:04.459213+0000","last_active":"2024-09-17T08:26:27.311118+0000","last_peered":"2024-09-17T08:26:27.311118+0000","last_clean":"2024-09-17T08:26:27.311118+0000","last_became_active":"2024-09-17T08:24:04.458862+0000","last_became_peered":"2024-09-17T08:24:04.458862+0000","last_unstale":"2024-09-17T08:26:27.311118+0000","last_undegraded":"2024-09-17T08:26:27.311118+0000","last_fullsized":"2024-09-17T08:26:27.311118+0000","mapping_epoch":287,"log_start":"309'5900","ondisk_log_start":"309'5900","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362807296,"num_objects":5536,"num_object_clones":0,"num_object_copies":16608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11224,"num_write_kb":536320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":472,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.624401+0000","last_change":"2024-09-17T08:25:37.983837+0000","last_active":"2024-09-17T08:26:11.624401+0000","last_peered":"2024-09-17T08:26:11.624401+0000","last_clean":"2024-09-17T08:26:11.624401+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:26:11.624401+0000","last_undegraded":"2024-09-17T08:26:11.624401+0000","last_fullsized":"2024-09-17T08:26:11.624401+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:25:37.983609+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.983609+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.256761+0000","scrub_duration":0.0081599319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":374,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.618391+0000","last_change":"2024-09-17T08:25:17.027645+0000","last_active":"2024-09-17T08:26:11.618391+0000","last_peered":"2024-09-17T08:26:11.618391+0000","last_clean":"2024-09-17T08:26:11.618391+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:26:11.618391+0000","last_undegraded":"2024-09-17T08:26:11.618391+0000","last_fullsized":"2024-09-17T08:26:11.618391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:25:17.027377+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.027377+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.587148+0000","scrub_duration":0.0088147999999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":384,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606741+0000","last_change":"2024-09-17T08:25:34.874991+0000","last_active":"2024-09-17T08:26:10.606741+0000","last_peered":"2024-09-17T08:26:10.606741+0000","last_clean":"2024-09-17T08:26:10.606741+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:26:10.606741+0000","last_undegraded":"2024-09-17T08:26:10.606741+0000","last_fullsized":"2024-09-17T08:26:10.606741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.874848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.874848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.051778+0000","scrub_duration":0.0082107689999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":245,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.359243+0000","last_change":"2024-09-17T08:26:23.359243+0000","last_active":"2024-09-17T08:26:23.359243+0000","last_peered":"2024-09-17T08:26:23.359243+0000","last_clean":"2024-09-17T08:26:23.359243+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:26:23.359243+0000","last_undegraded":"2024-09-17T08:26:23.359243+0000","last_fullsized":"2024-09-17T08:26:23.359243+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.359104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.359104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.887092+0000","scrub_duration":0.0035052579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":234,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604123+0000","last_change":"2024-09-17T08:24:28.859552+0000","last_active":"2024-09-17T08:26:10.604123+0000","last_peered":"2024-09-17T08:26:10.604123+0000","last_clean":"2024-09-17T08:26:10.604123+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:26:10.604123+0000","last_undegraded":"2024-09-17T08:26:10.604123+0000","last_fullsized":"2024-09-17T08:26:10.604123+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:28.859427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:28.859427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051469929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"326'10803","reported_seq":10851,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.810625+0000","last_change":"2024-09-17T08:24:04.456167+0000","last_active":"2024-09-17T08:26:23.810625+0000","last_peered":"2024-09-17T08:26:23.810625+0000","last_clean":"2024-09-17T08:26:23.810625+0000","last_became_active":"2024-09-17T08:24:04.455726+0000","last_became_peered":"2024-09-17T08:24:04.455726+0000","last_unstale":"2024-09-17T08:26:23.810625+0000","last_undegraded":"2024-09-17T08:26:23.810625+0000","last_fullsized":"2024-09-17T08:26:23.810625+0000","mapping_epoch":287,"log_start":"306'5300","ondisk_log_start":"306'5300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5503,"log_dups_size":0,"ondisk_log_size":5503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368902144,"num_objects":5629,"num_object_clones":0,"num_object_copies":16887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":525824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":356,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:25.062996+0000","last_change":"2024-09-17T08:26:25.062996+0000","last_active":"2024-09-17T08:26:25.062996+0000","last_peered":"2024-09-17T08:26:25.062996+0000","last_clean":"2024-09-17T08:26:25.062996+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:26:25.062996+0000","last_undegraded":"2024-09-17T08:26:25.062996+0000","last_fullsized":"2024-09-17T08:26:25.062996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:26:25.062850+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.062850+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.215968+0000","scrub_duration":0.0041814790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":351,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603920+0000","last_change":"2024-09-17T08:24:06.873073+0000","last_active":"2024-09-17T08:26:10.603920+0000","last_peered":"2024-09-17T08:26:10.603920+0000","last_clean":"2024-09-17T08:26:10.603920+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:26:10.603920+0000","last_undegraded":"2024-09-17T08:26:10.603920+0000","last_fullsized":"2024-09-17T08:26:10.603920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:24:06.872995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:06.872995+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019197509999999999,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598790+0000","last_change":"2024-09-17T08:25:42.221990+0000","last_active":"2024-09-17T08:26:10.598790+0000","last_peered":"2024-09-17T08:26:10.598790+0000","last_clean":"2024-09-17T08:26:10.598790+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:26:10.598790+0000","last_undegraded":"2024-09-17T08:26:10.598790+0000","last_fullsized":"2024-09-17T08:26:10.598790+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:42.221590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:25:42.221590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.243498+0000","scrub_duration":0.005773388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":616,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606887+0000","last_change":"2024-09-17T08:24:24.654508+0000","last_active":"2024-09-17T08:26:10.606887+0000","last_peered":"2024-09-17T08:26:10.606887+0000","last_clean":"2024-09-17T08:26:10.606887+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:26:10.606887+0000","last_undegraded":"2024-09-17T08:26:10.606887+0000","last_fullsized":"2024-09-17T08:26:10.606887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:24:24.654371+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014600804,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":365,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608627+0000","last_change":"2024-09-17T08:25:38.817905+0000","last_active":"2024-09-17T08:26:10.608627+0000","last_peered":"2024-09-17T08:26:10.608627+0000","last_clean":"2024-09-17T08:26:10.608627+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:26:10.608627+0000","last_undegraded":"2024-09-17T08:26:10.608627+0000","last_fullsized":"2024-09-17T08:26:10.608627+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.817546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:04.248082+0000","scrub_duration":0.002079291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":254,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.609130+0000","last_change":"2024-09-17T08:25:41.900666+0000","last_active":"2024-09-17T08:26:10.609130+0000","last_peered":"2024-09-17T08:26:10.609130+0000","last_clean":"2024-09-17T08:26:10.609130+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:26:10.609130+0000","last_undegraded":"2024-09-17T08:26:10.609130+0000","last_fullsized":"2024-09-17T08:26:10.609130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.900483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.900483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.112986+0000","scrub_duration":0.013601894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"327'11147","reported_seq":11195,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.344456+0000","last_change":"2024-09-17T08:24:04.458040+0000","last_active":"2024-09-17T08:26:27.344456+0000","last_peered":"2024-09-17T08:26:27.344456+0000","last_clean":"2024-09-17T08:26:27.344456+0000","last_became_active":"2024-09-17T08:24:04.457745+0000","last_became_peered":"2024-09-17T08:24:04.457745+0000","last_unstale":"2024-09-17T08:26:27.344456+0000","last_undegraded":"2024-09-17T08:26:27.344456+0000","last_fullsized":"2024-09-17T08:26:27.344456+0000","mapping_epoch":287,"log_start":"309'5800","ondisk_log_start":"309'5800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5347,"log_dups_size":0,"ondisk_log_size":5347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353697792,"num_objects":5397,"num_object_clones":0,"num_object_copies":16191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11147,"num_write_kb":529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":192,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604074+0000","last_change":"2024-09-17T08:25:35.125962+0000","last_active":"2024-09-17T08:26:10.604074+0000","last_peered":"2024-09-17T08:26:10.604074+0000","last_clean":"2024-09-17T08:26:10.604074+0000","last_became_active":"2024-09-17T08:23:18.670977+0000","last_became_peered":"2024-09-17T08:23:18.670977+0000","last_unstale":"2024-09-17T08:26:10.604074+0000","last_undegraded":"2024-09-17T08:26:10.604074+0000","last_fullsized":"2024-09-17T08:26:10.604074+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:25:35.125100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:35.125100+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.214432+0000","scrub_duration":0.0023912690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":317,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599244+0000","last_change":"2024-09-17T08:24:07.886685+0000","last_active":"2024-09-17T08:26:10.599244+0000","last_peered":"2024-09-17T08:26:10.599244+0000","last_clean":"2024-09-17T08:26:10.599244+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:26:10.599244+0000","last_undegraded":"2024-09-17T08:26:10.599244+0000","last_fullsized":"2024-09-17T08:26:10.599244+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:07.886609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:07.886609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001519378,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":334,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598716+0000","last_change":"2024-09-17T08:25:13.057707+0000","last_active":"2024-09-17T08:26:10.598716+0000","last_peered":"2024-09-17T08:26:10.598716+0000","last_clean":"2024-09-17T08:26:10.598716+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:26:10.598716+0000","last_undegraded":"2024-09-17T08:26:10.598716+0000","last_fullsized":"2024-09-17T08:26:10.598716+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:13.057534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.057534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.726370+0000","scrub_duration":0.011530043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":363,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:15.680589+0000","last_change":"2024-09-17T08:26:15.680685+0000","last_active":"2024-09-17T08:26:15.680589+0000","last_peered":"2024-09-17T08:26:15.680589+0000","last_clean":"2024-09-17T08:26:15.680589+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:26:15.680589+0000","last_undegraded":"2024-09-17T08:26:15.680589+0000","last_fullsized":"2024-09-17T08:26:15.680589+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.680565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.680565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:34.928434+0000","scrub_duration":0.0076375050000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"326'11111","reported_seq":11301,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.618303+0000","last_change":"2024-09-17T08:26:17.151968+0000","last_active":"2024-09-17T08:26:26.618303+0000","last_peered":"2024-09-17T08:26:26.618303+0000","last_clean":"2024-09-17T08:26:26.618303+0000","last_became_active":"2024-09-17T08:24:04.458851+0000","last_became_peered":"2024-09-17T08:24:04.458851+0000","last_unstale":"2024-09-17T08:26:26.618303+0000","last_undegraded":"2024-09-17T08:26:26.618303+0000","last_fullsized":"2024-09-17T08:26:26.618303+0000","mapping_epoch":287,"log_start":"303'4700","ondisk_log_start":"303'4700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10426","last_scrub_stamp":"2024-09-17T08:26:17.151829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.151829+0000","objects_scrubbed":6271,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.276228+0000","scrub_duration":2.6491713260000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360644608,"num_objects":5503,"num_object_clones":0,"num_object_copies":16509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11111,"num_write_kb":531648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":417,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.620497+0000","last_change":"2024-09-17T08:25:16.078092+0000","last_active":"2024-09-17T08:26:11.620497+0000","last_peered":"2024-09-17T08:26:11.620497+0000","last_clean":"2024-09-17T08:26:11.620497+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:26:11.620497+0000","last_undegraded":"2024-09-17T08:26:11.620497+0000","last_fullsized":"2024-09-17T08:26:11.620497+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:25:16.077960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:16.077960+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.021838+0000","scrub_duration":0.021317215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":244,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:17.341282+0000","last_change":"2024-09-17T08:26:17.341373+0000","last_active":"2024-09-17T08:26:17.341282+0000","last_peered":"2024-09-17T08:26:17.341282+0000","last_clean":"2024-09-17T08:26:17.341282+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:26:17.341282+0000","last_undegraded":"2024-09-17T08:26:17.341282+0000","last_fullsized":"2024-09-17T08:26:17.341282+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.341260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.341260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.774271+0000","scrub_duration":0.0033668219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":337,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598563+0000","last_change":"2024-09-17T08:24:26.864669+0000","last_active":"2024-09-17T08:26:10.598563+0000","last_peered":"2024-09-17T08:26:10.598563+0000","last_clean":"2024-09-17T08:26:10.598563+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:26:10.598563+0000","last_undegraded":"2024-09-17T08:26:10.598563+0000","last_fullsized":"2024-09-17T08:26:10.598563+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:26.864558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.864558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027307770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.618320+0000","last_change":"2024-09-17T08:25:40.906564+0000","last_active":"2024-09-17T08:26:11.618320+0000","last_peered":"2024-09-17T08:26:11.618320+0000","last_clean":"2024-09-17T08:26:11.618320+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:26:11.618320+0000","last_undegraded":"2024-09-17T08:26:11.618320+0000","last_fullsized":"2024-09-17T08:26:11.618320+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.906437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.906437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.936477+0000","scrub_duration":0.014127516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"327'11096","reported_seq":11146,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.342752+0000","last_change":"2024-09-17T08:24:04.458318+0000","last_active":"2024-09-17T08:26:27.342752+0000","last_peered":"2024-09-17T08:26:27.342752+0000","last_clean":"2024-09-17T08:26:27.342752+0000","last_became_active":"2024-09-17T08:24:04.458042+0000","last_became_peered":"2024-09-17T08:24:04.458042+0000","last_unstale":"2024-09-17T08:26:27.342752+0000","last_undegraded":"2024-09-17T08:26:27.342752+0000","last_fullsized":"2024-09-17T08:26:27.342752+0000","mapping_epoch":287,"log_start":"309'5800","ondisk_log_start":"309'5800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344981504,"num_objects":5264,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11096,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":408,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606486+0000","last_change":"2024-09-17T08:25:13.823972+0000","last_active":"2024-09-17T08:26:10.606486+0000","last_peered":"2024-09-17T08:26:10.606486+0000","last_clean":"2024-09-17T08:26:10.606486+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:26:10.606486+0000","last_undegraded":"2024-09-17T08:26:10.606486+0000","last_fullsized":"2024-09-17T08:26:10.606486+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:13.823819+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.823819+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.592729+0000","scrub_duration":0.0031292669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":349,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603391+0000","last_change":"2024-09-17T08:25:27.161139+0000","last_active":"2024-09-17T08:26:10.603391+0000","last_peered":"2024-09-17T08:26:10.603391+0000","last_clean":"2024-09-17T08:26:10.603391+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:26:10.603391+0000","last_undegraded":"2024-09-17T08:26:10.603391+0000","last_fullsized":"2024-09-17T08:26:10.603391+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:27.160999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:27.160999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:42.471047+0000","scrub_duration":0.0058695600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":556,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:13.026574+0000","last_change":"2024-09-17T08:25:17.907507+0000","last_active":"2024-09-17T08:26:13.026574+0000","last_peered":"2024-09-17T08:26:13.026574+0000","last_clean":"2024-09-17T08:26:13.026574+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:26:13.026574+0000","last_undegraded":"2024-09-17T08:26:13.026574+0000","last_fullsized":"2024-09-17T08:26:13.026574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:25:17.907373+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.907373+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.869934+0000","scrub_duration":0.001744106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":192,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.620995+0000","last_change":"2024-09-17T08:24:13.623939+0000","last_active":"2024-09-17T08:26:11.620995+0000","last_peered":"2024-09-17T08:26:11.620995+0000","last_clean":"2024-09-17T08:26:11.620995+0000","last_became_active":"2024-09-17T08:23:18.662935+0000","last_became_peered":"2024-09-17T08:23:18.662935+0000","last_unstale":"2024-09-17T08:26:11.620995+0000","last_undegraded":"2024-09-17T08:26:11.620995+0000","last_fullsized":"2024-09-17T08:26:11.620995+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.623844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.623844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001519287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"326'11030","reported_seq":11079,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.686316+0000","last_change":"2024-09-17T08:24:04.453548+0000","last_active":"2024-09-17T08:26:23.686316+0000","last_peered":"2024-09-17T08:26:23.686316+0000","last_clean":"2024-09-17T08:26:23.686316+0000","last_became_active":"2024-09-17T08:24:04.453268+0000","last_became_peered":"2024-09-17T08:24:04.453268+0000","last_unstale":"2024-09-17T08:26:23.686316+0000","last_undegraded":"2024-09-17T08:26:23.686316+0000","last_fullsized":"2024-09-17T08:26:23.686316+0000","mapping_epoch":287,"log_start":"308'5800","ondisk_log_start":"308'5800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372899840,"num_objects":5690,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11030,"num_write_kb":535040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":428,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609763+0000","last_change":"2024-09-17T08:25:29.258817+0000","last_active":"2024-09-17T08:26:11.609763+0000","last_peered":"2024-09-17T08:26:11.609763+0000","last_clean":"2024-09-17T08:26:11.609763+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:26:11.609763+0000","last_undegraded":"2024-09-17T08:26:11.609763+0000","last_fullsized":"2024-09-17T08:26:11.609763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:25:29.258616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.258616+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.967934+0000","scrub_duration":0.011223878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":363,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610326+0000","last_change":"2024-09-17T08:24:32.450419+0000","last_active":"2024-09-17T08:26:11.610326+0000","last_peered":"2024-09-17T08:26:11.610326+0000","last_clean":"2024-09-17T08:26:11.610326+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:26:11.610326+0000","last_undegraded":"2024-09-17T08:26:11.610326+0000","last_fullsized":"2024-09-17T08:26:11.610326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:24:32.450300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.450300+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058717939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":353,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.599236+0000","last_change":"2024-09-17T08:25:25.116593+0000","last_active":"2024-09-17T08:26:10.599236+0000","last_peered":"2024-09-17T08:26:10.599236+0000","last_clean":"2024-09-17T08:26:10.599236+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:26:10.599236+0000","last_undegraded":"2024-09-17T08:26:10.599236+0000","last_fullsized":"2024-09-17T08:26:10.599236+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:25.116441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:25:25.116441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.015212+0000","scrub_duration":0.008148832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":157,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598649+0000","last_change":"2024-09-17T08:24:10.881507+0000","last_active":"2024-09-17T08:26:10.598649+0000","last_peered":"2024-09-17T08:26:10.598649+0000","last_clean":"2024-09-17T08:26:10.598649+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:26:10.598649+0000","last_undegraded":"2024-09-17T08:26:10.598649+0000","last_fullsized":"2024-09-17T08:26:10.598649+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.881407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.881407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014082999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":382,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.624332+0000","last_change":"2024-09-17T08:25:33.033204+0000","last_active":"2024-09-17T08:26:11.624332+0000","last_peered":"2024-09-17T08:26:11.624332+0000","last_clean":"2024-09-17T08:26:11.624332+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:26:11.624332+0000","last_undegraded":"2024-09-17T08:26:11.624332+0000","last_fullsized":"2024-09-17T08:26:11.624332+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.032722+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.145927+0000","scrub_duration":0.0085113229999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":367,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.598717+0000","last_change":"2024-09-17T08:25:34.280542+0000","last_active":"2024-09-17T08:26:10.598717+0000","last_peered":"2024-09-17T08:26:10.598717+0000","last_clean":"2024-09-17T08:26:10.598717+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:26:10.598717+0000","last_undegraded":"2024-09-17T08:26:10.598717+0000","last_fullsized":"2024-09-17T08:26:10.598717+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.278744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.278744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.492017+0000","scrub_duration":0.011508979000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":357,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.609217+0000","last_change":"2024-09-17T08:25:32.809559+0000","last_active":"2024-09-17T08:26:10.609217+0000","last_peered":"2024-09-17T08:26:10.609217+0000","last_clean":"2024-09-17T08:26:10.609217+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:26:10.609217+0000","last_undegraded":"2024-09-17T08:26:10.609217+0000","last_fullsized":"2024-09-17T08:26:10.609217+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:32.809305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:32.809305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.753241+0000","scrub_duration":0.0024200179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":340,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.533286+0000","last_change":"2024-09-17T08:26:11.533286+0000","last_active":"2024-09-17T08:26:11.533286+0000","last_peered":"2024-09-17T08:26:11.533286+0000","last_clean":"2024-09-17T08:26:11.533286+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:26:11.533286+0000","last_undegraded":"2024-09-17T08:26:11.533286+0000","last_fullsized":"2024-09-17T08:26:11.533286+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.533066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.533066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.390854+0000","scrub_duration":0.0073762410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":252,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609835+0000","last_change":"2024-09-17T08:25:18.296524+0000","last_active":"2024-09-17T08:26:11.609835+0000","last_peered":"2024-09-17T08:26:11.609835+0000","last_clean":"2024-09-17T08:26:11.609835+0000","last_became_active":"2024-09-17T08:23:18.666495+0000","last_became_peered":"2024-09-17T08:23:18.666495+0000","last_unstale":"2024-09-17T08:26:11.609835+0000","last_undegraded":"2024-09-17T08:26:11.609835+0000","last_fullsized":"2024-09-17T08:26:11.609835+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.296432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.296432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.437556+0000","scrub_duration":0.0041815000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":404,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:17.976345+0000","last_change":"2024-09-17T08:26:17.976881+0000","last_active":"2024-09-17T08:26:17.976345+0000","last_peered":"2024-09-17T08:26:17.976345+0000","last_clean":"2024-09-17T08:26:17.976345+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:26:17.976345+0000","last_undegraded":"2024-09-17T08:26:17.976345+0000","last_fullsized":"2024-09-17T08:26:17.976345+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:26:17.976320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.976320+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.774813+0000","scrub_duration":0.0092893760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":260,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604158+0000","last_change":"2024-09-17T08:24:19.785545+0000","last_active":"2024-09-17T08:26:10.604158+0000","last_peered":"2024-09-17T08:26:10.604158+0000","last_clean":"2024-09-17T08:26:10.604158+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:26:10.604158+0000","last_undegraded":"2024-09-17T08:26:10.604158+0000","last_fullsized":"2024-09-17T08:26:10.604158+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:19.785421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.785421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005223391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":358,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.606669+0000","last_change":"2024-09-17T08:25:20.873823+0000","last_active":"2024-09-17T08:26:10.606669+0000","last_peered":"2024-09-17T08:26:10.606669+0000","last_clean":"2024-09-17T08:26:10.606669+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:26:10.606669+0000","last_undegraded":"2024-09-17T08:26:10.606669+0000","last_fullsized":"2024-09-17T08:26:10.606669+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:20.873748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:20.873748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:27.013947+0000","scrub_duration":0.001209087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":414,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.615037+0000","last_change":"2024-09-17T08:24:09.894035+0000","last_active":"2024-09-17T08:26:11.615037+0000","last_peered":"2024-09-17T08:26:11.615037+0000","last_clean":"2024-09-17T08:26:11.615037+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:26:11.615037+0000","last_undegraded":"2024-09-17T08:26:11.615037+0000","last_fullsized":"2024-09-17T08:26:11.615037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:09.893951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:09.893951+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019165969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":300,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:16.948097+0000","last_change":"2024-09-17T08:26:16.948193+0000","last_active":"2024-09-17T08:26:16.948097+0000","last_peered":"2024-09-17T08:26:16.948097+0000","last_clean":"2024-09-17T08:26:16.948097+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:26:16.948097+0000","last_undegraded":"2024-09-17T08:26:16.948097+0000","last_fullsized":"2024-09-17T08:26:16.948097+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.948072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.948072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.892623+0000","scrub_duration":0.0075166590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":370,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:18.029701+0000","last_change":"2024-09-17T08:25:11.139098+0000","last_active":"2024-09-17T08:26:18.029701+0000","last_peered":"2024-09-17T08:26:18.029701+0000","last_clean":"2024-09-17T08:26:18.029701+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:26:18.029701+0000","last_undegraded":"2024-09-17T08:26:18.029701+0000","last_fullsized":"2024-09-17T08:26:18.029701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:25:11.138956+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.138956+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004930266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":349,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.624077+0000","last_change":"2024-09-17T08:24:18.708208+0000","last_active":"2024-09-17T08:26:11.624077+0000","last_peered":"2024-09-17T08:26:11.624077+0000","last_clean":"2024-09-17T08:26:11.624077+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:26:11.624077+0000","last_undegraded":"2024-09-17T08:26:11.624077+0000","last_fullsized":"2024-09-17T08:26:11.624077+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:18.708083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:18.708083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010815099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":305,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:13.666634+0000","last_change":"2024-09-17T08:26:13.666735+0000","last_active":"2024-09-17T08:26:13.666634+0000","last_peered":"2024-09-17T08:26:13.666634+0000","last_clean":"2024-09-17T08:26:13.666634+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:26:13.666634+0000","last_undegraded":"2024-09-17T08:26:13.666634+0000","last_fullsized":"2024-09-17T08:26:13.666634+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:13.666621+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.666621+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:16.247569+0000","scrub_duration":0.0059880139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":356,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.355595+0000","last_change":"2024-09-17T08:25:15.032652+0000","last_active":"2024-09-17T08:26:22.355595+0000","last_peered":"2024-09-17T08:26:22.355595+0000","last_clean":"2024-09-17T08:26:22.355595+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:26:22.355595+0000","last_undegraded":"2024-09-17T08:26:22.355595+0000","last_fullsized":"2024-09-17T08:26:22.355595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:15.032525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:25:15.032525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.914352+0000","scrub_duration":0.013962988000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":564,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.031816+0000","last_change":"2024-09-17T08:25:23.783490+0000","last_active":"2024-09-17T08:26:23.031816+0000","last_peered":"2024-09-17T08:26:23.031816+0000","last_clean":"2024-09-17T08:26:23.031816+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:26:23.031816+0000","last_undegraded":"2024-09-17T08:26:23.031816+0000","last_fullsized":"2024-09-17T08:26:23.031816+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:23.783353+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.783353+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.268781+0000","scrub_duration":0.0064248070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":343,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609967+0000","last_change":"2024-09-17T08:23:30.540527+0000","last_active":"2024-09-17T08:26:11.609967+0000","last_peered":"2024-09-17T08:26:11.609967+0000","last_clean":"2024-09-17T08:26:11.609967+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:26:11.609967+0000","last_undegraded":"2024-09-17T08:26:11.609967+0000","last_fullsized":"2024-09-17T08:26:11.609967+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:30.540408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:23:30.540408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023462510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":265,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603927+0000","last_change":"2024-09-17T08:24:15.898072+0000","last_active":"2024-09-17T08:26:10.603927+0000","last_peered":"2024-09-17T08:26:10.603927+0000","last_clean":"2024-09-17T08:26:10.603927+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:26:10.603927+0000","last_undegraded":"2024-09-17T08:26:10.603927+0000","last_fullsized":"2024-09-17T08:26:10.603927+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:24:15.897984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.897984+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018990789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":232,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604801+0000","last_change":"2024-09-17T08:24:13.873686+0000","last_active":"2024-09-17T08:26:10.604801+0000","last_peered":"2024-09-17T08:26:10.604801+0000","last_clean":"2024-09-17T08:26:10.604801+0000","last_became_active":"2024-09-17T08:23:18.671502+0000","last_became_peered":"2024-09-17T08:23:18.671502+0000","last_unstale":"2024-09-17T08:26:10.604801+0000","last_undegraded":"2024-09-17T08:26:10.604801+0000","last_fullsized":"2024-09-17T08:26:10.604801+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.873601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.873601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016156409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":358,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.614450+0000","last_change":"2024-09-17T08:24:10.894915+0000","last_active":"2024-09-17T08:26:11.614450+0000","last_peered":"2024-09-17T08:26:11.614450+0000","last_clean":"2024-09-17T08:26:11.614450+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:26:11.614450+0000","last_undegraded":"2024-09-17T08:26:11.614450+0000","last_fullsized":"2024-09-17T08:26:11.614450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.894828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.894828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001103185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":368,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.355510+0000","last_change":"2024-09-17T08:24:08.507584+0000","last_active":"2024-09-17T08:26:22.355510+0000","last_peered":"2024-09-17T08:26:22.355510+0000","last_clean":"2024-09-17T08:26:22.355510+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:26:22.355510+0000","last_undegraded":"2024-09-17T08:26:22.355510+0000","last_fullsized":"2024-09-17T08:26:22.355510+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:08.507505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:08.507505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014868570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":428,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.708847+0000","last_change":"2024-09-17T08:25:33.258559+0000","last_active":"2024-09-17T08:26:22.708847+0000","last_peered":"2024-09-17T08:26:22.708847+0000","last_clean":"2024-09-17T08:26:22.708847+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:26:22.708847+0000","last_undegraded":"2024-09-17T08:26:22.708847+0000","last_fullsized":"2024-09-17T08:26:22.708847+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.258469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.258469+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.181972+0000","scrub_duration":0.0016288769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604940+0000","last_change":"2024-09-17T08:25:30.191078+0000","last_active":"2024-09-17T08:26:10.604940+0000","last_peered":"2024-09-17T08:26:10.604940+0000","last_clean":"2024-09-17T08:26:10.604940+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:26:10.604940+0000","last_undegraded":"2024-09-17T08:26:10.604940+0000","last_fullsized":"2024-09-17T08:26:10.604940+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:30.190970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:30.190970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.517112+0000","scrub_duration":0.0016272669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":222,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604304+0000","last_change":"2024-09-17T08:23:27.084326+0000","last_active":"2024-09-17T08:26:10.604304+0000","last_peered":"2024-09-17T08:26:10.604304+0000","last_clean":"2024-09-17T08:26:10.604304+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:26:10.604304+0000","last_undegraded":"2024-09-17T08:26:10.604304+0000","last_fullsized":"2024-09-17T08:26:10.604304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:23:27.084153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:23:27.084153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013939082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":338,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.613691+0000","last_change":"2024-09-17T08:24:07.897133+0000","last_active":"2024-09-17T08:26:11.613691+0000","last_peered":"2024-09-17T08:26:11.613691+0000","last_clean":"2024-09-17T08:26:11.613691+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:26:11.613691+0000","last_undegraded":"2024-09-17T08:26:11.613691+0000","last_fullsized":"2024-09-17T08:26:11.613691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:07.897050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:07.897050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015097719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":68,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723118+0000","last_change":"2024-09-17T08:24:26.616991+0000","last_active":"2024-09-17T08:26:26.723118+0000","last_peered":"2024-09-17T08:26:26.723118+0000","last_clean":"2024-09-17T08:26:26.723118+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:26:26.723118+0000","last_undegraded":"2024-09-17T08:26:26.723118+0000","last_fullsized":"2024-09-17T08:26:26.723118+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:24:26.616854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.616854+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002706263,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":227,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.605356+0000","last_change":"2024-09-17T08:25:24.143568+0000","last_active":"2024-09-17T08:26:10.605356+0000","last_peered":"2024-09-17T08:26:10.605356+0000","last_clean":"2024-09-17T08:26:10.605356+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:26:10.605356+0000","last_undegraded":"2024-09-17T08:26:10.605356+0000","last_fullsized":"2024-09-17T08:26:10.605356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:24.143382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:24.143382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:26.119340+0000","scrub_duration":0.0085248119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":234,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.609680+0000","last_change":"2024-09-17T08:25:41.337008+0000","last_active":"2024-09-17T08:26:11.609680+0000","last_peered":"2024-09-17T08:26:11.609680+0000","last_clean":"2024-09-17T08:26:11.609680+0000","last_became_active":"2024-09-17T08:23:18.671271+0000","last_became_peered":"2024-09-17T08:23:18.671271+0000","last_unstale":"2024-09-17T08:26:11.609680+0000","last_undegraded":"2024-09-17T08:26:11.609680+0000","last_fullsized":"2024-09-17T08:26:11.609680+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.336771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.336771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:01.147261+0000","scrub_duration":0.004352692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":363,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.623434+0000","last_change":"2024-09-17T08:25:26.873461+0000","last_active":"2024-09-17T08:26:11.623434+0000","last_peered":"2024-09-17T08:26:11.623434+0000","last_clean":"2024-09-17T08:26:11.623434+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:26:11.623434+0000","last_undegraded":"2024-09-17T08:26:11.623434+0000","last_fullsized":"2024-09-17T08:26:11.623434+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:25:26.873378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.470812+0000","scrub_duration":0.0028675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":373,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.613865+0000","last_change":"2024-09-17T08:24:21.848684+0000","last_active":"2024-09-17T08:26:11.613865+0000","last_peered":"2024-09-17T08:26:11.613865+0000","last_clean":"2024-09-17T08:26:11.613865+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:26:11.613865+0000","last_undegraded":"2024-09-17T08:26:11.613865+0000","last_fullsized":"2024-09-17T08:26:11.613865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:21.848510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.848510+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030240649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":210,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:22.498853+0000","last_change":"2024-09-17T08:26:22.498965+0000","last_active":"2024-09-17T08:26:22.498853+0000","last_peered":"2024-09-17T08:26:22.498853+0000","last_clean":"2024-09-17T08:26:22.498853+0000","last_became_active":"2024-09-17T08:23:18.669127+0000","last_became_peered":"2024-09-17T08:23:18.669127+0000","last_unstale":"2024-09-17T08:26:22.498853+0000","last_undegraded":"2024-09-17T08:26:22.498853+0000","last_fullsized":"2024-09-17T08:26:22.498853+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:26:22.498827+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.498827+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:38.115530+0000","scrub_duration":0.011342636999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":361,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:17.717152+0000","last_change":"2024-09-17T08:26:17.717152+0000","last_active":"2024-09-17T08:26:17.717152+0000","last_peered":"2024-09-17T08:26:17.717152+0000","last_clean":"2024-09-17T08:26:17.717152+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:26:17.717152+0000","last_undegraded":"2024-09-17T08:26:17.717152+0000","last_fullsized":"2024-09-17T08:26:17.717152+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.717066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.717066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:31.062923+0000","scrub_duration":0.0015956360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":337,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603158+0000","last_change":"2024-09-17T08:24:29.919412+0000","last_active":"2024-09-17T08:26:10.603158+0000","last_peered":"2024-09-17T08:26:10.603158+0000","last_clean":"2024-09-17T08:26:10.603158+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:26:10.603158+0000","last_undegraded":"2024-09-17T08:26:10.603158+0000","last_fullsized":"2024-09-17T08:26:10.603158+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:29.919294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.919294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048696069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":364,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.608540+0000","last_change":"2024-09-17T08:25:22.892701+0000","last_active":"2024-09-17T08:26:10.608540+0000","last_peered":"2024-09-17T08:26:10.608540+0000","last_clean":"2024-09-17T08:26:10.608540+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:26:10.608540+0000","last_undegraded":"2024-09-17T08:26:10.608540+0000","last_fullsized":"2024-09-17T08:26:10.608540+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:25:22.892571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.892571+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:32.568284+0000","scrub_duration":0.0024767180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":255,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610858+0000","last_change":"2024-09-17T08:26:11.160161+0000","last_active":"2024-09-17T08:26:11.610858+0000","last_peered":"2024-09-17T08:26:11.610858+0000","last_clean":"2024-09-17T08:26:11.610858+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:26:11.610858+0000","last_undegraded":"2024-09-17T08:26:11.610858+0000","last_fullsized":"2024-09-17T08:26:11.610858+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.159902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.159902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.996888+0000","scrub_duration":0.011043470999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":385,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.624264+0000","last_change":"2024-09-17T08:24:19.731683+0000","last_active":"2024-09-17T08:26:11.624264+0000","last_peered":"2024-09-17T08:26:11.624264+0000","last_clean":"2024-09-17T08:26:11.624264+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:26:11.624264+0000","last_undegraded":"2024-09-17T08:26:11.624264+0000","last_fullsized":"2024-09-17T08:26:11.624264+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:19.731549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.731549+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011256499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":483,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:23.028417+0000","last_change":"2024-09-17T08:26:21.131000+0000","last_active":"2024-09-17T08:26:23.028417+0000","last_peered":"2024-09-17T08:26:23.028417+0000","last_clean":"2024-09-17T08:26:23.028417+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:26:23.028417+0000","last_undegraded":"2024-09-17T08:26:23.028417+0000","last_fullsized":"2024-09-17T08:26:23.028417+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:26:21.130873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.130873+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:23.349562+0000","scrub_duration":0.012556618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":244,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:21.302054+0000","last_change":"2024-09-17T08:26:21.302147+0000","last_active":"2024-09-17T08:26:21.302054+0000","last_peered":"2024-09-17T08:26:21.302054+0000","last_clean":"2024-09-17T08:26:21.302054+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:26:21.302054+0000","last_undegraded":"2024-09-17T08:26:21.302054+0000","last_fullsized":"2024-09-17T08:26:21.302054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:21.302035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.302035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:50.505197+0000","scrub_duration":0.014448862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":343,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.603558+0000","last_change":"2024-09-17T08:24:04.885841+0000","last_active":"2024-09-17T08:26:10.603558+0000","last_peered":"2024-09-17T08:26:10.603558+0000","last_clean":"2024-09-17T08:26:10.603558+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:26:10.603558+0000","last_undegraded":"2024-09-17T08:26:10.603558+0000","last_fullsized":"2024-09-17T08:26:10.603558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:04.885747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:24:04.885747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001511376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":168,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:14.336228+0000","last_change":"2024-09-17T08:26:14.336368+0000","last_active":"2024-09-17T08:26:14.336228+0000","last_peered":"2024-09-17T08:26:14.336228+0000","last_clean":"2024-09-17T08:26:14.336228+0000","last_became_active":"2024-09-17T08:23:18.676257+0000","last_became_peered":"2024-09-17T08:23:18.676257+0000","last_unstale":"2024-09-17T08:26:14.336228+0000","last_undegraded":"2024-09-17T08:26:14.336228+0000","last_fullsized":"2024-09-17T08:26:14.336228+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:14.336205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:14.336205+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.735226+0000","scrub_duration":0.0072317960000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":381,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:16.157165+0000","last_change":"2024-09-17T08:26:16.157165+0000","last_active":"2024-09-17T08:26:16.157165+0000","last_peered":"2024-09-17T08:26:16.157165+0000","last_clean":"2024-09-17T08:26:16.157165+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:26:16.157165+0000","last_undegraded":"2024-09-17T08:26:16.157165+0000","last_fullsized":"2024-09-17T08:26:16.157165+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.157047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.157047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.892586+0000","scrub_duration":0.052471214000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":340,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:11.610148+0000","last_change":"2024-09-17T08:25:19.293077+0000","last_active":"2024-09-17T08:26:11.610148+0000","last_peered":"2024-09-17T08:26:11.610148+0000","last_clean":"2024-09-17T08:26:11.610148+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:26:11.610148+0000","last_undegraded":"2024-09-17T08:26:11.610148+0000","last_fullsized":"2024-09-17T08:26:11.610148+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:19.292858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.292858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013986980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":307,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:20.113140+0000","last_change":"2024-09-17T08:26:20.113140+0000","last_active":"2024-09-17T08:26:20.113140+0000","last_peered":"2024-09-17T08:26:20.113140+0000","last_clean":"2024-09-17T08:26:20.113140+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:26:20.113140+0000","last_undegraded":"2024-09-17T08:26:20.113140+0000","last_fullsized":"2024-09-17T08:26:20.113140+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:20.113014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:26:20.113014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:29.947022+0000","scrub_duration":0.0052786329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":460,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.600272+0000","last_change":"2024-09-17T08:24:23.759585+0000","last_active":"2024-09-17T08:26:10.600272+0000","last_peered":"2024-09-17T08:26:10.600272+0000","last_clean":"2024-09-17T08:26:10.600272+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:26:10.600272+0000","last_undegraded":"2024-09-17T08:26:10.600272+0000","last_fullsized":"2024-09-17T08:26:10.600272+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:24:23.759435+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071267730000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":233,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.604794+0000","last_change":"2024-09-17T08:25:34.089169+0000","last_active":"2024-09-17T08:26:10.604794+0000","last_peered":"2024-09-17T08:26:10.604794+0000","last_clean":"2024-09-17T08:26:10.604794+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:26:10.604794+0000","last_undegraded":"2024-09-17T08:26:10.604794+0000","last_fullsized":"2024-09-17T08:26:10.604794+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.089010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:53.698566+0000","scrub_duration":0.0037643389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":338,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:13.980310+0000","last_change":"2024-09-17T08:26:13.980481+0000","last_active":"2024-09-17T08:26:13.980310+0000","last_peered":"2024-09-17T08:26:13.980310+0000","last_clean":"2024-09-17T08:26:13.980310+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:26:13.980310+0000","last_undegraded":"2024-09-17T08:26:13.980310+0000","last_fullsized":"2024-09-17T08:26:13.980310+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:13.980289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.980289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:15.581630+0000","scrub_duration":0.014435847,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":354,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.612189+0000","last_change":"2024-09-17T08:25:38.883969+0000","last_active":"2024-09-17T08:26:10.612189+0000","last_peered":"2024-09-17T08:26:10.612189+0000","last_clean":"2024-09-17T08:26:10.612189+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:26:10.612189+0000","last_undegraded":"2024-09-17T08:26:10.612189+0000","last_fullsized":"2024-09-17T08:26:10.612189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.883814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.883814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.222351+0000","scrub_duration":0.012313661,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":415,"reported_epoch":325,"state":"active+clean","last_fresh":"2024-09-17T08:26:10.605618+0000","last_change":"2024-09-17T08:24:02.955324+0000","last_active":"2024-09-17T08:26:10.605618+0000","last_peered":"2024-09-17T08:26:10.605618+0000","last_clean":"2024-09-17T08:26:10.605618+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:26:10.605618+0000","last_undegraded":"2024-09-17T08:26:10.605618+0000","last_fullsized":"2024-09-17T08:26:10.605618+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:24:02.955231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:02.955231+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022957339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-17T08:26:18.169361+0000","last_change":"2024-09-17T08:26:18.169361+0000","last_active":"2024-09-17T08:26:18.169361+0000","last_peered":"2024-09-17T08:26:18.169361+0000","last_clean":"2024-09-17T08:26:18.169361+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:26:18.169361+0000","last_undegraded":"2024-09-17T08:26:18.169361+0000","last_fullsized":"2024-09-17T08:26:18.169361+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:18.169258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.169258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:39.643737+0000","scrub_duration":0.042744938000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5791809560,"num_objects":88377,"num_object_clones":0,"num_object_copies":265131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176683,"num_write_kb":8481857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17253937152,"data_stored":17253924936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87383,"ondisk_log_size":87383,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":651,"num_read_kb":646,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070667,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2070908,"kb_used_data":1494260,"kb_used_omap":1,"kb_used_meta":576638,"kb_avail":91674244,"statfs":{"total":95995035648,"available":93874425856,"internally_reserved":0,"allocated":1530122240,"data_stored":1529788710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":590477771},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.363,"15min":1.4370000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":5.3869999999999996,"5min":13.446,"15min":62.018000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":1.476,"15min":1.546},"min":{"1min":0.22,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":9.4169999999999998,"5min":13.417,"15min":61.896999999999998},"last":0.35999999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.7190000000000001,"15min":1.49},"min":{"1min":0.23999999999999999,"5min":0.23899999999999999,"15min":0.23100000000000001},"max":{"1min":5.8369999999999997,"5min":44.917999999999999,"15min":62.067},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.944,"15min":1.4810000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":5.5179999999999998,"5min":44.996000000000002,"15min":62.194000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.98799999999999999,"15min":0.89800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6379999999999999,"5min":4.4169999999999998,"15min":4.4169999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.92400000000000004,"15min":0.68000000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.849,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":4.8920000000000003,"5min":2.6949999999999998,"15min":1.627},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":74.376000000000005,"5min":74.376000000000005,"15min":74.376000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":4.681,"5min":2.7610000000000001,"15min":1.73},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":75.183000000000007,"5min":75.183000000000007,"15min":75.183000000000007},"last":0.307}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.97599999999999998,"15min":0.71699999999999997},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.962,"5min":4.391,"15min":5.335},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73699999999999999,"15min":0.61199999999999999},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":1.845,"5min":3.798,"15min":5.9669999999999996},"last":1.0620000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.88900000000000001,"15min":0.76600000000000001},"min":{"1min":0.27500000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.8959999999999999,"5min":8.6400000000000006,"15min":16.808},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.88300000000000001,"15min":0.753},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":1.9359999999999999,"5min":4.4889999999999999,"15min":11.842000000000001},"last":0.45500000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.95199999999999996,"15min":0.78800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":2.0680000000000001,"5min":6.6509999999999998,"15min":7.2359999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.66500000000000004},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.246},"max":{"1min":3.1819999999999999,"5min":3.3849999999999998,"15min":5.9400000000000004},"last":0.621}]}]},{"osd":6,"up_from":50,"seq":214748365009,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2932288,"kb_used_data":2437564,"kb_used_omap":1,"kb_used_meta":494590,"kb_avail":90812864,"statfs":{"total":95995035648,"available":92992372736,"internally_reserved":0,"allocated":2496065536,"data_stored":2495780398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":506460615},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":3.9220000000000002,"5min":1.6930000000000001,"15min":2.3599999999999999},"min":{"1min":0.24299999999999999,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":55.625,"5min":55.625,"15min":81.007000000000005},"last":1.0029999999999999},{"interface":"front","average":{"1min":4.274,"5min":1.819,"15min":2.4649999999999999},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":55.597000000000001,"5min":55.597000000000001,"15min":80.310000000000002},"last":1.4430000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.494,"15min":1.627},"min":{"1min":0.27900000000000003,"5min":0.26500000000000001,"15min":0.22500000000000001},"max":{"1min":3.7029999999999998,"5min":42.618000000000002,"15min":80.813000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.4690000000000001,"15min":1.6759999999999999},"min":{"1min":0.29499999999999998,"5min":0.251,"15min":0.24299999999999999},"max":{"1min":3.0670000000000002,"5min":42.790999999999997,"15min":80.281000000000006},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":1.8720000000000001,"15min":1.0389999999999999},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.9009999999999998,"5min":70.451999999999998,"15min":70.451999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.865,"15min":0.97499999999999998},"min":{"1min":0.29999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.9180000000000001,"5min":70.488,"15min":70.488},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2050000000000001,"15min":1.9399999999999999},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.1020000000000003,"5min":9.9309999999999992,"15min":80.427000000000007},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.278,"5min":1.1080000000000001,"15min":1.8700000000000001},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26000000000000001},"max":{"1min":3.0449999999999999,"5min":11.859,"15min":80.379999999999995},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66500000000000004,"15min":0.63500000000000001},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":4.6219999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.69499999999999995,"15min":0.58599999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5219999999999998,"5min":3.585,"15min":5.0330000000000004},"last":0.91700000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.84199999999999997,"15min":0.70499999999999996},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.245},"max":{"1min":3.3740000000000001,"5min":5.5659999999999998,"15min":6.1859999999999999},"last":1.413},{"interface":"front","average":{"1min":0.625,"5min":0.73099999999999998,"15min":0.61399999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.9580000000000002,"5min":3.464,"15min":4.9969999999999999},"last":1.359}]},{"osd":7,"last update":"Tue Sep 17 08:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.70799999999999996},"min":{"1min":0.29199999999999998,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":2.948,"5min":2.948,"15min":4.774},"last":1.4950000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.749,"15min":0.64800000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":3.516,"5min":3.5590000000000002,"15min":5.0599999999999996},"last":0.441}]}]},{"osd":5,"up_from":41,"seq":176093659350,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2305920,"kb_used_data":1729144,"kb_used_omap":1,"kb_used_meta":576638,"kb_avail":91439232,"statfs":{"total":95995035648,"available":93633773568,"internally_reserved":0,"allocated":1770643456,"data_stored":1770361795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590477766},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":7.4660000000000002,"5min":3.0750000000000002,"15min":2.1040000000000001},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.20300000000000001},"max":{"1min":64.218999999999994,"5min":65.941000000000003,"15min":65.941000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":7.7880000000000003,"5min":3.2050000000000001,"15min":2.0990000000000002},"min":{"1min":0.316,"5min":0.23000000000000001,"15min":0.22800000000000001},"max":{"1min":65.405000000000001,"5min":66.135000000000005,"15min":66.135000000000005},"last":0.38800000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":2.2549999999999999,"15min":1.3759999999999999},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":3.1309999999999998,"5min":66.981999999999999,"15min":66.981999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":2.3559999999999999,"15min":1.4430000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":2.9129999999999998,"5min":66.316000000000003,"15min":66.316000000000003},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":2.2280000000000002,"5min":3.0339999999999998,"15min":1.498},"min":{"1min":0.28199999999999997,"5min":0.221,"15min":0.221},"max":{"1min":22.245000000000001,"5min":131.727,"15min":131.727},"last":1.05},{"interface":"front","average":{"1min":2.3679999999999999,"5min":3.1850000000000001,"15min":1.7989999999999999},"min":{"1min":0.313,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.146000000000001,"5min":131.50399999999999,"15min":131.50399999999999},"last":1.1639999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":2.1469999999999998,"15min":1.718},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.253},"max":{"1min":3.4260000000000002,"5min":67.676000000000002,"15min":75.286000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":2.177,"15min":1.6819999999999999},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27100000000000002},"max":{"1min":5.4870000000000001,"5min":67.691999999999993,"15min":77.206999999999994},"last":0.748}]},{"osd":4,"last update":"Tue Sep 17 08:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.72199999999999998,"15min":0.76500000000000001},"min":{"1min":0.26800000000000002,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":3.3460000000000001,"5min":4.7569999999999997,"15min":42.421999999999997},"last":2.0419999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.626,"15min":0.73899999999999999},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.24399999999999999},"max":{"1min":3.395,"5min":3.395,"15min":42.191000000000003},"last":1.077}]},{"osd":6,"last update":"Tue Sep 17 08:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.8120000000000003,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.403},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":4.4000000000000004,"5min":4.9189999999999996,"15min":5.4400000000000004},"last":0.46800000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.755},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":3.2989999999999999,"5min":4.9409999999999998,"15min":7.5270000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.26900000000000002},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":0.35299999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953691,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3267852,"kb_used_data":2777224,"kb_used_omap":1,"kb_used_meta":490494,"kb_avail":90477300,"statfs":{"total":95995035648,"available":92648755200,"internally_reserved":0,"allocated":2843877376,"data_stored":2843579984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":502266310},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.93700000000000006,"15min":1.0109999999999999},"min":{"1min":0.254,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":3.4849999999999999,"5min":4.5229999999999997,"15min":23.786000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":0.96699999999999997,"15min":1.056},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":3.7450000000000001,"5min":6.2939999999999996,"15min":23.245000000000001},"last":0.89000000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.2609999999999999,"15min":1.1619999999999999},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":5.5839999999999996,"5min":9.3789999999999996,"15min":23.135000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.2410000000000001,"15min":1.1699999999999999},"min":{"1min":0.39300000000000002,"5min":0.28100000000000003,"15min":0.24099999999999999},"max":{"1min":3.52,"5min":9.0719999999999992,"15min":23.434999999999999},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.78600000000000003,"15min":0.63600000000000001},"min":{"1min":0.29599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6499999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.86199999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":2.3159999999999998,"5min":1.2929999999999999,"15min":1.9430000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":28.138000000000002,"5min":28.138000000000002,"15min":154.137},"last":0.93799999999999994},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.403,"15min":1.8859999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.26700000000000002},"max":{"1min":28.117000000000001,"5min":28.117000000000001,"15min":153.54400000000001},"last":1.524}]},{"osd":5,"last update":"Tue Sep 17 08:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73299999999999998,"15min":0.64300000000000002},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.23000000000000001},"max":{"1min":4.093,"5min":4.093,"15min":6.2329999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.73899999999999999,"15min":0.67800000000000005},"min":{"1min":0.27400000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":3.9409999999999998,"5min":4.0229999999999997,"15min":4.5300000000000002},"last":0.95799999999999996}]},{"osd":6,"last update":"Tue Sep 17 08:25:45 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.60399999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":2.5529999999999999,"5min":3.0110000000000001,"15min":4.827},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":3.6659999999999999,"5min":3.6659999999999999,"15min":6.0010000000000003},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":2.1989999999999998,"5min":4.0259999999999998,"15min":4.875},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.70499999999999996},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.6499999999999999,"5min":2.7949999999999999,"15min":4.6950000000000003},"last":1.502}]}]},{"osd":3,"up_from":27,"seq":115964117214,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2456708,"kb_used_data":1839804,"kb_used_omap":1,"kb_used_meta":616830,"kb_avail":91288444,"statfs":{"total":95995035648,"available":93479366656,"internally_reserved":0,"allocated":1883959296,"data_stored":1883616345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":631634376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:36 2024","interfaces":[{"interface":"back","average":{"1min":8.9649999999999999,"5min":3.7069999999999999,"15min":2.2309999999999999},"min":{"1min":0.54600000000000004,"5min":0.156,"15min":0.156},"max":{"1min":130.40299999999999,"5min":130.40299999999999,"15min":130.40299999999999},"last":0.622},{"interface":"front","average":{"1min":8.2680000000000007,"5min":3.0899999999999999,"15min":2.0230000000000001},"min":{"1min":0.33800000000000002,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":130.70099999999999,"5min":130.70099999999999,"15min":130.70099999999999},"last":1.48}]},{"osd":1,"last update":"Tue Sep 17 08:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":2.153,"15min":1.6919999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.186},"max":{"1min":3.5529999999999999,"5min":71.459999999999994,"15min":71.459999999999994},"last":1.609},{"interface":"front","average":{"1min":1.278,"5min":2.29,"15min":1.706},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.188},"max":{"1min":3.718,"5min":71.438999999999993,"15min":71.438999999999993},"last":0.36799999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.657,"15min":1.268},"min":{"1min":0.253,"5min":0.215,"15min":0.215},"max":{"1min":3.968,"5min":60.423999999999999,"15min":60.423999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.617,"5min":1.8919999999999999,"15min":1.7250000000000001},"min":{"1min":0.35299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.8959999999999999,"5min":60.368000000000002,"15min":60.368000000000002},"last":1.1120000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.306,"15min":1.3420000000000001},"min":{"1min":0.35199999999999998,"5min":0.28499999999999998,"15min":0.252},"max":{"1min":3.7589999999999999,"5min":5.5460000000000003,"15min":35.841000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.194,"15min":1.274},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":3.9889999999999999,"5min":7.5999999999999996,"15min":35.857999999999997},"last":5.1589999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:26:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3440000000000001,"15min":1.2789999999999999},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.29799999999999999},"max":{"1min":3.2570000000000001,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29499999999999998},"max":{"1min":4.71,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.35799999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:25:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.127,"15min":1.179},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":4.2409999999999997,"5min":7.6669999999999998,"15min":10.945},"last":3.6200000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.536,"15min":1.4430000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":4.4829999999999997,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.33000000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":2.9420000000000002,"15min":1.8169999999999999},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":4.4720000000000004,"5min":163.006,"15min":163.006},"last":3.4889999999999999},{"interface":"front","average":{"1min":1.456,"5min":3.089,"15min":1.8420000000000001},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.28999999999999998},"max":{"1min":3.8079999999999998,"5min":162.83699999999999,"15min":162.83699999999999},"last":2.532}]}]},{"osd":2,"up_from":227,"seq":974957576266,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2318640,"kb_used_data":2122664,"kb_used_omap":4,"kb_used_meta":195963,"kb_avail":91426512,"statfs":{"total":95995035648,"available":93620748288,"internally_reserved":0,"allocated":2173607936,"data_stored":2173285757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":200666787},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":2.2759999999999998,"5min":1.327,"15min":1.091},"min":{"1min":0.313,"5min":0.188,"15min":0.188},"max":{"1min":5.6399999999999997,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":2.9249999999999998,"5min":1.3280000000000001,"15min":1.419},"min":{"1min":0.23300000000000001,"5min":0.184,"15min":0.184},"max":{"1min":9.1940000000000008,"5min":9.1940000000000008,"15min":9.1940000000000008},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.8899999999999999,"15min":1.45},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.6950000000000003,"5min":41.405000000000001,"15min":41.405000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":1.7490000000000001,"5min":1.8660000000000001,"15min":1.47},"min":{"1min":0.30499999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":5.617,"5min":41.25,"15min":41.25},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":2.5019999999999998,"5min":1.7749999999999999,"15min":1.542},"min":{"1min":0.33800000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":6.9720000000000004,"5min":44.185000000000002,"15min":44.185000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.7669999999999999,"5min":1.7430000000000001,"15min":1.4610000000000001},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.9109999999999996,"5min":43.994,"15min":43.994},"last":0.318}]},{"osd":4,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.0409999999999999,"15min":0.89700000000000002},"min":{"1min":0.29799999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":1.444},{"interface":"front","average":{"1min":1.389,"5min":0.92500000000000004,"15min":0.85499999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.6749999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":0.38100000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":0.97499999999999998,"15min":0.95199999999999996},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.2240000000000002,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":3.2669999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":0.995,"15min":0.94499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.9770000000000003,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":3.1659999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":0.98899999999999999,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.317,"15min":0.317},"max":{"1min":4.359,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.784,"5min":1.0549999999999999,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.5739999999999998,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.7240000000000002,"5min":4.7240000000000002,"15min":4.7240000000000002},"last":8.9849999999999994},{"interface":"front","average":{"1min":1.496,"5min":1.0680000000000001,"15min":0.93300000000000005},"min":{"1min":0.433,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.2770000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.96399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1720460,"kb_used_data":1085572,"kb_used_omap":1,"kb_used_meta":634878,"kb_avail":92024692,"statfs":{"total":95995035648,"available":94233284608,"internally_reserved":0,"allocated":1111625728,"data_stored":1111275175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":650115526},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":3.3559999999999999,"5min":1.6719999999999999,"15min":1.6930000000000001},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.155},"max":{"1min":27.670999999999999,"5min":27.670999999999999,"15min":31.585999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":2.6320000000000001,"5min":1.8109999999999999,"15min":1.802},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.17199999999999999},"max":{"1min":26.120999999999999,"5min":26.120999999999999,"15min":35.807000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":2.726,"5min":1.5649999999999999,"15min":1.05},"min":{"1min":0.33900000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":26.297999999999998,"5min":26.297999999999998,"15min":26.297999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":2.266,"5min":1.4930000000000001,"15min":1.4870000000000001},"min":{"1min":0.22900000000000001,"5min":0.217,"15min":0.217},"max":{"1min":26.602,"5min":26.602,"15min":26.602},"last":0.39200000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.5589999999999999,"15min":1.383},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.184},"max":{"1min":2.8490000000000002,"5min":5.2629999999999999,"15min":8.0559999999999992},"last":1.23},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.7589999999999999,"15min":1.5740000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.189},"max":{"1min":2.944,"5min":22.984999999999999,"15min":22.984999999999999},"last":1.6919999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.3009999999999999,"15min":1.2769999999999999},"min":{"1min":0.35299999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":2.3199999999999998,"5min":16.884,"15min":16.884},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.1870000000000001,"15min":1.173},"min":{"1min":0.39200000000000002,"5min":0.30499999999999999,"15min":0.25700000000000001},"max":{"1min":2.2480000000000002,"5min":9.6890000000000001,"15min":10.208},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.212,"15min":1.139},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":1.8129999999999999,"5min":4.5,"15min":7.9139999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.88,"5min":1.3300000000000001,"15min":1.3320000000000001},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":3.206,"5min":16.853999999999999,"15min":16.853999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.389,"15min":1.254},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":2.7389999999999999,"5min":9.7159999999999993,"15min":10.253},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.262,"15min":1.24},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.27200000000000002},"max":{"1min":2.855,"5min":5.5469999999999997,"15min":7.9459999999999997},"last":0.49199999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.9670000000000001,"15min":1.5429999999999999},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.33100000000000002},"max":{"1min":2.7519999999999998,"5min":47.442,"15min":47.442},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.7509999999999999,"15min":1.45},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.378},"max":{"1min":2.6970000000000001,"5min":47.475000000000001,"15min":47.475000000000001},"last":0.501}]}]},{"osd":0,"up_from":8,"seq":34359738603,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4082828,"kb_used_data":3469960,"kb_used_omap":1,"kb_used_meta":612734,"kb_avail":89662324,"statfs":{"total":95995035648,"available":91814219776,"internally_reserved":0,"allocated":3553239040,"data_stored":3552942038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":627440070},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":2.105,"15min":1.917},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":10.848000000000001,"5min":41.247,"15min":61.649000000000001},"last":2.8159999999999998},{"interface":"front","average":{"1min":2.1749999999999998,"5min":2.387,"15min":2.125},"min":{"1min":0.182,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1669999999999998,"5min":40.436,"15min":61.281999999999996},"last":3.415}]},{"osd":2,"last update":"Tue Sep 17 08:25:33 2024","interfaces":[{"interface":"back","average":{"1min":2.1309999999999998,"5min":2.6030000000000002,"15min":1.623},"min":{"1min":0.39700000000000002,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":9.1189999999999998,"5min":99.120999999999995,"15min":99.120999999999995},"last":2.597},{"interface":"front","average":{"1min":2.2959999999999998,"5min":2.75,"15min":1.506},"min":{"1min":0.27800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":5.274,"5min":99.256,"15min":99.256},"last":4.6859999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:25:46 2024","interfaces":[{"interface":"back","average":{"1min":2.1160000000000001,"5min":2.0299999999999998,"15min":2.069},"min":{"1min":0.39500000000000002,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":5.968,"5min":38.697000000000003,"15min":61.926000000000002},"last":2.7010000000000001},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.6659999999999999,"15min":1.819},"min":{"1min":0.35099999999999998,"5min":0.223,"15min":0.223},"max":{"1min":10.868,"5min":38.950000000000003,"15min":61.616},"last":3.3490000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.627,"15min":1.413},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26300000000000001},"max":{"1min":5.242,"5min":41.158000000000001,"15min":41.158000000000001},"last":4.6429999999999998},{"interface":"front","average":{"1min":1.923,"5min":1.542,"15min":1.423},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":6.8920000000000003,"5min":38.978000000000002,"15min":38.978000000000002},"last":2.8809999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:25:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.454,"15min":1.2849999999999999},"min":{"1min":0.41899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":10.805999999999999,"5min":39.058,"15min":39.058},"last":4.6230000000000002},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.6839999999999999,"15min":1.4570000000000001},"min":{"1min":0.307,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.7590000000000003,"5min":43.485999999999997,"15min":43.485999999999997},"last":3.117}]},{"osd":6,"last update":"Tue Sep 17 08:25:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.3680000000000001,"15min":1.3009999999999999},"min":{"1min":0.377,"5min":0.317,"15min":0.27700000000000002},"max":{"1min":10.827999999999999,"5min":38.917999999999999,"15min":38.917999999999999},"last":3.3119999999999998},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.7709999999999999,"15min":1.518},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.2400000000000002,"5min":41.189,"15min":41.189},"last":3.1459999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":2.294,"5min":1.8,"15min":1.651},"min":{"1min":0.42599999999999999,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":6.8680000000000003,"5min":39.045999999999999,"15min":39.045999999999999},"last":2.6259999999999999},{"interface":"front","average":{"1min":1.786,"5min":1.5569999999999999,"15min":1.5049999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.307},"max":{"1min":10.769,"5min":39.082999999999998,"15min":39.082999999999998},"last":3.399}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3545305088,"data_stored":3545301016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1102905344,"data_stored":1102905344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2168651776,"data_stored":2168651776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1864433664,"data_stored":1864433664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2816806912,"data_stored":2816802840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1758396416,"data_stored":1758396416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2469597184,"data_stored":2469593112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1527840768,"data_stored":1527840768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:26:30.713 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-17T08:26:30.713 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:26:30.714 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-17T08:26:30.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:26:30.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:30.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:30.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:30.808 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[29565]: pgmap v735: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:26:30.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:26:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[40649]: pgmap v735: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:26:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:26:30.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:30 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:26:30.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:30.895 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:30 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:26:30.896 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:30 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:26:30.896 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:30 smithi173 ceph-mon[33187]: pgmap v735: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-17T08:26:30.896 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:30 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:26:30.896 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:30 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:26:30.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:26:30.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:30.937 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:30.937 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:31.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:31.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:31.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:31.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:31.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.251 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:31.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:31.312 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:31.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:31.397 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:31.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:31.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:31.534 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:31.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:31.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:31.672 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:31.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:31.774 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:31.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:31.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:31.843 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 10.8 scrub ok 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 10.8 scrub ok 2024-09-17T08:26:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:26:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:26:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:26:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:26:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:26:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:31 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:26:31.908 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:31.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:31.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:31.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 10.8 scrub ok 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:26:32.046 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:31 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:26:32.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:32.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:32.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:32.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:32.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:32.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:32.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:32.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:32.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:32.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:32.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:32.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:32.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:32.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:26:32.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:32.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:32.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:32.759 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:32.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:32.910 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:32.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:32.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:32.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:26:32.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:26:32.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:26:32.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:26:32.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: pgmap v736: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 967 op/s 2024-09-17T08:26:32.915 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:32.915 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: pgmap v736: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 967 op/s 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:32.937 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: pgmap v736: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 967 op/s 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:32.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:33.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:33.060 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:33.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:33.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:33.190 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:33.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:33.291 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:33.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:33.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:33.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:33.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.467 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:33.467 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:33.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:33.605 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:33.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:33.619 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:33.619 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:26:33.619 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:26:33.619 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:26:33.619 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:33.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:33.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:33.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:33.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:33.869 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:33.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:33.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:33.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:34.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:34.024 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:26:34.024 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:26:34.024 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:26:34.024 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:26:34.025 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: 10.2 scrub starts 2024-09-17T08:26:34.025 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:34.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: 10.2 scrub starts 2024-09-17T08:26:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: 10.2 scrub starts 2024-09-17T08:26:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:34.143 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:34.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:34.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:34.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:26:34.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:34.422 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:34.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:34.563 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:34.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:34.710 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:34.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:34.759 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:34.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:34.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.848 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:34.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:34.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:34.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:34.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:35.027 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:34 smithi173 ceph-mon[33187]: pgmap v737: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:26:35.027 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:34 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:26:35.027 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:34 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:26:35.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[29565]: pgmap v737: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:26:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:26:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:26:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[40649]: pgmap v737: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-17T08:26:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:26:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:34 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:26:35.124 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:35.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:35.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:35.262 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:35.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:35.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:35.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.405 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:35.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:35.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:35.539 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:35.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:35.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:35.674 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:35.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:35.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:35.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:35.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:35.955 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:35.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:35.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:36.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: 10.2 scrub ok 2024-09-17T08:26:36.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: 10.e scrub starts 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: 10.0 scrub starts 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:35 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:26:36.049 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: 10.2 scrub ok 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: 10.e scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: 10.0 scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: 10.2 scrub ok 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: 10.e scrub starts 2024-09-17T08:26:36.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: 10.0 scrub starts 2024-09-17T08:26:36.051 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T08:26:36.051 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:26:36.051 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:35 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:26:36.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:36.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.101 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:36.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:36.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:36.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:36.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:36.382 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.383 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:36.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:36.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:36.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:36.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:36.653 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:36.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:36.784 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:36.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:36.796 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:26:36.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:36.917 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:36.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:36.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:37.019 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:37.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 10.d scrub starts 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: pgmap v739: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 10.0 scrub ok 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 10.e scrub ok 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 10.f scrub starts 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:26:37.085 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:26:37.086 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:26:37.086 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:36 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:26:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 10.d scrub starts 2024-09-17T08:26:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: pgmap v739: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 10.0 scrub ok 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 10.e scrub ok 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 10.f scrub starts 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 10.d scrub starts 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: pgmap v739: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:26:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 10.0 scrub ok 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 10.e scrub ok 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 10.f scrub starts 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:26:37.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:36 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:26:37.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:37.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:37.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:37.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:137290 8.6 e193/165 8:6be15e8a:::benchmark_data_smithi079_115040_object2270:head v 193'8528, mlcod=192'8526)", 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 515.62505076399998, 2024-09-17T08:26:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.078277779000000006, 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:26:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457621+0000", 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457631+0000", 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0086e-05 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457661+0000", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0505999999999999e-05 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457674+0000", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2907e-05 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457787+0000", 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000112409 2024-09-17T08:26:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.457801+0000", 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4423000000000001e-05 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535864+0000", 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.078062520999999996 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535899+0000", 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5722999999999997e-05 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:137294 8.7 e193/165 8:ec500667:::benchmark_data_smithi079_115040_object2274:head v 193'8664, mlcod=192'8662)", 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 515.61304155699997, 2024-09-17T08:26:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069534631, 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469631+0000", 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469630+0000", 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.469634+0000", 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7230000000000002e-06 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535475+0000", 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.065841199000000003 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535491+0000", 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5633000000000001e-05 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535544+0000", 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2677999999999999e-05 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.535564+0000", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0066999999999999e-05 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.539079+0000", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003515308 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:01.539165+0000", 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6195999999999999e-05 2024-09-17T08:26:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:144399 8.7 e196/165 8:eb4c6be9:::benchmark_data_smithi079_115040_object9379:head v 196'9087, mlcod=196'9083)", 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 509.99143928199999, 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091227230000000006, 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091233+0000", 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091232+0000", 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091240+0000", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2179999999999999e-06 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091248+0000", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4850000000000003e-06 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091253+0000", 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0170000000000004e-06 2024-09-17T08:26:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091286+0000", 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3386000000000002e-05 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.091300+0000", 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3349999999999999e-05 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182423+0000", 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.091123327000000004 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182460+0000", 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7219999999999999e-05 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15096.0:144403 8.5 e196/165 8:a8e6c9af:::benchmark_data_smithi079_115040_object9383:head v 196'9006, mlcod=196'9004)", 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 509.98983203900002, 2024-09-17T08:26:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.099445130000000007, 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092840+0000", 2024-09-17T08:26:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092839+0000", 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.092847+0000", 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9810000000000003e-06 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182416+0000", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.089568712999999994 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182427+0000", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1003e-05 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182468+0000", 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1108e-05 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.182485+0000", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6549999999999999e-05 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.192241+0000", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0097565510000000005 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:18:07.192285+0000", 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4254e-05 2024-09-17T08:26:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24377.0:575 4.a 4:55b0279d:reshard::reshard.0000000009:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 395.45191813100001, 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.87298637199999995, 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24377", 2024-09-17T08:26:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3352227429", 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 575 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630754+0000", 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630753+0000", 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630758+0000", 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6380000000000001e-06 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630769+0000", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1270999999999999e-05 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630774+0000", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0549999999999999e-06 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630825+0000", 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0308e-05 2024-09-17T08:26:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for peered", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.630832+0000", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4100000000000002e-06 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.496920+0000", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.86608754099999996 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497058+0000", 2024-09-17T08:26:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013873499999999999 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497248+0000", 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018980700000000001 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497885+0000", 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00063677199999999997 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497916+0000", 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1127999999999997e-05 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503609+0000", 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056926040000000004 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503683+0000", 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4165999999999994e-05 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503740+0000", 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7522000000000003e-05 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24377.0:594 4.a 4:50d64734:::obj_delete_at_hint.0000000009:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 395.19345110299997, 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.61468610300000004, 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24377", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3352227429", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 594 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889221+0000", 2024-09-17T08:26:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889219+0000", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889225+0000", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7610000000000004e-06 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889236+0000", 2024-09-17T08:26:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0321e-05 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889240+0000", 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8509999999999998e-06 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889280+0000", 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9335000000000002e-05 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for peered", 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:01.889286+0000", 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0650000000000004e-06 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497469+0000", 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.60818299799999997 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497557+0000", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.776e-05 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497664+0000", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000107094 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.497949+0000", 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028577799999999999 2024-09-17T08:26:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.498102+0000", 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000152256 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503812+0000", 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005710499 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503870+0000", 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8198000000000001e-05 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:20:02.503907+0000", 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6820000000000003e-05 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:188021 9.0 e267/230 9:0da46bce:::benchmark_data_smithi079_143589_object56822:head v 267'11558, mlcod=267'11556)", 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:33.528479+0000", 2024-09-17T08:26:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 243.55419360100001, 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022105917999999999, 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528479+0000", 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528479+0000", 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528478+0000", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528487+0000", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5319999999999995e-06 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528499+0000", 2024-09-17T08:26:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1799e-05 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528506+0000", 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.379e-06 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528549+0000", 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3203e-05 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.528568+0000", 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8919e-05 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.550453+0000", 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.021884826 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:33.550585+0000", 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013137199999999999 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:190713 9.b e267/221 9:d716f497:::benchmark_data_smithi079_143589_object59514:head v 267'11841, mlcod=267'11838)", 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:36.451602+0000", 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 240.631070317, 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.046070910999999999, 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451602+0000", 2024-09-17T08:26:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451602+0000", 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451601+0000", 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451610+0000", 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1099999999999992e-06 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451626+0000", 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5736999999999999e-05 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451634+0000", 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3850000000000002e-06 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451681+0000", 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6730000000000002e-05 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.451701+0000", 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0020000000000001e-05 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.497639+0000", 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.045938514999999999 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.497673+0000", 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3501000000000003e-05 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:190720 9.0 e267/230 9:0398f831:::benchmark_data_smithi079_143589_object59521:head v 267'11737, mlcod=267'11735)", 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:22:36.457050+0000", 2024-09-17T08:26:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 240.625622351, 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.045549060000000002, 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.457050+0000", 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.457050+0000", 2024-09-17T08:26:37.394 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.457049+0000", 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.457058+0000", 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3549999999999994e-06 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.395 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.497455+0000", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040396543 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.497487+0000", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2223e-05 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.498079+0000", 2024-09-17T08:26:37.396 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000592346 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.498098+0000", 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9035e-05 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.502496+0000", 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043982300000000004 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.397 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:22:36.502599+0000", 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102513 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152285 10.0 e322/287 10:087558ae:::benchmark_data_smithi079_172150_object19753:head v 322'9536, mlcod=322'9533)", 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.907724+0000", 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.174948407000002, 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.07374211, 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.398 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907724+0000", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907724+0000", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.399 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907723+0000", 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907730+0000", 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.075e-06 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907741+0000", 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1038e-05 2024-09-17T08:26:37.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.907748+0000", 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2590000000000003e-06 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.908167+0000", 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000418701 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.401 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.908185+0000", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8102999999999999e-05 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.981440+0000", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.073255038999999994 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.981466+0000", 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5763e-05 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.402 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152293 10.f e322/287 10:f65539e8:::benchmark_data_smithi079_172150_object19761:head v 322'9511, mlcod=322'9509)", 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.909422+0000", 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.173250494000001, 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069154715000000005, 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.403 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909422+0000", 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909422+0000", 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909421+0000", 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.404 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909432+0000", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.15e-05 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909444+0000", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2238e-05 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909451+0000", 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9489999999999999e-06 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.405 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909511+0000", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9283000000000001e-05 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.909529+0000", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8643000000000001e-05 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913662+0000", 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041326690000000003 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.406 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978576+0000", 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.064914423999999998 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152292 10.a e322/287 10:589bad9b:::benchmark_data_smithi079_172150_object19760:head v 322'9419, mlcod=322'9416)", 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.911054+0000", 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.171618248999998, 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067587649, 2024-09-17T08:26:37.407 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911054+0000", 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911054+0000", 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.408 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911053+0000", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911060+0000", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1049999999999997e-06 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911071+0000", 2024-09-17T08:26:37.409 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0417e-05 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911077+0000", 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2489999999999998e-06 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911127+0000", 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0318999999999997e-05 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.410 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.911146+0000", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8680000000000001e-05 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978552+0000", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067406705999999997 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978642+0000", 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9124999999999998e-05 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.411 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152299 10.0 e322/287 10:062890c0:::benchmark_data_smithi079_172150_object19767:head v 322'9537, mlcod=322'9534)", 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.913389+0000", 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.169283387999997, 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.077152707000000001, 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.412 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913389+0000", 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913389+0000", 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913388+0000", 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.413 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913394+0000", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2779999999999999e-06 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913404+0000", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6700000000000006e-06 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913411+0000", 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6499999999999999e-06 2024-09-17T08:26:37.414 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.913596+0000", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018555399999999999 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978969+0000", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.065372344999999998 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.990509+0000", 2024-09-17T08:26:37.415 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011540727000000001 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.990542+0000", 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2239000000000001e-05 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152307 10.f e322/287 10:f7433f29:::benchmark_data_smithi079_172150_object19775:head v 322'9512, mlcod=322'9509)", 2024-09-17T08:26:37.416 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.920279+0000", 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.162393657999999, 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.065061678999999997, 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920279+0000", 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.417 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920279+0000", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920277+0000", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920288+0000", 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0565e-05 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978517+0000", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.058229230999999999 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978540+0000", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3181999999999999e-05 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978655+0000", 2024-09-17T08:26:37.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000114861 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978709+0000", 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3415999999999998e-05 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.985311+0000", 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.006602742 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.985340+0000", 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8963999999999999e-05 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:152306 10.0 e322/287 10:0423433e:::benchmark_data_smithi079_172150_object19774:head v 322'9538, mlcod=322'9534)", 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:04.920821+0000", 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.161851134999999, 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.069745418000000003, 2024-09-17T08:26:37.421 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920821+0000", 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920821+0000", 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920820+0000", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.920827+0000", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.618e-06 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978509+0000", 2024-09-17T08:26:37.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.057681467 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.978538+0000", 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9359000000000001e-05 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.981499+0000", 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029610959999999999 2024-09-17T08:26:37.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.981517+0000", 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7989e-05 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.990547+0000", 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0090295740000000003 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:04.990567+0000", 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9959e-05 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:186979 10.7 e328/287 10:ea3628bd:::benchmark_data_smithi079_172150_object54447:head v 328'11867, mlcod=328'11864)", 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:34.659349+0000", 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 2.4233230620000001, 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022335073, 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659349+0000", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659349+0000", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659349+0000", 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:37.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659356+0000", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4000000000000003e-06 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659364+0000", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4579999999999993e-06 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659370+0000", 2024-09-17T08:26:37.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5079999999999996e-06 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659549+0000", 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000178598 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.659567+0000", 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8439000000000001e-05 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.675192+0000", 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.015624603000000001 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.681684+0000", 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064926760000000002 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:187021 10.7 e328/287 10:e174d7f1:::benchmark_data_smithi079_172150_object54489:head v 328'11870, mlcod=328'11868)", 2024-09-17T08:26:37.430 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:34.697813+0000", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 2.384859397, 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023088750000000002, 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697813+0000", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697813+0000", 2024-09-17T08:26:37.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697812+0000", 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697819+0000", 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.652e-06 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697828+0000", 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8780000000000004e-06 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.697833+0000", 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4759999999999997e-06 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.702500+0000", 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0046668940000000004 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.702518+0000", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7972999999999999e-05 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.720798+0000", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.018280115 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.720902+0000", 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000103426 2024-09-17T08:26:37.434 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:188504 10.f e329/287 10:f2543177:::benchmark_data_smithi079_172150_object55972:head v 329'11764, mlcod=329'11761)", 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.061776+0000", 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.0208960229999999, 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022845974000000002, 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061776+0000", 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061776+0000", 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061775+0000", 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061786+0000", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.101e-05 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061800+0000", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3524e-05 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061805+0000", 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6389999999999997e-06 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061857+0000", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1604999999999998e-05 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.061870+0000", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3152e-05 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.084563+0000", 2024-09-17T08:26:37.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022692989 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.084622+0000", 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8977000000000001e-05 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:188505 10.f e329/287 10:f812f335:::benchmark_data_smithi079_172150_object55973:head v 329'11765, mlcod=329'11761)", 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.062379+0000", 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.0202936810000001, 2024-09-17T08:26:37.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.029109765999999999, 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062379+0000", 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062379+0000", 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062378+0000", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062386+0000", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2910000000000007e-06 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062398+0000", 2024-09-17T08:26:37.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2183e-05 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.062405+0000", 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2729999999999999e-06 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066737+0000", 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0043321649999999998 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066755+0000", 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8593e-05 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.091416+0000", 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024660687000000001 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.091488+0000", 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2279000000000004e-05 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:188506 10.f e329/287 10:f269de38:::benchmark_data_smithi079_172150_object55974:head v 329'11766, mlcod=329'11761)", 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.066775+0000", 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.0158970549999999, 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.024739573000000001, 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066775+0000", 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066775+0000", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066774+0000", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066784+0000", 2024-09-17T08:26:37.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4900000000000006e-06 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066796+0000", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1938e-05 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.066802+0000", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6540000000000002e-06 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.072419+0000", 2024-09-17T08:26:37.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056162809999999999 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.072440+0000", 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1149999999999999e-05 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.091492+0000", 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019052089000000001 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:26:37.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.091515+0000", 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2840000000000002e-05 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:37.448 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:37.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:37.505 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:37.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:37.606 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:37.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.632 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:37.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:37.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:37.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:37.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:37.902 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:37.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:38.004 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:38.037 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:38.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: 10.d scrub ok 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[40649]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: 10.d scrub ok 2024-09-17T08:26:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:38.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:26:38.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:37 smithi079 ceph-mon[29565]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T08:26:38.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:38.173 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:38.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:26:38.229 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: 10.d scrub ok 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1893101255' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:26:38.229 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:37 smithi173 ceph-mon[33187]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T08:26:38.264 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:26:38.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:38.309 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:38.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:38.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:38.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:38.451 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:38.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:38.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:38.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:38.578 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:38.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:38.722 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:38.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:38.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:38.825 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:39.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:38 smithi173 ceph-mon[33187]: pgmap v741: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:39.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:38 smithi173 ceph-mon[33187]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:39.040 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:39.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:38 smithi079 ceph-mon[40649]: pgmap v741: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:39.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:38 smithi079 ceph-mon[40649]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:38 smithi079 ceph-mon[29565]: pgmap v741: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:26:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:38 smithi079 ceph-mon[29565]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:26:39.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:39.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.170 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:39.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:39.271 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:39.302 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:39.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:39.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:39.436 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:39.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:39.539 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:39.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.576 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:39.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:39.677 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:39.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.712 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:39.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:39.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:39.855 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:39.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:39.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:39.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:39.991 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:39.992 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:40.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:40.125 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:40.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:40.226 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:40.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:40.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:40.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:40.397 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:40.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:40.499 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:40.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:40.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:40.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:40.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:40.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:40.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:194083 10.c 10:3837c52e:::benchmark_data_smithi079_172150_object61551:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.777387+0000", 2024-09-17T08:26:40.787 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0077224329999999999, 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0077546480000000003, 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 194083 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777387+0000", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777387+0000", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777384+0000", 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:26:40.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777391+0000", 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8809999999999998e-06 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777409+0000", 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7649e-05 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777418+0000", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7280000000000001e-06 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777466+0000", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8690000000000003e-05 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777616+0000", 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015001899999999999 2024-09-17T08:26:40.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,4", 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.777740+0000", 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012397999999999999 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.783613+0000", 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0058723020000000003 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.783873+0000", 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026039100000000001 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:194086 10.c 10:3c338f9d:::benchmark_data_smithi079_172150_object61554:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.779064+0000", 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0060444970000000002, 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0061701689999999997, 2024-09-17T08:26:40.793 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 194086 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779064+0000", 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779064+0000", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779063+0000", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779069+0000", 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6749999999999999e-06 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779084+0000", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5690999999999999e-05 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.779092+0000", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3270000000000003e-06 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780679+0000", 2024-09-17T08:26:40.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0015867190000000001 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780803+0000", 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012417600000000001 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,4", 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780982+0000", 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000179178 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.783618+0000", 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026359690000000002 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784331+0000", 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071310299999999998 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:194088 10.9 e330/287 10:9c701599:::benchmark_data_smithi079_172150_object61556:head v 330'12035, mlcod=330'12032)", 2024-09-17T08:26:40.798 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.780452+0000", 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0046574310000000001, 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0048696720000000002, 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780452+0000", 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780452+0000", 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780451+0000", 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780459+0000", 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5560000000000002e-06 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780471+0000", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2462e-05 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.780477+0000", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.451e-06 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.782232+0000", 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017550319999999999 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.782255+0000", 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3887e-05 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:194093 10.c 10.32634a3c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.784480+0000", 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00062921699999999995, 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090110699999999995, 2024-09-17T08:26:40.802 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 194093 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784480+0000", 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784480+0000", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784478+0000", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784485+0000", 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1160000000000001e-06 2024-09-17T08:26:40.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784501+0000", 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5483000000000001e-05 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.784508+0000", 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4309999999999997e-06 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:194099 10.c 10.5d6efecc (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.785199+0000", 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4294967295.9999099, 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024816999999999998, 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 194099 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785199+0000", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785199+0000", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785198+0000", 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785206+0000", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8869999999999992e-06 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785222+0000", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5522000000000001e-05 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785229+0000", 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7100000000000007e-06 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.808 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.15576.0:194083 10.c e330/287 ondisk, result = 0)", 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.785248+0000", 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4294967295.9998608, 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025298299999999997, 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785248+0000", 2024-09-17T08:26:40.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785248+0000", 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785247+0000", 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785251+0000", 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7589999999999999e-06 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785260+0000", 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2220000000000008e-06 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785266+0000", 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3290000000000001e-06 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785326+0000", 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0210999999999997e-05 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785333+0000", 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1810000000000004e-06 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:194097 10.1 e330/287)", 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.785423+0000", 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4294967295.9996862, 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012778299999999999, 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785423+0000", 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785423+0000", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785422+0000", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785428+0000", 2024-09-17T08:26:40.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8279999999999999e-06 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785436+0000", 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5110000000000003e-06 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785442+0000", 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5899999999999998e-06 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.815 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.15576.0:194086 10.c e330/287 ondisk, result = 0)", 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:40.785475+0000", 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4294967295.9996333, 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011487, 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785475+0000", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785475+0000", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785475+0000", 2024-09-17T08:26:40.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785478+0000", 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0750000000000002e-06 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785485+0000", 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2740000000000002e-06 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785490+0000", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.437e-06 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785568+0000", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7836999999999994e-05 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:40.785573+0000", 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5430000000000001e-06 2024-09-17T08:26:40.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 8 2024-09-17T08:26:40.820 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:26:40.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:40.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:40.856 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:40.885 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":741,"stamp":"2024-09-17T08:26:37.580836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5095823662,"num_objects":77944,"num_object_clones":0,"num_object_copies":233832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77944,"num_whiteouts":0,"num_read":2005,"num_read_kb":1851,"num_write":188207,"num_write_kb":8484316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":1775729,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88383,"ondisk_log_size":88383,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19196804,"kb_used_data":15024832,"kb_used_omap":15,"kb_used_meta":4171760,"kb_avail":730764412,"statfs":{"total":767960285184,"available":748302757888,"internally_reserved":0,"allocated":15385427968,"data_stored":15383016384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":4271883029},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,8,10,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-547946496,"num_objects":-8361,"num_object_clones":0,"num_object_copies":-25083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8361,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8361,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003066"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":339,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738897+0000","last_change":"2024-09-17T08:24:22.829264+0000","last_active":"2024-09-17T08:26:26.738897+0000","last_peered":"2024-09-17T08:26:26.738897+0000","last_clean":"2024-09-17T08:26:26.738897+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:26:26.738897+0000","last_undegraded":"2024-09-17T08:26:26.738897+0000","last_fullsized":"2024-09-17T08:26:26.738897+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:22.829138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:24:22.829138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002109297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":371,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.802002+0000","last_change":"2024-09-17T08:26:23.241237+0000","last_active":"2024-09-17T08:26:36.802002+0000","last_peered":"2024-09-17T08:26:36.802002+0000","last_clean":"2024-09-17T08:26:36.802002+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:26:36.802002+0000","last_undegraded":"2024-09-17T08:26:36.802002+0000","last_fullsized":"2024-09-17T08:26:36.802002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.241125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.241125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:49.606056+0000","scrub_duration":0.002185354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":416,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737180+0000","last_change":"2024-09-17T08:26:21.027223+0000","last_active":"2024-09-17T08:26:26.737180+0000","last_peered":"2024-09-17T08:26:26.737180+0000","last_clean":"2024-09-17T08:26:26.737180+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:26:26.737180+0000","last_undegraded":"2024-09-17T08:26:26.737180+0000","last_fullsized":"2024-09-17T08:26:26.737180+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:26:21.027087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.027087+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:45.075711+0000","scrub_duration":0.004246901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":370,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:30.172227+0000","last_change":"2024-09-17T08:26:30.172498+0000","last_active":"2024-09-17T08:26:30.172227+0000","last_peered":"2024-09-17T08:26:30.172227+0000","last_clean":"2024-09-17T08:26:30.172227+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:26:30.172227+0000","last_undegraded":"2024-09-17T08:26:30.172227+0000","last_fullsized":"2024-09-17T08:26:30.172227+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.172198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.172198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.460859+0000","scrub_duration":0.0047583649999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":347,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737075+0000","last_change":"2024-09-17T08:25:12.813837+0000","last_active":"2024-09-17T08:26:26.737075+0000","last_peered":"2024-09-17T08:26:26.737075+0000","last_clean":"2024-09-17T08:26:26.737075+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:26:26.737075+0000","last_undegraded":"2024-09-17T08:26:26.737075+0000","last_fullsized":"2024-09-17T08:26:26.737075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.813704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.813704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.648729+0000","scrub_duration":0.0071490010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":472,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723367+0000","last_change":"2024-09-17T08:24:29.572754+0000","last_active":"2024-09-17T08:26:26.723367+0000","last_peered":"2024-09-17T08:26:26.723367+0000","last_clean":"2024-09-17T08:26:26.723367+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:26:26.723367+0000","last_undegraded":"2024-09-17T08:26:26.723367+0000","last_fullsized":"2024-09-17T08:26:26.723367+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:29.572400+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.572400+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041638179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723513+0000","last_change":"2024-09-17T08:25:29.240062+0000","last_active":"2024-09-17T08:26:26.723513+0000","last_peered":"2024-09-17T08:26:26.723513+0000","last_clean":"2024-09-17T08:26:26.723513+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:26:26.723513+0000","last_undegraded":"2024-09-17T08:26:26.723513+0000","last_fullsized":"2024-09-17T08:26:26.723513+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.239407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.239407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.084448+0000","scrub_duration":0.0062858009999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":368,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723341+0000","last_change":"2024-09-17T08:25:28.285107+0000","last_active":"2024-09-17T08:26:26.723341+0000","last_peered":"2024-09-17T08:26:26.723341+0000","last_clean":"2024-09-17T08:26:26.723341+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:26:26.723341+0000","last_undegraded":"2024-09-17T08:26:26.723341+0000","last_fullsized":"2024-09-17T08:26:26.723341+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.284981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:28.287282+0000","scrub_duration":0.0042235850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":355,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.731221+0000","last_change":"2024-09-17T08:25:14.860865+0000","last_active":"2024-09-17T08:26:26.731221+0000","last_peered":"2024-09-17T08:26:26.731221+0000","last_clean":"2024-09-17T08:26:26.731221+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:26:26.731221+0000","last_undegraded":"2024-09-17T08:26:26.731221+0000","last_fullsized":"2024-09-17T08:26:26.731221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.860746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.860746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.290009+0000","scrub_duration":0.0074367399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":388,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723984+0000","last_change":"2024-09-17T08:24:16.615788+0000","last_active":"2024-09-17T08:26:26.723984+0000","last_peered":"2024-09-17T08:26:26.723984+0000","last_clean":"2024-09-17T08:26:26.723984+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:26:26.723984+0000","last_undegraded":"2024-09-17T08:26:26.723984+0000","last_fullsized":"2024-09-17T08:26:26.723984+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:24:16.615683+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:24:16.615683+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001772476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":380,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730180+0000","last_change":"2024-09-17T08:24:32.685119+0000","last_active":"2024-09-17T08:26:26.730180+0000","last_peered":"2024-09-17T08:26:26.730180+0000","last_clean":"2024-09-17T08:26:26.730180+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:26:26.730180+0000","last_undegraded":"2024-09-17T08:26:26.730180+0000","last_fullsized":"2024-09-17T08:26:26.730180+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:32.684474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.684474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038163649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723377+0000","last_change":"2024-09-17T08:24:25.823602+0000","last_active":"2024-09-17T08:26:26.723377+0000","last_peered":"2024-09-17T08:26:26.723377+0000","last_clean":"2024-09-17T08:26:26.723377+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:26:26.723377+0000","last_undegraded":"2024-09-17T08:26:26.723377+0000","last_fullsized":"2024-09-17T08:26:26.723377+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:25.823472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.823472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":381,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.731759+0000","last_change":"2024-09-17T08:26:11.668739+0000","last_active":"2024-09-17T08:26:26.731759+0000","last_peered":"2024-09-17T08:26:26.731759+0000","last_clean":"2024-09-17T08:26:26.731759+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:26:26.731759+0000","last_undegraded":"2024-09-17T08:26:26.731759+0000","last_fullsized":"2024-09-17T08:26:26.731759+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.668207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.668207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.803189+0000","scrub_duration":0.0090449410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":383,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723593+0000","last_change":"2024-09-17T08:25:22.208990+0000","last_active":"2024-09-17T08:26:26.723593+0000","last_peered":"2024-09-17T08:26:26.723593+0000","last_clean":"2024-09-17T08:26:26.723593+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:26:26.723593+0000","last_undegraded":"2024-09-17T08:26:26.723593+0000","last_fullsized":"2024-09-17T08:26:26.723593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:25:22.208698+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.208698+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":257,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.778179+0000","last_change":"2024-09-17T08:24:27.797180+0000","last_active":"2024-09-17T08:26:34.778179+0000","last_peered":"2024-09-17T08:26:34.778179+0000","last_clean":"2024-09-17T08:26:34.778179+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:26:34.778179+0000","last_undegraded":"2024-09-17T08:26:34.778179+0000","last_fullsized":"2024-09-17T08:26:34.778179+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:27.796880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:27.796880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049344319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":370,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.777311+0000","last_change":"2024-09-17T08:24:20.752436+0000","last_active":"2024-09-17T08:26:34.777311+0000","last_peered":"2024-09-17T08:26:34.777311+0000","last_clean":"2024-09-17T08:26:34.777311+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:26:34.777311+0000","last_undegraded":"2024-09-17T08:26:34.777311+0000","last_fullsized":"2024-09-17T08:26:34.777311+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:20.752313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:20.752313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027283759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"329'11911","reported_seq":12114,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.648295+0000","last_change":"2024-09-17T08:25:28.636224+0000","last_active":"2024-09-17T08:26:36.648295+0000","last_peered":"2024-09-17T08:26:36.648295+0000","last_clean":"2024-09-17T08:26:36.648295+0000","last_became_active":"2024-09-17T08:24:04.453791+0000","last_became_peered":"2024-09-17T08:24:04.453791+0000","last_unstale":"2024-09-17T08:26:36.648295+0000","last_undegraded":"2024-09-17T08:26:36.648295+0000","last_fullsized":"2024-09-17T08:26:36.648295+0000","mapping_epoch":287,"log_start":"307'5500","ondisk_log_start":"307'5500","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6965","last_scrub_stamp":"2024-09-17T08:25:28.635952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.635952+0000","objects_scrubbed":6875,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:52.900306+0000","scrub_duration":2.5130719930000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313458688,"num_objects":4783,"num_object_clones":0,"num_object_copies":14349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":534208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":479,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.725514+0000","last_change":"2024-09-17T08:24:17.608622+0000","last_active":"2024-09-17T08:26:26.725514+0000","last_peered":"2024-09-17T08:26:26.725514+0000","last_clean":"2024-09-17T08:26:26.725514+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:26:26.725514+0000","last_undegraded":"2024-09-17T08:26:26.725514+0000","last_fullsized":"2024-09-17T08:26:26.725514+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:24:17.608509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:17.608509+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025730330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":367,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730802+0000","last_change":"2024-09-17T08:25:21.870284+0000","last_active":"2024-09-17T08:26:26.730802+0000","last_peered":"2024-09-17T08:26:26.730802+0000","last_clean":"2024-09-17T08:26:26.730802+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:26:26.730802+0000","last_undegraded":"2024-09-17T08:26:26.730802+0000","last_fullsized":"2024-09-17T08:26:26.730802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:21.870174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:21.870174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.767809+0000","scrub_duration":0.0028963090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":246,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724277+0000","last_change":"2024-09-17T08:26:22.325911+0000","last_active":"2024-09-17T08:26:26.724277+0000","last_peered":"2024-09-17T08:26:26.724277+0000","last_clean":"2024-09-17T08:26:26.724277+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:26:26.724277+0000","last_undegraded":"2024-09-17T08:26:26.724277+0000","last_fullsized":"2024-09-17T08:26:26.724277+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:22.325816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.325816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.578172+0000","scrub_duration":0.0015290900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":336,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.735692+0000","last_change":"2024-09-17T08:25:22.077723+0000","last_active":"2024-09-17T08:26:27.735692+0000","last_peered":"2024-09-17T08:26:27.735692+0000","last_clean":"2024-09-17T08:26:27.735692+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:26:27.735692+0000","last_undegraded":"2024-09-17T08:26:27.735692+0000","last_fullsized":"2024-09-17T08:26:27.735692+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:22.077429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.077429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:35.599110+0000","scrub_duration":0.0073898080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"328'11389","reported_seq":11535,"reported_epoch":328,"state":"active+clean+scrubbing","last_fresh":"2024-09-17T08:26:33.826580+0000","last_change":"2024-09-17T08:26:32.282976+0000","last_active":"2024-09-17T08:26:33.826580+0000","last_peered":"2024-09-17T08:26:33.826580+0000","last_clean":"2024-09-17T08:26:33.826580+0000","last_became_active":"2024-09-17T08:24:04.455271+0000","last_became_peered":"2024-09-17T08:24:04.455271+0000","last_unstale":"2024-09-17T08:26:33.826580+0000","last_undegraded":"2024-09-17T08:26:33.826580+0000","last_fullsized":"2024-09-17T08:26:33.826580+0000","mapping_epoch":287,"log_start":"309'5900","ondisk_log_start":"309'5900","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":4349,"log_size":5489,"log_dups_size":0,"ondisk_log_size":5489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318439424,"num_objects":4859,"num_object_clones":0,"num_object_copies":14577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11389,"num_write_kb":519936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":385,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723769+0000","last_change":"2024-09-17T08:25:23.209427+0000","last_active":"2024-09-17T08:26:26.723769+0000","last_peered":"2024-09-17T08:26:26.723769+0000","last_clean":"2024-09-17T08:26:26.723769+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:26:26.723769+0000","last_undegraded":"2024-09-17T08:26:26.723769+0000","last_fullsized":"2024-09-17T08:26:26.723769+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:25:23.209287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.209287+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.695831+0000","scrub_duration":0.0047713479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":236,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.725338+0000","last_change":"2024-09-17T08:25:33.132550+0000","last_active":"2024-09-17T08:26:26.725338+0000","last_peered":"2024-09-17T08:26:26.725338+0000","last_clean":"2024-09-17T08:26:26.725338+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:26:26.725338+0000","last_undegraded":"2024-09-17T08:26:26.725338+0000","last_fullsized":"2024-09-17T08:26:26.725338+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:33.131878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.131878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:58.566704+0000","scrub_duration":0.008861519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":515,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:33.037022+0000","last_change":"2024-09-17T08:25:11.078189+0000","last_active":"2024-09-17T08:26:33.037022+0000","last_peered":"2024-09-17T08:26:33.037022+0000","last_clean":"2024-09-17T08:26:33.037022+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:26:33.037022+0000","last_undegraded":"2024-09-17T08:26:33.037022+0000","last_fullsized":"2024-09-17T08:26:33.037022+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:11.078003+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.078003+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:39.754804+0000","scrub_duration":0.0057575789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":374,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.231865+0000","last_change":"2024-09-17T08:26:32.232090+0000","last_active":"2024-09-17T08:26:32.231865+0000","last_peered":"2024-09-17T08:26:32.231865+0000","last_clean":"2024-09-17T08:26:32.231865+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:26:32.231865+0000","last_undegraded":"2024-09-17T08:26:32.231865+0000","last_fullsized":"2024-09-17T08:26:32.231865+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:32.231846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:26:32.231846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.225179+0000","scrub_duration":0.0087096559999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"328'11749","reported_seq":11799,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.706813+0000","last_change":"2024-09-17T08:24:04.459231+0000","last_active":"2024-09-17T08:26:32.706813+0000","last_peered":"2024-09-17T08:26:32.706813+0000","last_clean":"2024-09-17T08:26:32.706813+0000","last_became_active":"2024-09-17T08:24:04.458930+0000","last_became_peered":"2024-09-17T08:24:04.458930+0000","last_unstale":"2024-09-17T08:26:32.706813+0000","last_undegraded":"2024-09-17T08:26:32.706813+0000","last_fullsized":"2024-09-17T08:26:32.706813+0000","mapping_epoch":287,"log_start":"312'6800","ondisk_log_start":"312'6800","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":334168064,"num_objects":5099,"num_object_clones":0,"num_object_copies":15297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11749,"num_write_kb":539136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":238,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.735424+0000","last_change":"2024-09-17T08:25:39.196006+0000","last_active":"2024-09-17T08:26:27.735424+0000","last_peered":"2024-09-17T08:26:27.735424+0000","last_clean":"2024-09-17T08:26:27.735424+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:26:27.735424+0000","last_undegraded":"2024-09-17T08:26:27.735424+0000","last_fullsized":"2024-09-17T08:26:27.735424+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:39.195587+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.195587+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:00.706411+0000","scrub_duration":0.0050104839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":236,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723447+0000","last_change":"2024-09-17T08:25:12.167258+0000","last_active":"2024-09-17T08:26:26.723447+0000","last_peered":"2024-09-17T08:26:26.723447+0000","last_clean":"2024-09-17T08:26:26.723447+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:26:26.723447+0000","last_undegraded":"2024-09-17T08:26:26.723447+0000","last_fullsized":"2024-09-17T08:26:26.723447+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.167132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.167132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020782719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":550,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.729244+0000","last_change":"2024-09-17T08:24:14.547197+0000","last_active":"2024-09-17T08:26:27.729244+0000","last_peered":"2024-09-17T08:26:27.729244+0000","last_clean":"2024-09-17T08:26:27.729244+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:26:27.729244+0000","last_undegraded":"2024-09-17T08:26:27.729244+0000","last_fullsized":"2024-09-17T08:26:27.729244+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:24:14.547105+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:24:14.547105+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013424540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724721+0000","last_change":"2024-09-17T08:25:28.215581+0000","last_active":"2024-09-17T08:26:26.724721+0000","last_peered":"2024-09-17T08:26:26.724721+0000","last_clean":"2024-09-17T08:26:26.724721+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:26:26.724721+0000","last_undegraded":"2024-09-17T08:26:26.724721+0000","last_fullsized":"2024-09-17T08:26:26.724721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.215457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.215457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.036848+0000","scrub_duration":0.0070925789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"328'11746","reported_seq":11797,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:33.832806+0000","last_change":"2024-09-17T08:24:04.455364+0000","last_active":"2024-09-17T08:26:33.832806+0000","last_peered":"2024-09-17T08:26:33.832806+0000","last_clean":"2024-09-17T08:26:33.832806+0000","last_became_active":"2024-09-17T08:24:04.455098+0000","last_became_peered":"2024-09-17T08:24:04.455098+0000","last_unstale":"2024-09-17T08:26:33.832806+0000","last_undegraded":"2024-09-17T08:26:33.832806+0000","last_fullsized":"2024-09-17T08:26:33.832806+0000","mapping_epoch":287,"log_start":"310'6200","ondisk_log_start":"310'6200","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5546,"log_dups_size":0,"ondisk_log_size":5546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317456384,"num_objects":4844,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11746,"num_write_kb":530880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":423,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.728979+0000","last_change":"2024-09-17T08:24:25.581898+0000","last_active":"2024-09-17T08:26:26.728979+0000","last_peered":"2024-09-17T08:26:26.728979+0000","last_clean":"2024-09-17T08:26:26.728979+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:26:26.728979+0000","last_undegraded":"2024-09-17T08:26:26.728979+0000","last_fullsized":"2024-09-17T08:26:26.728979+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:24:25.581740+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.581740+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004957894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":374,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.733005+0000","last_change":"2024-09-17T08:26:12.688090+0000","last_active":"2024-09-17T08:26:26.733005+0000","last_peered":"2024-09-17T08:26:26.733005+0000","last_clean":"2024-09-17T08:26:26.733005+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:26:26.733005+0000","last_undegraded":"2024-09-17T08:26:26.733005+0000","last_fullsized":"2024-09-17T08:26:26.733005+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:12.687091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.687091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.608122+0000","scrub_duration":0.006317067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":246,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723627+0000","last_change":"2024-09-17T08:24:30.522675+0000","last_active":"2024-09-17T08:26:26.723627+0000","last_peered":"2024-09-17T08:26:26.723627+0000","last_clean":"2024-09-17T08:26:26.723627+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:26:26.723627+0000","last_undegraded":"2024-09-17T08:26:26.723627+0000","last_fullsized":"2024-09-17T08:26:26.723627+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.522581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001427191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":340,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737959+0000","last_change":"2024-09-17T08:25:40.909763+0000","last_active":"2024-09-17T08:26:26.737959+0000","last_peered":"2024-09-17T08:26:26.737959+0000","last_clean":"2024-09-17T08:26:26.737959+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:26:26.737959+0000","last_undegraded":"2024-09-17T08:26:26.737959+0000","last_fullsized":"2024-09-17T08:26:26.737959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.909493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.909493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:51.177881+0000","scrub_duration":0.014995481999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"328'11582","reported_seq":11633,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:33.672936+0000","last_change":"2024-09-17T08:24:04.454563+0000","last_active":"2024-09-17T08:26:33.672936+0000","last_peered":"2024-09-17T08:26:33.672936+0000","last_clean":"2024-09-17T08:26:33.672936+0000","last_became_active":"2024-09-17T08:24:04.454206+0000","last_became_peered":"2024-09-17T08:24:04.454206+0000","last_unstale":"2024-09-17T08:26:33.672936+0000","last_undegraded":"2024-09-17T08:26:33.672936+0000","last_fullsized":"2024-09-17T08:26:33.672936+0000","mapping_epoch":287,"log_start":"310'6400","ondisk_log_start":"310'6400","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5182,"log_dups_size":0,"ondisk_log_size":5182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322699264,"num_objects":4924,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":528192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":386,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727108+0000","last_change":"2024-09-17T08:25:12.405766+0000","last_active":"2024-09-17T08:26:26.727108+0000","last_peered":"2024-09-17T08:26:26.727108+0000","last_clean":"2024-09-17T08:26:26.727108+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:26:26.727108+0000","last_undegraded":"2024-09-17T08:26:26.727108+0000","last_fullsized":"2024-09-17T08:26:26.727108+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:25:12.405574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.405574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00623603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":250,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:29.708578+0000","last_change":"2024-09-17T08:26:29.708674+0000","last_active":"2024-09-17T08:26:29.708578+0000","last_peered":"2024-09-17T08:26:29.708578+0000","last_clean":"2024-09-17T08:26:29.708578+0000","last_became_active":"2024-09-17T08:23:18.669266+0000","last_became_peered":"2024-09-17T08:23:18.669266+0000","last_unstale":"2024-09-17T08:26:29.708578+0000","last_undegraded":"2024-09-17T08:26:29.708578+0000","last_fullsized":"2024-09-17T08:26:29.708578+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:29.708555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:26:29.708555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:33.829571+0000","scrub_duration":0.0021163079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":583,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:28.030126+0000","last_change":"2024-09-17T08:24:12.872168+0000","last_active":"2024-09-17T08:26:28.030126+0000","last_peered":"2024-09-17T08:26:28.030126+0000","last_clean":"2024-09-17T08:26:28.030126+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:26:28.030126+0000","last_undegraded":"2024-09-17T08:26:28.030126+0000","last_fullsized":"2024-09-17T08:26:28.030126+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:24:12.872090+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.872090+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":366,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730541+0000","last_change":"2024-09-17T08:25:29.822800+0000","last_active":"2024-09-17T08:26:26.730541+0000","last_peered":"2024-09-17T08:26:26.730541+0000","last_clean":"2024-09-17T08:26:26.730541+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:26:26.730541+0000","last_undegraded":"2024-09-17T08:26:26.730541+0000","last_fullsized":"2024-09-17T08:26:26.730541+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.822689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.822689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.095715+0000","scrub_duration":0.0085561849999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"328'11613","reported_seq":11664,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.638348+0000","last_change":"2024-09-17T08:24:04.453118+0000","last_active":"2024-09-17T08:26:32.638348+0000","last_peered":"2024-09-17T08:26:32.638348+0000","last_clean":"2024-09-17T08:26:32.638348+0000","last_became_active":"2024-09-17T08:24:04.452780+0000","last_became_peered":"2024-09-17T08:24:04.452780+0000","last_unstale":"2024-09-17T08:26:32.638348+0000","last_undegraded":"2024-09-17T08:26:32.638348+0000","last_fullsized":"2024-09-17T08:26:32.638348+0000","mapping_epoch":287,"log_start":"311'6600","ondisk_log_start":"311'6600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":333119488,"num_objects":5083,"num_object_clones":0,"num_object_copies":15249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":534272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":395,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.736993+0000","last_change":"2024-09-17T08:24:03.927221+0000","last_active":"2024-09-17T08:26:26.736993+0000","last_peered":"2024-09-17T08:26:26.736993+0000","last_clean":"2024-09-17T08:26:26.736993+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:26:26.736993+0000","last_undegraded":"2024-09-17T08:26:26.736993+0000","last_fullsized":"2024-09-17T08:26:26.736993+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:24:03.927109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.927109+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026636759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":236,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.725076+0000","last_change":"2024-09-17T08:25:31.165549+0000","last_active":"2024-09-17T08:26:26.725076+0000","last_peered":"2024-09-17T08:26:26.725076+0000","last_clean":"2024-09-17T08:26:26.725076+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:26:26.725076+0000","last_undegraded":"2024-09-17T08:26:26.725076+0000","last_fullsized":"2024-09-17T08:26:26.725076+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:31.165432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:25:31.165432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:36.847226+0000","scrub_duration":0.001729558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":353,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.726950+0000","last_change":"2024-09-17T08:26:23.149322+0000","last_active":"2024-09-17T08:26:26.726950+0000","last_peered":"2024-09-17T08:26:26.726950+0000","last_clean":"2024-09-17T08:26:26.726950+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:26:26.726950+0000","last_undegraded":"2024-09-17T08:26:26.726950+0000","last_fullsized":"2024-09-17T08:26:26.726950+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.149070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.149070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.469461+0000","scrub_duration":0.0050165310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":366,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723521+0000","last_change":"2024-09-17T08:25:39.349334+0000","last_active":"2024-09-17T08:26:26.723521+0000","last_peered":"2024-09-17T08:26:26.723521+0000","last_clean":"2024-09-17T08:26:26.723521+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:26:26.723521+0000","last_undegraded":"2024-09-17T08:26:26.723521+0000","last_fullsized":"2024-09-17T08:26:26.723521+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:39.349120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.349120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.525815+0000","scrub_duration":0.005278652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"328'11512","reported_seq":11564,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.674787+0000","last_change":"2024-09-17T08:24:04.456347+0000","last_active":"2024-09-17T08:26:32.674787+0000","last_peered":"2024-09-17T08:26:32.674787+0000","last_clean":"2024-09-17T08:26:32.674787+0000","last_became_active":"2024-09-17T08:24:04.456039+0000","last_became_peered":"2024-09-17T08:24:04.456039+0000","last_unstale":"2024-09-17T08:26:32.674787+0000","last_undegraded":"2024-09-17T08:26:32.674787+0000","last_fullsized":"2024-09-17T08:26:32.674787+0000","mapping_epoch":287,"log_start":"311'6500","ondisk_log_start":"311'6500","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326303768,"num_objects":4980,"num_object_clones":0,"num_object_copies":14940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11512,"num_write_kb":527681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":420,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.803081+0000","last_change":"2024-09-17T08:26:12.052601+0000","last_active":"2024-09-17T08:26:36.803081+0000","last_peered":"2024-09-17T08:26:36.803081+0000","last_clean":"2024-09-17T08:26:36.803081+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:26:36.803081+0000","last_undegraded":"2024-09-17T08:26:36.803081+0000","last_fullsized":"2024-09-17T08:26:36.803081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:26:12.052367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.052367+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.321528+0000","scrub_duration":0.0032600939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":373,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.789823+0000","last_change":"2024-09-17T08:26:27.789924+0000","last_active":"2024-09-17T08:26:27.789823+0000","last_peered":"2024-09-17T08:26:27.789823+0000","last_clean":"2024-09-17T08:26:27.789823+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:26:27.789823+0000","last_undegraded":"2024-09-17T08:26:27.789823+0000","last_fullsized":"2024-09-17T08:26:27.789823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:26:27.789796+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:27.956988+0000","scrub_duration":0.007393897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":311,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.779068+0000","last_change":"2024-09-17T08:24:15.691704+0000","last_active":"2024-09-17T08:26:34.779068+0000","last_peered":"2024-09-17T08:26:34.779068+0000","last_clean":"2024-09-17T08:26:34.779068+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:26:34.779068+0000","last_undegraded":"2024-09-17T08:26:34.779068+0000","last_fullsized":"2024-09-17T08:26:34.779068+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:15.691622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.691622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034962180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":251,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723581+0000","last_change":"2024-09-17T08:24:12.919326+0000","last_active":"2024-09-17T08:26:26.723581+0000","last_peered":"2024-09-17T08:26:26.723581+0000","last_clean":"2024-09-17T08:26:26.723581+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:26:26.723581+0000","last_undegraded":"2024-09-17T08:26:26.723581+0000","last_fullsized":"2024-09-17T08:26:26.723581+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:12.919247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.919247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013913059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"328'11628","reported_seq":11838,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.823775+0000","last_change":"2024-09-17T08:25:36.457603+0000","last_active":"2024-09-17T08:26:32.823775+0000","last_peered":"2024-09-17T08:26:32.823775+0000","last_clean":"2024-09-17T08:26:32.823775+0000","last_became_active":"2024-09-17T08:24:04.458943+0000","last_became_peered":"2024-09-17T08:24:04.458943+0000","last_unstale":"2024-09-17T08:26:32.823775+0000","last_undegraded":"2024-09-17T08:26:32.823775+0000","last_fullsized":"2024-09-17T08:26:32.823775+0000","mapping_epoch":287,"log_start":"311'6600","ondisk_log_start":"311'6600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'7478","last_scrub_stamp":"2024-09-17T08:25:36.457326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:36.457326+0000","objects_scrubbed":7398,"log_size":5028,"log_dups_size":0,"ondisk_log_size":5028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:50.045951+0000","scrub_duration":2.6174414289999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":5056,"num_object_clones":0,"num_object_copies":15168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11628,"num_write_kb":533888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":268,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723338+0000","last_change":"2024-09-17T08:24:30.840463+0000","last_active":"2024-09-17T08:26:26.723338+0000","last_peered":"2024-09-17T08:26:26.723338+0000","last_clean":"2024-09-17T08:26:26.723338+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:26:26.723338+0000","last_undegraded":"2024-09-17T08:26:26.723338+0000","last_fullsized":"2024-09-17T08:26:26.723338+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:24:30.840289+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.840289+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056047780000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724524+0000","last_change":"2024-09-17T08:25:14.210636+0000","last_active":"2024-09-17T08:26:26.724524+0000","last_peered":"2024-09-17T08:26:26.724524+0000","last_clean":"2024-09-17T08:26:26.724524+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:26:26.724524+0000","last_undegraded":"2024-09-17T08:26:26.724524+0000","last_fullsized":"2024-09-17T08:26:26.724524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.210528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.210528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.818087+0000","scrub_duration":0.0019364110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":575,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.777942+0000","last_change":"2024-09-17T08:26:22.193822+0000","last_active":"2024-09-17T08:26:34.777942+0000","last_peered":"2024-09-17T08:26:34.777942+0000","last_clean":"2024-09-17T08:26:34.777942+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:26:34.777942+0000","last_undegraded":"2024-09-17T08:26:34.777942+0000","last_fullsized":"2024-09-17T08:26:34.777942+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:26:22.193701+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.193701+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.638128+0000","scrub_duration":0.0030149489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":370,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.775439+0000","last_change":"2024-09-17T08:25:37.001934+0000","last_active":"2024-09-17T08:26:34.775439+0000","last_peered":"2024-09-17T08:26:34.775439+0000","last_clean":"2024-09-17T08:26:34.775439+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:26:34.775439+0000","last_undegraded":"2024-09-17T08:26:34.775439+0000","last_fullsized":"2024-09-17T08:26:34.775439+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:37.001438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.001438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:54.154447+0000","scrub_duration":0.0093143449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"330'11687","reported_seq":11861,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:37.340719+0000","last_change":"2024-09-17T08:26:25.894830+0000","last_active":"2024-09-17T08:26:37.340719+0000","last_peered":"2024-09-17T08:26:37.340719+0000","last_clean":"2024-09-17T08:26:37.340719+0000","last_became_active":"2024-09-17T08:24:04.457614+0000","last_became_peered":"2024-09-17T08:24:04.457614+0000","last_unstale":"2024-09-17T08:26:37.340719+0000","last_undegraded":"2024-09-17T08:26:37.340719+0000","last_fullsized":"2024-09-17T08:26:37.340719+0000","mapping_epoch":287,"log_start":"310'6400","ondisk_log_start":"310'6400","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10898","last_scrub_stamp":"2024-09-17T08:26:25.894587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.894587+0000","objects_scrubbed":5529,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.232827+0000","scrub_duration":1.670279176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306905088,"num_objects":4683,"num_object_clones":0,"num_object_copies":14049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11687,"num_write_kb":523840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":441,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:28.030273+0000","last_change":"2024-09-17T08:26:27.354029+0000","last_active":"2024-09-17T08:26:28.030273+0000","last_peered":"2024-09-17T08:26:28.030273+0000","last_clean":"2024-09-17T08:26:28.030273+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:26:28.030273+0000","last_undegraded":"2024-09-17T08:26:28.030273+0000","last_fullsized":"2024-09-17T08:26:28.030273+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:26:27.353856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:27.353856+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:57.299917+0000","scrub_duration":0.0088356479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":247,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723483+0000","last_change":"2024-09-17T08:26:24.311721+0000","last_active":"2024-09-17T08:26:26.723483+0000","last_peered":"2024-09-17T08:26:26.723483+0000","last_clean":"2024-09-17T08:26:26.723483+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:26:26.723483+0000","last_undegraded":"2024-09-17T08:26:26.723483+0000","last_fullsized":"2024-09-17T08:26:26.723483+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:24.311602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:26:24.311602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.713175+0000","scrub_duration":0.0041065229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":384,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.775855+0000","last_change":"2024-09-17T08:26:30.174985+0000","last_active":"2024-09-17T08:26:34.775855+0000","last_peered":"2024-09-17T08:26:34.775855+0000","last_clean":"2024-09-17T08:26:34.775855+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:26:34.775855+0000","last_undegraded":"2024-09-17T08:26:34.775855+0000","last_fullsized":"2024-09-17T08:26:34.775855+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.174489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.174489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:38.176927+0000","scrub_duration":0.0034970859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":351,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738932+0000","last_change":"2024-09-17T08:25:29.848581+0000","last_active":"2024-09-17T08:26:26.738932+0000","last_peered":"2024-09-17T08:26:26.738932+0000","last_clean":"2024-09-17T08:26:26.738932+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:26:26.738932+0000","last_undegraded":"2024-09-17T08:26:26.738932+0000","last_fullsized":"2024-09-17T08:26:26.738932+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.848449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.848449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:31.490576+0000","scrub_duration":0.0072796190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"329'11731","reported_seq":11920,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.655828+0000","last_change":"2024-09-17T08:26:18.800526+0000","last_active":"2024-09-17T08:26:36.655828+0000","last_peered":"2024-09-17T08:26:36.655828+0000","last_clean":"2024-09-17T08:26:36.655828+0000","last_became_active":"2024-09-17T08:24:04.453753+0000","last_became_peered":"2024-09-17T08:24:04.453753+0000","last_unstale":"2024-09-17T08:26:36.655828+0000","last_undegraded":"2024-09-17T08:26:36.655828+0000","last_fullsized":"2024-09-17T08:26:36.655828+0000","mapping_epoch":287,"log_start":"307'5300","ondisk_log_start":"307'5300","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10467","last_scrub_stamp":"2024-09-17T08:26:18.800219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.800219+0000","objects_scrubbed":6104,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.431424+0000","scrub_duration":2.2200420680000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312672256,"num_objects":4771,"num_object_clones":0,"num_object_copies":14313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":528064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":254,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727070+0000","last_change":"2024-09-17T08:25:19.161609+0000","last_active":"2024-09-17T08:26:26.727070+0000","last_peered":"2024-09-17T08:26:26.727070+0000","last_clean":"2024-09-17T08:26:26.727070+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:26:26.727070+0000","last_undegraded":"2024-09-17T08:26:26.727070+0000","last_fullsized":"2024-09-17T08:26:26.727070+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:25:19.161454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.161454+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015453119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":917,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:32.620455+0000","last_change":"2024-09-17T08:24:11.910215+0000","last_active":"2024-09-17T08:26:32.620455+0000","last_peered":"2024-09-17T08:26:32.620455+0000","last_clean":"2024-09-17T08:26:32.620455+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:26:32.620455+0000","last_undegraded":"2024-09-17T08:26:32.620455+0000","last_fullsized":"2024-09-17T08:26:32.620455+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:24:11.910131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:11.910131+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001356576,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":661,"num_read_kb":656,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":343,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.767111+0000","last_change":"2024-09-17T08:26:15.566206+0000","last_active":"2024-09-17T08:26:34.767111+0000","last_peered":"2024-09-17T08:26:34.767111+0000","last_clean":"2024-09-17T08:26:34.767111+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:26:34.767111+0000","last_undegraded":"2024-09-17T08:26:34.767111+0000","last_fullsized":"2024-09-17T08:26:34.767111+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.566090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.566090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:37.464727+0000","scrub_duration":0.0094002110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":350,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.577739+0000","last_change":"2024-09-17T08:26:36.581654+0000","last_active":"2024-09-17T08:26:36.577739+0000","last_peered":"2024-09-17T08:26:36.577739+0000","last_clean":"2024-09-17T08:26:36.577739+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:26:36.577739+0000","last_undegraded":"2024-09-17T08:26:36.577739+0000","last_fullsized":"2024-09-17T08:26:36.577739+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:36.577718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:26:36.577718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:04.804800+0000","scrub_duration":0.0075585510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":361,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730982+0000","last_change":"2024-09-17T08:25:18.912338+0000","last_active":"2024-09-17T08:26:26.730982+0000","last_peered":"2024-09-17T08:26:26.730982+0000","last_clean":"2024-09-17T08:26:26.730982+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:26:26.730982+0000","last_undegraded":"2024-09-17T08:26:26.730982+0000","last_fullsized":"2024-09-17T08:26:26.730982+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.912202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.912202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.360980+0000","scrub_duration":0.0058719879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"330'11982","reported_seq":12156,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:37.379446+0000","last_change":"2024-09-17T08:26:30.014820+0000","last_active":"2024-09-17T08:26:37.379446+0000","last_peered":"2024-09-17T08:26:37.379446+0000","last_clean":"2024-09-17T08:26:37.379446+0000","last_became_active":"2024-09-17T08:24:04.458862+0000","last_became_peered":"2024-09-17T08:24:04.458862+0000","last_unstale":"2024-09-17T08:26:37.379446+0000","last_undegraded":"2024-09-17T08:26:37.379446+0000","last_fullsized":"2024-09-17T08:26:37.379446+0000","mapping_epoch":287,"log_start":"312'6700","ondisk_log_start":"312'6700","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'11417","last_scrub_stamp":"2024-09-17T08:26:30.014628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.014628+0000","objects_scrubbed":5405,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:51.272301+0000","scrub_duration":1.851197124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313131008,"num_objects":4778,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11982,"num_write_kb":536320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":475,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.776970+0000","last_change":"2024-09-17T08:25:37.983837+0000","last_active":"2024-09-17T08:26:34.776970+0000","last_peered":"2024-09-17T08:26:34.776970+0000","last_clean":"2024-09-17T08:26:34.776970+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:26:34.776970+0000","last_undegraded":"2024-09-17T08:26:34.776970+0000","last_fullsized":"2024-09-17T08:26:34.776970+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:25:37.983609+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.983609+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.256761+0000","scrub_duration":0.0081599319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":378,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.802483+0000","last_change":"2024-09-17T08:25:17.027645+0000","last_active":"2024-09-17T08:26:36.802483+0000","last_peered":"2024-09-17T08:26:36.802483+0000","last_clean":"2024-09-17T08:26:36.802483+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:26:36.802483+0000","last_undegraded":"2024-09-17T08:26:36.802483+0000","last_fullsized":"2024-09-17T08:26:36.802483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:25:17.027377+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.027377+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.587148+0000","scrub_duration":0.0088147999999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":386,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.733130+0000","last_change":"2024-09-17T08:25:34.874991+0000","last_active":"2024-09-17T08:26:26.733130+0000","last_peered":"2024-09-17T08:26:26.733130+0000","last_clean":"2024-09-17T08:26:26.733130+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:26:26.733130+0000","last_undegraded":"2024-09-17T08:26:26.733130+0000","last_fullsized":"2024-09-17T08:26:26.733130+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.874848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.874848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.051778+0000","scrub_duration":0.0082107689999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":247,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727101+0000","last_change":"2024-09-17T08:26:23.359243+0000","last_active":"2024-09-17T08:26:26.727101+0000","last_peered":"2024-09-17T08:26:26.727101+0000","last_clean":"2024-09-17T08:26:26.727101+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:26:26.727101+0000","last_undegraded":"2024-09-17T08:26:26.727101+0000","last_fullsized":"2024-09-17T08:26:26.727101+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.359104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.359104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.887092+0000","scrub_duration":0.0035052579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":236,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724346+0000","last_change":"2024-09-17T08:24:28.859552+0000","last_active":"2024-09-17T08:26:26.724346+0000","last_peered":"2024-09-17T08:26:26.724346+0000","last_clean":"2024-09-17T08:26:26.724346+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:26:26.724346+0000","last_undegraded":"2024-09-17T08:26:26.724346+0000","last_fullsized":"2024-09-17T08:26:26.724346+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:28.859427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:28.859427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051469929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"328'11547","reported_seq":11598,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:33.797831+0000","last_change":"2024-09-17T08:24:04.456167+0000","last_active":"2024-09-17T08:26:33.797831+0000","last_peered":"2024-09-17T08:26:33.797831+0000","last_clean":"2024-09-17T08:26:33.797831+0000","last_became_active":"2024-09-17T08:24:04.455726+0000","last_became_peered":"2024-09-17T08:24:04.455726+0000","last_unstale":"2024-09-17T08:26:33.797831+0000","last_undegraded":"2024-09-17T08:26:33.797831+0000","last_fullsized":"2024-09-17T08:26:33.797831+0000","mapping_epoch":287,"log_start":"309'6000","ondisk_log_start":"309'6000","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5547,"log_dups_size":0,"ondisk_log_size":5547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320143360,"num_objects":4885,"num_object_clones":0,"num_object_copies":14655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":525824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":357,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.725317+0000","last_change":"2024-09-17T08:26:25.062996+0000","last_active":"2024-09-17T08:26:26.725317+0000","last_peered":"2024-09-17T08:26:26.725317+0000","last_clean":"2024-09-17T08:26:26.725317+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:26:26.725317+0000","last_undegraded":"2024-09-17T08:26:26.725317+0000","last_fullsized":"2024-09-17T08:26:26.725317+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:26:25.062850+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.062850+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.215968+0000","scrub_duration":0.0041814790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":353,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737838+0000","last_change":"2024-09-17T08:24:06.873073+0000","last_active":"2024-09-17T08:26:26.737838+0000","last_peered":"2024-09-17T08:26:26.737838+0000","last_clean":"2024-09-17T08:26:26.737838+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:26:26.737838+0000","last_undegraded":"2024-09-17T08:26:26.737838+0000","last_fullsized":"2024-09-17T08:26:26.737838+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:24:06.872995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:06.872995+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019197509999999999,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":241,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.770593+0000","last_change":"2024-09-17T08:25:42.221990+0000","last_active":"2024-09-17T08:26:34.770593+0000","last_peered":"2024-09-17T08:26:34.770593+0000","last_clean":"2024-09-17T08:26:34.770593+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:26:34.770593+0000","last_undegraded":"2024-09-17T08:26:34.770593+0000","last_fullsized":"2024-09-17T08:26:34.770593+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:42.221590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:25:42.221590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.243498+0000","scrub_duration":0.005773388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":629,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.733278+0000","last_change":"2024-09-17T08:26:25.775622+0000","last_active":"2024-09-17T08:26:26.733278+0000","last_peered":"2024-09-17T08:26:26.733278+0000","last_clean":"2024-09-17T08:26:26.733278+0000","last_became_active":"2024-09-17T08:09:45.227728+0000","last_became_peered":"2024-09-17T08:09:45.227728+0000","last_unstale":"2024-09-17T08:26:26.733278+0000","last_undegraded":"2024-09-17T08:26:26.733278+0000","last_fullsized":"2024-09-17T08:26:26.733278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.557225+0000","scrub_duration":0.012846286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":367,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.731477+0000","last_change":"2024-09-17T08:25:38.817905+0000","last_active":"2024-09-17T08:26:26.731477+0000","last_peered":"2024-09-17T08:26:26.731477+0000","last_clean":"2024-09-17T08:26:26.731477+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:26:26.731477+0000","last_undegraded":"2024-09-17T08:26:26.731477+0000","last_fullsized":"2024-09-17T08:26:26.731477+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.817546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:04.248082+0000","scrub_duration":0.002079291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":256,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.741111+0000","last_change":"2024-09-17T08:25:41.900666+0000","last_active":"2024-09-17T08:26:26.741111+0000","last_peered":"2024-09-17T08:26:26.741111+0000","last_clean":"2024-09-17T08:26:26.741111+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:26:26.741111+0000","last_undegraded":"2024-09-17T08:26:26.741111+0000","last_fullsized":"2024-09-17T08:26:26.741111+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.900483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.900483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.112986+0000","scrub_duration":0.013601894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"330'11823","reported_seq":11875,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:37.364145+0000","last_change":"2024-09-17T08:24:04.458040+0000","last_active":"2024-09-17T08:26:37.364145+0000","last_peered":"2024-09-17T08:26:37.364145+0000","last_clean":"2024-09-17T08:26:37.364145+0000","last_became_active":"2024-09-17T08:24:04.457745+0000","last_became_peered":"2024-09-17T08:24:04.457745+0000","last_unstale":"2024-09-17T08:26:37.364145+0000","last_undegraded":"2024-09-17T08:26:37.364145+0000","last_fullsized":"2024-09-17T08:26:37.364145+0000","mapping_epoch":287,"log_start":"311'6500","ondisk_log_start":"311'6500","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5323,"log_dups_size":0,"ondisk_log_size":5323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309395456,"num_objects":4721,"num_object_clones":0,"num_object_copies":14163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11823,"num_write_kb":529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":194,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723553+0000","last_change":"2024-09-17T08:25:35.125962+0000","last_active":"2024-09-17T08:26:26.723553+0000","last_peered":"2024-09-17T08:26:26.723553+0000","last_clean":"2024-09-17T08:26:26.723553+0000","last_became_active":"2024-09-17T08:23:18.670977+0000","last_became_peered":"2024-09-17T08:23:18.670977+0000","last_unstale":"2024-09-17T08:26:26.723553+0000","last_undegraded":"2024-09-17T08:26:26.723553+0000","last_fullsized":"2024-09-17T08:26:26.723553+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:25:35.125100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:35.125100+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.214432+0000","scrub_duration":0.0023912690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":331,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:29.545228+0000","last_change":"2024-09-17T08:26:29.545228+0000","last_active":"2024-09-17T08:26:29.545228+0000","last_peered":"2024-09-17T08:26:29.545228+0000","last_clean":"2024-09-17T08:26:29.545228+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:26:29.545228+0000","last_undegraded":"2024-09-17T08:26:29.545228+0000","last_fullsized":"2024-09-17T08:26:29.545228+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:29.545114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:29.545114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:32.014523+0000","scrub_duration":0.0047629450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":337,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.742242+0000","last_change":"2024-09-17T08:25:13.057707+0000","last_active":"2024-09-17T08:26:27.742242+0000","last_peered":"2024-09-17T08:26:27.742242+0000","last_clean":"2024-09-17T08:26:27.742242+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:26:27.742242+0000","last_undegraded":"2024-09-17T08:26:27.742242+0000","last_fullsized":"2024-09-17T08:26:27.742242+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:13.057534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.057534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.726370+0000","scrub_duration":0.011530043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":365,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.733202+0000","last_change":"2024-09-17T08:26:15.680685+0000","last_active":"2024-09-17T08:26:26.733202+0000","last_peered":"2024-09-17T08:26:26.733202+0000","last_clean":"2024-09-17T08:26:26.733202+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:26:26.733202+0000","last_undegraded":"2024-09-17T08:26:26.733202+0000","last_fullsized":"2024-09-17T08:26:26.733202+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.680565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.680565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:34.928434+0000","scrub_duration":0.0076375050000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"329'11819","reported_seq":12012,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:36.623082+0000","last_change":"2024-09-17T08:26:17.151968+0000","last_active":"2024-09-17T08:26:36.623082+0000","last_peered":"2024-09-17T08:26:36.623082+0000","last_clean":"2024-09-17T08:26:36.623082+0000","last_became_active":"2024-09-17T08:24:04.458851+0000","last_became_peered":"2024-09-17T08:24:04.458851+0000","last_unstale":"2024-09-17T08:26:36.623082+0000","last_undegraded":"2024-09-17T08:26:36.623082+0000","last_fullsized":"2024-09-17T08:26:36.623082+0000","mapping_epoch":287,"log_start":"307'5400","ondisk_log_start":"307'5400","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10426","last_scrub_stamp":"2024-09-17T08:26:17.151829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.151829+0000","objects_scrubbed":6271,"log_size":6419,"log_dups_size":0,"ondisk_log_size":6419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.276228+0000","scrub_duration":2.6491713260000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314245120,"num_objects":4795,"num_object_clones":0,"num_object_copies":14385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11819,"num_write_kb":531648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":420,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.778026+0000","last_change":"2024-09-17T08:25:16.078092+0000","last_active":"2024-09-17T08:26:34.778026+0000","last_peered":"2024-09-17T08:26:34.778026+0000","last_clean":"2024-09-17T08:26:34.778026+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:26:34.778026+0000","last_undegraded":"2024-09-17T08:26:34.778026+0000","last_fullsized":"2024-09-17T08:26:34.778026+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:25:16.077960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:16.077960+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:44.021838+0000","scrub_duration":0.021317215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":246,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724654+0000","last_change":"2024-09-17T08:26:17.341373+0000","last_active":"2024-09-17T08:26:26.724654+0000","last_peered":"2024-09-17T08:26:26.724654+0000","last_clean":"2024-09-17T08:26:26.724654+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:26:26.724654+0000","last_undegraded":"2024-09-17T08:26:26.724654+0000","last_fullsized":"2024-09-17T08:26:26.724654+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.341260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.341260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.774271+0000","scrub_duration":0.0033668219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":351,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:31.574646+0000","last_change":"2024-09-17T08:26:31.574646+0000","last_active":"2024-09-17T08:26:31.574646+0000","last_peered":"2024-09-17T08:26:31.574646+0000","last_clean":"2024-09-17T08:26:31.574646+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:26:31.574646+0000","last_undegraded":"2024-09-17T08:26:31.574646+0000","last_fullsized":"2024-09-17T08:26:31.574646+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:31.574528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:26:31.574528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:40.768540+0000","scrub_duration":0.0037572360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":368,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.777043+0000","last_change":"2024-09-17T08:25:40.906564+0000","last_active":"2024-09-17T08:26:34.777043+0000","last_peered":"2024-09-17T08:26:34.777043+0000","last_clean":"2024-09-17T08:26:34.777043+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:26:34.777043+0000","last_undegraded":"2024-09-17T08:26:34.777043+0000","last_fullsized":"2024-09-17T08:26:34.777043+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.906437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.906437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.936477+0000","scrub_duration":0.014127516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"330'11793","reported_seq":11846,"reported_epoch":330,"state":"active+clean","last_fresh":"2024-09-17T08:26:37.371596+0000","last_change":"2024-09-17T08:24:04.458318+0000","last_active":"2024-09-17T08:26:37.371596+0000","last_peered":"2024-09-17T08:26:37.371596+0000","last_clean":"2024-09-17T08:26:37.371596+0000","last_became_active":"2024-09-17T08:24:04.458042+0000","last_became_peered":"2024-09-17T08:24:04.458042+0000","last_unstale":"2024-09-17T08:26:37.371596+0000","last_undegraded":"2024-09-17T08:26:37.371596+0000","last_fullsized":"2024-09-17T08:26:37.371596+0000","mapping_epoch":287,"log_start":"311'6500","ondisk_log_start":"311'6500","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5293,"log_dups_size":0,"ondisk_log_size":5293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":299302912,"num_objects":4567,"num_object_clones":0,"num_object_copies":13701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11793,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":410,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.732887+0000","last_change":"2024-09-17T08:25:13.823972+0000","last_active":"2024-09-17T08:26:26.732887+0000","last_peered":"2024-09-17T08:26:26.732887+0000","last_clean":"2024-09-17T08:26:26.732887+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:26:26.732887+0000","last_undegraded":"2024-09-17T08:26:26.732887+0000","last_fullsized":"2024-09-17T08:26:26.732887+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:13.823819+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.823819+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.592729+0000","scrub_duration":0.0031292669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":352,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.734945+0000","last_change":"2024-09-17T08:25:27.161139+0000","last_active":"2024-09-17T08:26:27.734945+0000","last_peered":"2024-09-17T08:26:27.734945+0000","last_clean":"2024-09-17T08:26:27.734945+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:26:27.734945+0000","last_undegraded":"2024-09-17T08:26:27.734945+0000","last_fullsized":"2024-09-17T08:26:27.734945+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:27.160999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:27.160999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:42.471047+0000","scrub_duration":0.0058695600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":560,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.731881+0000","last_change":"2024-09-17T08:25:17.907507+0000","last_active":"2024-09-17T08:26:27.731881+0000","last_peered":"2024-09-17T08:26:27.731881+0000","last_clean":"2024-09-17T08:26:27.731881+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:26:27.731881+0000","last_undegraded":"2024-09-17T08:26:27.731881+0000","last_fullsized":"2024-09-17T08:26:27.731881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:25:17.907373+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.907373+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.869934+0000","scrub_duration":0.001744106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":206,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.778998+0000","last_change":"2024-09-17T08:26:32.132061+0000","last_active":"2024-09-17T08:26:34.778998+0000","last_peered":"2024-09-17T08:26:34.778998+0000","last_clean":"2024-09-17T08:26:34.778998+0000","last_became_active":"2024-09-17T08:23:18.662935+0000","last_became_peered":"2024-09-17T08:23:18.662935+0000","last_unstale":"2024-09-17T08:26:34.778998+0000","last_undegraded":"2024-09-17T08:26:34.778998+0000","last_fullsized":"2024-09-17T08:26:34.778998+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:32.131951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:26:32.131951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:45.669382+0000","scrub_duration":0.0022176209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"328'11800","reported_seq":11851,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:33.686464+0000","last_change":"2024-09-17T08:24:04.453548+0000","last_active":"2024-09-17T08:26:33.686464+0000","last_peered":"2024-09-17T08:26:33.686464+0000","last_clean":"2024-09-17T08:26:33.686464+0000","last_became_active":"2024-09-17T08:24:04.453268+0000","last_became_peered":"2024-09-17T08:24:04.453268+0000","last_unstale":"2024-09-17T08:26:33.686464+0000","last_undegraded":"2024-09-17T08:26:33.686464+0000","last_fullsized":"2024-09-17T08:26:33.686464+0000","mapping_epoch":287,"log_start":"311'6600","ondisk_log_start":"311'6600","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5200,"log_dups_size":0,"ondisk_log_size":5200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322437120,"num_objects":4920,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11800,"num_write_kb":535040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":429,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727019+0000","last_change":"2024-09-17T08:25:29.258817+0000","last_active":"2024-09-17T08:26:26.727019+0000","last_peered":"2024-09-17T08:26:26.727019+0000","last_clean":"2024-09-17T08:26:26.727019+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:26:26.727019+0000","last_undegraded":"2024-09-17T08:26:26.727019+0000","last_fullsized":"2024-09-17T08:26:26.727019+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:25:29.258616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.258616+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:40.967934+0000","scrub_duration":0.011223878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":364,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723882+0000","last_change":"2024-09-17T08:24:32.450419+0000","last_active":"2024-09-17T08:26:26.723882+0000","last_peered":"2024-09-17T08:26:26.723882+0000","last_clean":"2024-09-17T08:26:26.723882+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:26:26.723882+0000","last_undegraded":"2024-09-17T08:26:26.723882+0000","last_fullsized":"2024-09-17T08:26:26.723882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:24:32.450300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.450300+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058717939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":356,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.736697+0000","last_change":"2024-09-17T08:25:25.116593+0000","last_active":"2024-09-17T08:26:27.736697+0000","last_peered":"2024-09-17T08:26:27.736697+0000","last_clean":"2024-09-17T08:26:27.736697+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:26:27.736697+0000","last_undegraded":"2024-09-17T08:26:27.736697+0000","last_fullsized":"2024-09-17T08:26:27.736697+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:25.116441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:25:25.116441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.015212+0000","scrub_duration":0.008148832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":171,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:30.599024+0000","last_change":"2024-09-17T08:26:30.599024+0000","last_active":"2024-09-17T08:26:30.599024+0000","last_peered":"2024-09-17T08:26:30.599024+0000","last_clean":"2024-09-17T08:26:30.599024+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:26:30.599024+0000","last_undegraded":"2024-09-17T08:26:30.599024+0000","last_fullsized":"2024-09-17T08:26:30.599024+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.598835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.598835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.854877+0000","scrub_duration":0.0038840160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":385,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.776889+0000","last_change":"2024-09-17T08:25:33.033204+0000","last_active":"2024-09-17T08:26:34.776889+0000","last_peered":"2024-09-17T08:26:34.776889+0000","last_clean":"2024-09-17T08:26:34.776889+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:26:34.776889+0000","last_undegraded":"2024-09-17T08:26:34.776889+0000","last_fullsized":"2024-09-17T08:26:34.776889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.032722+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.145927+0000","scrub_duration":0.0085113229999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":370,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.736767+0000","last_change":"2024-09-17T08:25:34.280542+0000","last_active":"2024-09-17T08:26:27.736767+0000","last_peered":"2024-09-17T08:26:27.736767+0000","last_clean":"2024-09-17T08:26:27.736767+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:26:27.736767+0000","last_undegraded":"2024-09-17T08:26:27.736767+0000","last_fullsized":"2024-09-17T08:26:27.736767+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.278744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.278744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.492017+0000","scrub_duration":0.011508979000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":359,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.741350+0000","last_change":"2024-09-17T08:25:32.809559+0000","last_active":"2024-09-17T08:26:26.741350+0000","last_peered":"2024-09-17T08:26:26.741350+0000","last_clean":"2024-09-17T08:26:26.741350+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:26:26.741350+0000","last_undegraded":"2024-09-17T08:26:26.741350+0000","last_fullsized":"2024-09-17T08:26:26.741350+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:32.809305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:32.809305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:33.753241+0000","scrub_duration":0.0024200179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":343,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.734896+0000","last_change":"2024-09-17T08:26:11.533286+0000","last_active":"2024-09-17T08:26:27.734896+0000","last_peered":"2024-09-17T08:26:27.734896+0000","last_clean":"2024-09-17T08:26:27.734896+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:26:27.734896+0000","last_undegraded":"2024-09-17T08:26:27.734896+0000","last_fullsized":"2024-09-17T08:26:27.734896+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.533066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.533066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.390854+0000","scrub_duration":0.0073762410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":253,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727181+0000","last_change":"2024-09-17T08:25:18.296524+0000","last_active":"2024-09-17T08:26:26.727181+0000","last_peered":"2024-09-17T08:26:26.727181+0000","last_clean":"2024-09-17T08:26:26.727181+0000","last_became_active":"2024-09-17T08:23:18.666495+0000","last_became_peered":"2024-09-17T08:23:18.666495+0000","last_unstale":"2024-09-17T08:26:26.727181+0000","last_undegraded":"2024-09-17T08:26:26.727181+0000","last_fullsized":"2024-09-17T08:26:26.727181+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.296432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.296432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:41.437556+0000","scrub_duration":0.0041815000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":406,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.741465+0000","last_change":"2024-09-17T08:26:17.976881+0000","last_active":"2024-09-17T08:26:26.741465+0000","last_peered":"2024-09-17T08:26:26.741465+0000","last_clean":"2024-09-17T08:26:26.741465+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:26:26.741465+0000","last_undegraded":"2024-09-17T08:26:26.741465+0000","last_fullsized":"2024-09-17T08:26:26.741465+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:26:17.976320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.976320+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.774813+0000","scrub_duration":0.0092893760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":262,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738862+0000","last_change":"2024-09-17T08:24:19.785545+0000","last_active":"2024-09-17T08:26:26.738862+0000","last_peered":"2024-09-17T08:26:26.738862+0000","last_clean":"2024-09-17T08:26:26.738862+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:26:26.738862+0000","last_undegraded":"2024-09-17T08:26:26.738862+0000","last_fullsized":"2024-09-17T08:26:26.738862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:19.785421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.785421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005223391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":360,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730903+0000","last_change":"2024-09-17T08:25:20.873823+0000","last_active":"2024-09-17T08:26:26.730903+0000","last_peered":"2024-09-17T08:26:26.730903+0000","last_clean":"2024-09-17T08:26:26.730903+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:26:26.730903+0000","last_undegraded":"2024-09-17T08:26:26.730903+0000","last_fullsized":"2024-09-17T08:26:26.730903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:20.873748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:20.873748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001209087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":416,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738334+0000","last_change":"2024-09-17T08:24:09.894035+0000","last_active":"2024-09-17T08:26:26.738334+0000","last_peered":"2024-09-17T08:26:26.738334+0000","last_clean":"2024-09-17T08:26:26.738334+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:26:26.738334+0000","last_undegraded":"2024-09-17T08:26:26.738334+0000","last_fullsized":"2024-09-17T08:26:26.738334+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:09.893951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:09.893951+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019165969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":302,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737753+0000","last_change":"2024-09-17T08:26:16.948193+0000","last_active":"2024-09-17T08:26:26.737753+0000","last_peered":"2024-09-17T08:26:26.737753+0000","last_clean":"2024-09-17T08:26:26.737753+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:26:26.737753+0000","last_undegraded":"2024-09-17T08:26:26.737753+0000","last_fullsized":"2024-09-17T08:26:26.737753+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.948072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.948072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.892623+0000","scrub_duration":0.0075166590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":374,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:28.030159+0000","last_change":"2024-09-17T08:25:11.139098+0000","last_active":"2024-09-17T08:26:28.030159+0000","last_peered":"2024-09-17T08:26:28.030159+0000","last_clean":"2024-09-17T08:26:28.030159+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:26:28.030159+0000","last_undegraded":"2024-09-17T08:26:28.030159+0000","last_fullsized":"2024-09-17T08:26:28.030159+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:25:11.138956+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.138956+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004930266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":352,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.777058+0000","last_change":"2024-09-17T08:24:18.708208+0000","last_active":"2024-09-17T08:26:34.777058+0000","last_peered":"2024-09-17T08:26:34.777058+0000","last_clean":"2024-09-17T08:26:34.777058+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:26:34.777058+0000","last_undegraded":"2024-09-17T08:26:34.777058+0000","last_fullsized":"2024-09-17T08:26:34.777058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:18.708083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:18.708083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010815099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":307,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.731155+0000","last_change":"2024-09-17T08:26:13.666735+0000","last_active":"2024-09-17T08:26:26.731155+0000","last_peered":"2024-09-17T08:26:26.731155+0000","last_clean":"2024-09-17T08:26:26.731155+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:26:26.731155+0000","last_undegraded":"2024-09-17T08:26:26.731155+0000","last_fullsized":"2024-09-17T08:26:26.731155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:13.666621+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.666621+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:16.247569+0000","scrub_duration":0.0059880139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":359,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.779705+0000","last_change":"2024-09-17T08:25:15.032652+0000","last_active":"2024-09-17T08:26:34.779705+0000","last_peered":"2024-09-17T08:26:34.779705+0000","last_clean":"2024-09-17T08:26:34.779705+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:26:34.779705+0000","last_undegraded":"2024-09-17T08:26:34.779705+0000","last_fullsized":"2024-09-17T08:26:34.779705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:15.032525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:25:15.032525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013962988000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":568,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:28.036441+0000","last_change":"2024-09-17T08:25:23.783490+0000","last_active":"2024-09-17T08:26:28.036441+0000","last_peered":"2024-09-17T08:26:28.036441+0000","last_clean":"2024-09-17T08:26:28.036441+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:26:28.036441+0000","last_undegraded":"2024-09-17T08:26:28.036441+0000","last_fullsized":"2024-09-17T08:26:28.036441+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:23.783353+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.783353+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:37.268781+0000","scrub_duration":0.0064248070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":355,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:31.188177+0000","last_change":"2024-09-17T08:26:31.188294+0000","last_active":"2024-09-17T08:26:31.188177+0000","last_peered":"2024-09-17T08:26:31.188177+0000","last_clean":"2024-09-17T08:26:31.188177+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:26:31.188177+0000","last_undegraded":"2024-09-17T08:26:31.188177+0000","last_fullsized":"2024-09-17T08:26:31.188177+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:31.188153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:31.188153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:57.198487+0000","scrub_duration":0.0090076900000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":267,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723413+0000","last_change":"2024-09-17T08:24:15.898072+0000","last_active":"2024-09-17T08:26:26.723413+0000","last_peered":"2024-09-17T08:26:26.723413+0000","last_clean":"2024-09-17T08:26:26.723413+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:26:26.723413+0000","last_undegraded":"2024-09-17T08:26:26.723413+0000","last_fullsized":"2024-09-17T08:26:26.723413+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:24:15.897984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.897984+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018990789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":234,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738402+0000","last_change":"2024-09-17T08:24:13.873686+0000","last_active":"2024-09-17T08:26:26.738402+0000","last_peered":"2024-09-17T08:26:26.738402+0000","last_clean":"2024-09-17T08:26:26.738402+0000","last_became_active":"2024-09-17T08:23:18.671502+0000","last_became_peered":"2024-09-17T08:23:18.671502+0000","last_unstale":"2024-09-17T08:26:26.738402+0000","last_undegraded":"2024-09-17T08:26:26.738402+0000","last_fullsized":"2024-09-17T08:26:26.738402+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.873601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.873601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016156409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":360,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.740591+0000","last_change":"2024-09-17T08:24:10.894915+0000","last_active":"2024-09-17T08:26:26.740591+0000","last_peered":"2024-09-17T08:26:26.740591+0000","last_clean":"2024-09-17T08:26:26.740591+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:26:26.740591+0000","last_undegraded":"2024-09-17T08:26:26.740591+0000","last_fullsized":"2024-09-17T08:26:26.740591+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.894828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.894828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001103185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":371,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.775353+0000","last_change":"2024-09-17T08:24:08.507584+0000","last_active":"2024-09-17T08:26:34.775353+0000","last_peered":"2024-09-17T08:26:34.775353+0000","last_clean":"2024-09-17T08:26:34.775353+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:26:34.775353+0000","last_undegraded":"2024-09-17T08:26:34.775353+0000","last_fullsized":"2024-09-17T08:26:34.775353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:08.507505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:08.507505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014868570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":429,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724670+0000","last_change":"2024-09-17T08:25:33.258559+0000","last_active":"2024-09-17T08:26:26.724670+0000","last_peered":"2024-09-17T08:26:26.724670+0000","last_clean":"2024-09-17T08:26:26.724670+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:26:26.724670+0000","last_undegraded":"2024-09-17T08:26:26.724670+0000","last_fullsized":"2024-09-17T08:26:26.724670+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.258469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.258469+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.181972+0000","scrub_duration":0.0016288769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724846+0000","last_change":"2024-09-17T08:25:30.191078+0000","last_active":"2024-09-17T08:26:26.724846+0000","last_peered":"2024-09-17T08:26:26.724846+0000","last_clean":"2024-09-17T08:26:26.724846+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:26:26.724846+0000","last_undegraded":"2024-09-17T08:26:26.724846+0000","last_fullsized":"2024-09-17T08:26:26.724846+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:30.190970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:30.190970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:59.517112+0000","scrub_duration":0.0016272669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.725405+0000","last_change":"2024-09-17T08:26:26.350432+0000","last_active":"2024-09-17T08:26:26.725405+0000","last_peered":"2024-09-17T08:26:26.725405+0000","last_clean":"2024-09-17T08:26:26.725405+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:26:26.725405+0000","last_undegraded":"2024-09-17T08:26:26.725405+0000","last_fullsized":"2024-09-17T08:26:26.725405+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:26.349968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:26.349968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:29.549108+0000","scrub_duration":0.004346824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":340,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.736925+0000","last_change":"2024-09-17T08:24:07.897133+0000","last_active":"2024-09-17T08:26:26.736925+0000","last_peered":"2024-09-17T08:26:26.736925+0000","last_clean":"2024-09-17T08:26:26.736925+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:26:26.736925+0000","last_undegraded":"2024-09-17T08:26:26.736925+0000","last_fullsized":"2024-09-17T08:26:26.736925+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:07.897050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:07.897050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015097719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":69,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.713727+0000","last_change":"2024-09-17T08:24:26.616991+0000","last_active":"2024-09-17T08:26:27.713727+0000","last_peered":"2024-09-17T08:26:27.713727+0000","last_clean":"2024-09-17T08:26:27.713727+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:26:27.713727+0000","last_undegraded":"2024-09-17T08:26:27.713727+0000","last_fullsized":"2024-09-17T08:26:27.713727+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:24:26.616854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.616854+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002706263,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":230,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.734829+0000","last_change":"2024-09-17T08:25:24.143568+0000","last_active":"2024-09-17T08:26:27.734829+0000","last_peered":"2024-09-17T08:26:27.734829+0000","last_clean":"2024-09-17T08:26:27.734829+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:26:27.734829+0000","last_undegraded":"2024-09-17T08:26:27.734829+0000","last_fullsized":"2024-09-17T08:26:27.734829+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:24.143382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:24.143382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085248119999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.726887+0000","last_change":"2024-09-17T08:25:41.337008+0000","last_active":"2024-09-17T08:26:26.726887+0000","last_peered":"2024-09-17T08:26:26.726887+0000","last_clean":"2024-09-17T08:26:26.726887+0000","last_became_active":"2024-09-17T08:23:18.671271+0000","last_became_peered":"2024-09-17T08:23:18.671271+0000","last_unstale":"2024-09-17T08:26:26.726887+0000","last_undegraded":"2024-09-17T08:26:26.726887+0000","last_fullsized":"2024-09-17T08:26:26.726887+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.336771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.336771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:01.147261+0000","scrub_duration":0.004352692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":365,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.730108+0000","last_change":"2024-09-17T08:25:26.873461+0000","last_active":"2024-09-17T08:26:26.730108+0000","last_peered":"2024-09-17T08:26:26.730108+0000","last_clean":"2024-09-17T08:26:26.730108+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:26:26.730108+0000","last_undegraded":"2024-09-17T08:26:26.730108+0000","last_fullsized":"2024-09-17T08:26:26.730108+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:25:26.873378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:30.470812+0000","scrub_duration":0.0028675900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":375,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737107+0000","last_change":"2024-09-17T08:24:21.848684+0000","last_active":"2024-09-17T08:26:26.737107+0000","last_peered":"2024-09-17T08:26:26.737107+0000","last_clean":"2024-09-17T08:26:26.737107+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:26:26.737107+0000","last_undegraded":"2024-09-17T08:26:26.737107+0000","last_fullsized":"2024-09-17T08:26:26.737107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:21.848510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.848510+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030240649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":213,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.735492+0000","last_change":"2024-09-17T08:26:22.498965+0000","last_active":"2024-09-17T08:26:27.735492+0000","last_peered":"2024-09-17T08:26:27.735492+0000","last_clean":"2024-09-17T08:26:27.735492+0000","last_became_active":"2024-09-17T08:23:18.669127+0000","last_became_peered":"2024-09-17T08:23:18.669127+0000","last_unstale":"2024-09-17T08:26:27.735492+0000","last_undegraded":"2024-09-17T08:26:27.735492+0000","last_fullsized":"2024-09-17T08:26:27.735492+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:26:22.498827+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.498827+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:38.115530+0000","scrub_duration":0.011342636999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":363,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.732689+0000","last_change":"2024-09-17T08:26:17.717152+0000","last_active":"2024-09-17T08:26:26.732689+0000","last_peered":"2024-09-17T08:26:26.732689+0000","last_clean":"2024-09-17T08:26:26.732689+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:26:26.732689+0000","last_undegraded":"2024-09-17T08:26:26.732689+0000","last_fullsized":"2024-09-17T08:26:26.732689+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.717066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.717066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:31.062923+0000","scrub_duration":0.0015956360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":340,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.740156+0000","last_change":"2024-09-17T08:24:29.919412+0000","last_active":"2024-09-17T08:26:27.740156+0000","last_peered":"2024-09-17T08:26:27.740156+0000","last_clean":"2024-09-17T08:26:27.740156+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:26:27.740156+0000","last_undegraded":"2024-09-17T08:26:27.740156+0000","last_fullsized":"2024-09-17T08:26:27.740156+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:29.919294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.919294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048696069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":366,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.731408+0000","last_change":"2024-09-17T08:25:22.892701+0000","last_active":"2024-09-17T08:26:26.731408+0000","last_peered":"2024-09-17T08:26:26.731408+0000","last_clean":"2024-09-17T08:26:26.731408+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:26:26.731408+0000","last_undegraded":"2024-09-17T08:26:26.731408+0000","last_fullsized":"2024-09-17T08:26:26.731408+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:25:22.892571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.892571+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:32.568284+0000","scrub_duration":0.0024767180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":256,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723449+0000","last_change":"2024-09-17T08:26:11.160161+0000","last_active":"2024-09-17T08:26:26.723449+0000","last_peered":"2024-09-17T08:26:26.723449+0000","last_clean":"2024-09-17T08:26:26.723449+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:26:26.723449+0000","last_undegraded":"2024-09-17T08:26:26.723449+0000","last_fullsized":"2024-09-17T08:26:26.723449+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.159902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.159902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.996888+0000","scrub_duration":0.011043470999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":388,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.776761+0000","last_change":"2024-09-17T08:24:19.731683+0000","last_active":"2024-09-17T08:26:34.776761+0000","last_peered":"2024-09-17T08:26:34.776761+0000","last_clean":"2024-09-17T08:26:34.776761+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:26:34.776761+0000","last_undegraded":"2024-09-17T08:26:34.776761+0000","last_fullsized":"2024-09-17T08:26:34.776761+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:19.731549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.731549+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011256499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":485,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.729423+0000","last_change":"2024-09-17T08:26:21.131000+0000","last_active":"2024-09-17T08:26:27.729423+0000","last_peered":"2024-09-17T08:26:27.729423+0000","last_clean":"2024-09-17T08:26:27.729423+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:26:27.729423+0000","last_undegraded":"2024-09-17T08:26:27.729423+0000","last_fullsized":"2024-09-17T08:26:27.729423+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:26:21.130873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.130873+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:23.349562+0000","scrub_duration":0.012556618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":246,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724703+0000","last_change":"2024-09-17T08:26:21.302147+0000","last_active":"2024-09-17T08:26:26.724703+0000","last_peered":"2024-09-17T08:26:26.724703+0000","last_clean":"2024-09-17T08:26:26.724703+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:26:26.724703+0000","last_undegraded":"2024-09-17T08:26:26.724703+0000","last_fullsized":"2024-09-17T08:26:26.724703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:21.302035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.302035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:50.505197+0000","scrub_duration":0.014448862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":345,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.737395+0000","last_change":"2024-09-17T08:24:04.885841+0000","last_active":"2024-09-17T08:26:26.737395+0000","last_peered":"2024-09-17T08:26:26.737395+0000","last_clean":"2024-09-17T08:26:26.737395+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:26:26.737395+0000","last_undegraded":"2024-09-17T08:26:26.737395+0000","last_fullsized":"2024-09-17T08:26:26.737395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:04.885747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:24:04.885747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001511376,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":170,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.727027+0000","last_change":"2024-09-17T08:26:14.336368+0000","last_active":"2024-09-17T08:26:26.727027+0000","last_peered":"2024-09-17T08:26:26.727027+0000","last_clean":"2024-09-17T08:26:26.727027+0000","last_became_active":"2024-09-17T08:23:18.676257+0000","last_became_peered":"2024-09-17T08:23:18.676257+0000","last_unstale":"2024-09-17T08:26:26.727027+0000","last_undegraded":"2024-09-17T08:26:26.727027+0000","last_fullsized":"2024-09-17T08:26:26.727027+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:14.336205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:14.336205+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.735226+0000","scrub_duration":0.0072317960000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":384,"reported_epoch":329,"state":"active+clean","last_fresh":"2024-09-17T08:26:34.776264+0000","last_change":"2024-09-17T08:26:16.157165+0000","last_active":"2024-09-17T08:26:34.776264+0000","last_peered":"2024-09-17T08:26:34.776264+0000","last_clean":"2024-09-17T08:26:34.776264+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:26:34.776264+0000","last_undegraded":"2024-09-17T08:26:34.776264+0000","last_fullsized":"2024-09-17T08:26:34.776264+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.157047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.157047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.892586+0000","scrub_duration":0.052471214000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":341,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724297+0000","last_change":"2024-09-17T08:25:19.293077+0000","last_active":"2024-09-17T08:26:26.724297+0000","last_peered":"2024-09-17T08:26:26.724297+0000","last_clean":"2024-09-17T08:26:26.724297+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:26:26.724297+0000","last_undegraded":"2024-09-17T08:26:26.724297+0000","last_fullsized":"2024-09-17T08:26:26.724297+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:19.292858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.292858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013986980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":308,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723662+0000","last_change":"2024-09-17T08:26:20.113140+0000","last_active":"2024-09-17T08:26:26.723662+0000","last_peered":"2024-09-17T08:26:26.723662+0000","last_clean":"2024-09-17T08:26:26.723662+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:26:26.723662+0000","last_undegraded":"2024-09-17T08:26:26.723662+0000","last_fullsized":"2024-09-17T08:26:26.723662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:20.113014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:26:20.113014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:29.947022+0000","scrub_duration":0.0052786329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":463,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-17T08:26:27.734966+0000","last_change":"2024-09-17T08:24:23.759585+0000","last_active":"2024-09-17T08:26:27.734966+0000","last_peered":"2024-09-17T08:26:27.734966+0000","last_clean":"2024-09-17T08:26:27.734966+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:26:27.734966+0000","last_undegraded":"2024-09-17T08:26:27.734966+0000","last_fullsized":"2024-09-17T08:26:27.734966+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:24:23.759435+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071267730000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":235,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.724772+0000","last_change":"2024-09-17T08:25:34.089169+0000","last_active":"2024-09-17T08:26:26.724772+0000","last_peered":"2024-09-17T08:26:26.724772+0000","last_clean":"2024-09-17T08:26:26.724772+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:26:26.724772+0000","last_undegraded":"2024-09-17T08:26:26.724772+0000","last_fullsized":"2024-09-17T08:26:26.724772+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.089010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:53.698566+0000","scrub_duration":0.0037643389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":340,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.741220+0000","last_change":"2024-09-17T08:26:13.980481+0000","last_active":"2024-09-17T08:26:26.741220+0000","last_peered":"2024-09-17T08:26:26.741220+0000","last_clean":"2024-09-17T08:26:26.741220+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:26:26.741220+0000","last_undegraded":"2024-09-17T08:26:26.741220+0000","last_fullsized":"2024-09-17T08:26:26.741220+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:13.980289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.980289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:15.581630+0000","scrub_duration":0.014435847,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":356,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.740537+0000","last_change":"2024-09-17T08:25:38.883969+0000","last_active":"2024-09-17T08:26:26.740537+0000","last_peered":"2024-09-17T08:26:26.740537+0000","last_clean":"2024-09-17T08:26:26.740537+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:26:26.740537+0000","last_undegraded":"2024-09-17T08:26:26.740537+0000","last_fullsized":"2024-09-17T08:26:26.740537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.883814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.883814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:43.222351+0000","scrub_duration":0.012313661,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":417,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.738758+0000","last_change":"2024-09-17T08:24:02.955324+0000","last_active":"2024-09-17T08:26:26.738758+0000","last_peered":"2024-09-17T08:26:26.738758+0000","last_clean":"2024-09-17T08:26:26.738758+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:26:26.738758+0000","last_undegraded":"2024-09-17T08:26:26.738758+0000","last_fullsized":"2024-09-17T08:26:26.738758+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:24:02.955231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:02.955231+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022957339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":368,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-17T08:26:26.723739+0000","last_change":"2024-09-17T08:26:18.169361+0000","last_active":"2024-09-17T08:26:26.723739+0000","last_peered":"2024-09-17T08:26:26.723739+0000","last_clean":"2024-09-17T08:26:26.723739+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:26:26.723739+0000","last_undegraded":"2024-09-17T08:26:26.723739+0000","last_fullsized":"2024-09-17T08:26:26.723739+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:18.169258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.169258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:39.643737+0000","scrub_duration":0.042744938000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5095227416,"num_objects":77748,"num_object_clones":0,"num_object_copies":233244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187312,"num_write_kb":8481857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15270817792,"data_stored":15270805576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87412,"ondisk_log_size":87412,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":661,"num_read_kb":656,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070669,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1891676,"kb_used_data":1301076,"kb_used_omap":1,"kb_used_meta":590590,"kb_avail":91853476,"statfs":{"total":95995035648,"available":94057959424,"internally_reserved":0,"allocated":1332301824,"data_stored":1332001080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":604764619},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.363,"15min":1.4370000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":5.3869999999999996,"5min":13.446,"15min":62.018000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":1.476,"15min":1.546},"min":{"1min":0.22,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":9.4169999999999998,"5min":13.417,"15min":61.896999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.7190000000000001,"15min":1.49},"min":{"1min":0.23999999999999999,"5min":0.23899999999999999,"15min":0.23100000000000001},"max":{"1min":5.8369999999999997,"5min":44.917999999999999,"15min":62.067},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.944,"15min":1.4810000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":5.5179999999999998,"5min":44.996000000000002,"15min":62.194000000000003},"last":1.1399999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.081,"15min":0.92900000000000005},"min":{"1min":0.26600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.3109999999999999,"5min":9.3109999999999999,"15min":9.3109999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.0660000000000001,"15min":0.72699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":4.8920000000000003,"5min":2.6949999999999998,"15min":1.627},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":74.376000000000005,"5min":74.376000000000005,"15min":74.376000000000005},"last":0.37},{"interface":"front","average":{"1min":4.681,"5min":2.7610000000000001,"15min":1.73},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":75.183000000000007,"5min":75.183000000000007,"15min":75.183000000000007},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.97599999999999998,"15min":0.71699999999999997},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.962,"5min":4.391,"15min":5.335},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73699999999999999,"15min":0.61199999999999999},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":1.845,"5min":3.798,"15min":5.9669999999999996},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.88900000000000001,"15min":0.76600000000000001},"min":{"1min":0.27500000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.8959999999999999,"5min":8.6400000000000006,"15min":16.808},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.88300000000000001,"15min":0.753},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":1.9359999999999999,"5min":4.4889999999999999,"15min":11.842000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.95199999999999996,"15min":0.78800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":2.0680000000000001,"5min":6.6509999999999998,"15min":7.2359999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.66500000000000004},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.246},"max":{"1min":3.1819999999999999,"5min":3.3849999999999998,"15min":5.9400000000000004},"last":0.73699999999999999}]}]},{"osd":6,"up_from":50,"seq":214748365011,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2597596,"kb_used_data":2119380,"kb_used_omap":1,"kb_used_meta":478206,"kb_avail":91147556,"statfs":{"total":95995035648,"available":93335097344,"internally_reserved":0,"allocated":2170245120,"data_stored":2169951914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":489683399},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":3.9220000000000002,"5min":1.6930000000000001,"15min":2.3599999999999999},"min":{"1min":0.24299999999999999,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":55.625,"5min":55.625,"15min":81.007000000000005},"last":0.998},{"interface":"front","average":{"1min":4.274,"5min":1.819,"15min":2.4649999999999999},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":55.597000000000001,"5min":55.597000000000001,"15min":80.310000000000002},"last":0.245}]},{"osd":1,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.494,"15min":1.627},"min":{"1min":0.27900000000000003,"5min":0.26500000000000001,"15min":0.22500000000000001},"max":{"1min":3.7029999999999998,"5min":42.618000000000002,"15min":80.813000000000002},"last":1.48},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.4690000000000001,"15min":1.6759999999999999},"min":{"1min":0.29499999999999998,"5min":0.251,"15min":0.24299999999999999},"max":{"1min":3.0670000000000002,"5min":42.790999999999997,"15min":80.281000000000006},"last":0.77000000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:26:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":2.0259999999999998,"15min":1.0900000000000001},"min":{"1min":0.28799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.5410000000000004,"5min":70.451999999999998,"15min":70.451999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4139999999999999,"5min":2.0419999999999998,"15min":1.034},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.9950000000000001,"5min":70.488,"15min":70.488},"last":1.0249999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2050000000000001,"15min":1.9399999999999999},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.1020000000000003,"5min":9.9309999999999992,"15min":80.427000000000007},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.278,"5min":1.1080000000000001,"15min":1.8700000000000001},"min":{"1min":0.29799999999999999,"5min":0.26700000000000002,"15min":0.26000000000000001},"max":{"1min":3.0449999999999999,"5min":11.859,"15min":80.379999999999995},"last":0.80600000000000005}]},{"osd":4,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66500000000000004,"15min":0.63500000000000001},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":4.6219999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.69499999999999995,"15min":0.58599999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5219999999999998,"5min":3.585,"15min":5.0330000000000004},"last":0.94299999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:25:49 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.84199999999999997,"15min":0.70499999999999996},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.245},"max":{"1min":3.3740000000000001,"5min":5.5659999999999998,"15min":6.1859999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.73099999999999998,"15min":0.61399999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.9580000000000002,"5min":3.464,"15min":4.9969999999999999},"last":0.40200000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.70799999999999996},"min":{"1min":0.29199999999999998,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":2.948,"5min":2.948,"15min":4.774},"last":0.378},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.749,"15min":0.64800000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":3.516,"5min":3.5590000000000002,"15min":5.0599999999999996},"last":0.84299999999999997}]}]},{"osd":5,"up_from":41,"seq":176093659351,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2182860,"kb_used_data":1614276,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":91562292,"statfs":{"total":95995035648,"available":93759787008,"internally_reserved":0,"allocated":1653018624,"data_stored":1652732929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":582089158},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5129999999999999,"5min":3.4409999999999998,"15min":2.3010000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.20300000000000001},"max":{"1min":26.158000000000001,"5min":65.941000000000003,"15min":65.941000000000003},"last":1.2010000000000001},{"interface":"front","average":{"1min":3.0539999999999998,"5min":3.46,"15min":2.2709999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":26.984000000000002,"5min":66.135000000000005,"15min":66.135000000000005},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":2.109,"15min":1.417},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":4.0469999999999997,"5min":66.981999999999999,"15min":66.981999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":2.181,"15min":1.474},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":4.0599999999999996,"5min":66.316000000000003,"15min":66.316000000000003},"last":0.372}]},{"osd":2,"last update":"Tue Sep 17 08:25:30 2024","interfaces":[{"interface":"back","average":{"1min":2.2280000000000002,"5min":3.0339999999999998,"15min":1.498},"min":{"1min":0.28199999999999997,"5min":0.221,"15min":0.221},"max":{"1min":22.245000000000001,"5min":131.727,"15min":131.727},"last":1.1819999999999999},{"interface":"front","average":{"1min":2.3679999999999999,"5min":3.1850000000000001,"15min":1.7989999999999999},"min":{"1min":0.313,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.146000000000001,"5min":131.50399999999999,"15min":131.50399999999999},"last":1.3129999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":2.3279999999999998,"15min":1.8},"min":{"1min":0.32600000000000001,"5min":0.26200000000000001,"15min":0.253},"max":{"1min":6.6879999999999997,"5min":67.676000000000002,"15min":75.286000000000001},"last":1.155},{"interface":"front","average":{"1min":1.175,"5min":2.1909999999999998,"15min":1.722},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27100000000000002},"max":{"1min":3.4100000000000001,"5min":67.691999999999993,"15min":77.206999999999994},"last":1.3500000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.83599999999999997,"15min":0.81599999999999995},"min":{"1min":0.28899999999999998,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":6.3600000000000003,"5min":6.3600000000000003,"15min":42.421999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.52,"5min":0.83599999999999997,"15min":0.81299999999999994},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":42.191000000000003},"last":1.3640000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.8120000000000003,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.26700000000000002},"max":{"1min":4.4000000000000004,"5min":4.9189999999999996,"15min":5.4400000000000004},"last":1.3320000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.755},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":3.2989999999999999,"5min":4.9409999999999998,"15min":7.5270000000000001},"last":1.292},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.26900000000000002},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":0.67600000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953692,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3065500,"kb_used_data":2579092,"kb_used_omap":1,"kb_used_meta":486398,"kb_avail":90679652,"statfs":{"total":95995035648,"available":92855963648,"internally_reserved":0,"allocated":2640990208,"data_stored":2640688782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":498072006},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.93700000000000006,"15min":1.0109999999999999},"min":{"1min":0.254,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":3.4849999999999999,"5min":4.5229999999999997,"15min":23.786000000000001},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":0.96699999999999997,"15min":1.056},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":3.7450000000000001,"5min":6.2939999999999996,"15min":23.245000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.2609999999999999,"15min":1.1619999999999999},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":5.5839999999999996,"5min":9.3789999999999996,"15min":23.135000000000002},"last":1.377},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.2410000000000001,"15min":1.1699999999999999},"min":{"1min":0.39300000000000002,"5min":0.28100000000000003,"15min":0.24099999999999999},"max":{"1min":3.52,"5min":9.0719999999999992,"15min":23.434999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.78600000000000003,"15min":0.63600000000000001},"min":{"1min":0.29599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6499999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:25:58 2024","interfaces":[{"interface":"back","average":{"1min":2.3159999999999998,"5min":1.2929999999999999,"15min":1.9430000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":28.138000000000002,"5min":28.138000000000002,"15min":154.137},"last":1.456},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.403,"15min":1.8859999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.26700000000000002},"max":{"1min":28.117000000000001,"5min":28.117000000000001,"15min":153.54400000000001},"last":1.613}]},{"osd":5,"last update":"Tue Sep 17 08:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73299999999999998,"15min":0.64300000000000002},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.23000000000000001},"max":{"1min":4.093,"5min":4.093,"15min":6.2329999999999997},"last":0.626},{"interface":"front","average":{"1min":0.745,"5min":0.73899999999999999,"15min":0.67800000000000005},"min":{"1min":0.27400000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":3.9409999999999998,"5min":4.0229999999999997,"15min":4.5300000000000002},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:25:45 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.60399999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":2.5529999999999999,"5min":3.0110000000000001,"15min":4.827},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":3.6659999999999999,"5min":3.6659999999999999,"15min":6.0010000000000003},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":2.1989999999999998,"5min":4.0259999999999998,"15min":4.875},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.70499999999999996},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.6499999999999999,"5min":2.7949999999999999,"15min":4.6950000000000003},"last":0.53700000000000003}]}]},{"osd":3,"up_from":27,"seq":115964117216,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2225444,"kb_used_data":1608604,"kb_used_omap":1,"kb_used_meta":616830,"kb_avail":91519708,"statfs":{"total":95995035648,"available":93716180992,"internally_reserved":0,"allocated":1647210496,"data_stored":1646900331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":631634376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:25:36 2024","interfaces":[{"interface":"back","average":{"1min":8.9649999999999999,"5min":3.7069999999999999,"15min":2.2309999999999999},"min":{"1min":0.54600000000000004,"5min":0.156,"15min":0.156},"max":{"1min":130.40299999999999,"5min":130.40299999999999,"15min":130.40299999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":8.2680000000000007,"5min":3.0899999999999999,"15min":2.0230000000000001},"min":{"1min":0.33800000000000002,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":130.70099999999999,"5min":130.70099999999999,"15min":130.70099999999999},"last":1.1759999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":2.153,"15min":1.6919999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.186},"max":{"1min":3.5529999999999999,"5min":71.459999999999994,"15min":71.459999999999994},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.278,"5min":2.29,"15min":1.706},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.188},"max":{"1min":3.718,"5min":71.438999999999993,"15min":71.438999999999993},"last":1.8680000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:25:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.657,"15min":1.268},"min":{"1min":0.253,"5min":0.215,"15min":0.215},"max":{"1min":3.968,"5min":60.423999999999999,"15min":60.423999999999999},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.617,"5min":1.8919999999999999,"15min":1.7250000000000001},"min":{"1min":0.35299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.8959999999999999,"5min":60.368000000000002,"15min":60.368000000000002},"last":4.3440000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.306,"15min":1.3420000000000001},"min":{"1min":0.35199999999999998,"5min":0.28499999999999998,"15min":0.252},"max":{"1min":3.7589999999999999,"5min":5.5460000000000003,"15min":35.841000000000001},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.194,"15min":1.274},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":3.9889999999999999,"5min":7.5999999999999996,"15min":35.857999999999997},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:26:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3440000000000001,"15min":1.2789999999999999},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.29799999999999999},"max":{"1min":3.2570000000000001,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29499999999999998},"max":{"1min":4.71,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":0.49099999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:25:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.127,"15min":1.179},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":4.2409999999999997,"5min":7.6669999999999998,"15min":10.945},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.536,"15min":1.4430000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":4.4829999999999997,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.37}]},{"osd":7,"last update":"Tue Sep 17 08:26:07 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":2.9420000000000002,"15min":1.8169999999999999},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":4.4720000000000004,"5min":163.006,"15min":163.006},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.456,"5min":3.089,"15min":1.8420000000000001},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.28999999999999998},"max":{"1min":3.8079999999999998,"5min":162.83699999999999,"15min":162.83699999999999},"last":1.9239999999999999}]}]},{"osd":2,"up_from":227,"seq":974957576268,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2035996,"kb_used_data":1840020,"kb_used_omap":4,"kb_used_meta":195963,"kb_avail":91709156,"statfs":{"total":95995035648,"available":93910175744,"internally_reserved":0,"allocated":1884180480,"data_stored":1883887053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":200666787},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9510000000000001,"5min":1.5229999999999999,"15min":1.1559999999999999},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":5.6509999999999998,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":3.4009999999999998},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3240000000000001,"15min":1.4179999999999999},"min":{"1min":0.30599999999999999,"5min":0.184,"15min":0.184},"max":{"1min":5.9329999999999998,"5min":9.1940000000000008,"15min":9.1940000000000008},"last":5.0590000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.96,"15min":1.474},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":5.2789999999999999,"5min":41.405000000000001,"15min":41.405000000000001},"last":0.216},{"interface":"front","average":{"1min":1.423,"5min":1.8959999999999999,"15min":1.48},"min":{"1min":0.20999999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":5.1449999999999996,"5min":41.25,"15min":41.25},"last":3.4329999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.7130000000000001,"15min":1.5209999999999999},"min":{"1min":0.218,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.1280000000000001,"5min":44.185000000000002,"15min":44.185000000000002},"last":0.496},{"interface":"front","average":{"1min":1.823,"5min":1.843,"15min":1.494},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":7.492,"5min":43.994,"15min":43.994},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.153,"15min":0.93400000000000005},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.2169999999999996,"5min":5.524,"15min":5.524},"last":3.4620000000000002},{"interface":"front","average":{"1min":1.1140000000000001,"5min":0.98399999999999999,"15min":0.875},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.4699999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":0.28899999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.0509999999999999,"15min":0.97699999999999998},"min":{"1min":0.38100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.1059999999999999,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":3.5049999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0229999999999999,"15min":0.95499999999999996},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.1659999999999999,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.32300000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.212,"15min":0.85799999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.258,"5min":1.131,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.181,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":3.484}]},{"osd":7,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.127,"15min":1.0249999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"last":0.35099999999999998},{"interface":"front","average":{"1min":1.966,"5min":1.288,"15min":1.006},"min":{"1min":0.53800000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.468,"5min":7.468,"15min":7.468},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1572972,"kb_used_data":942180,"kb_used_omap":1,"kb_used_meta":630782,"kb_avail":92172180,"statfs":{"total":95995035648,"available":94384312320,"internally_reserved":0,"allocated":964792320,"data_stored":964474553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":645921222},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:01 2024","interfaces":[{"interface":"back","average":{"1min":3.3559999999999999,"5min":1.6719999999999999,"15min":1.6930000000000001},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.155},"max":{"1min":27.670999999999999,"5min":27.670999999999999,"15min":31.585999999999999},"last":2.411},{"interface":"front","average":{"1min":2.6320000000000001,"5min":1.8109999999999999,"15min":1.802},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.17199999999999999},"max":{"1min":26.120999999999999,"5min":26.120999999999999,"15min":35.807000000000002},"last":3.722}]},{"osd":2,"last update":"Tue Sep 17 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.712,"15min":1.099},"min":{"1min":0.28399999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":15.044,"5min":26.297999999999998,"15min":26.297999999999998},"last":3.7589999999999999},{"interface":"front","average":{"1min":2.3519999999999999,"5min":1.6659999999999999,"15min":1.5449999999999999},"min":{"1min":0.25800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":8.2899999999999991,"5min":26.602,"15min":26.602},"last":0.439}]},{"osd":3,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.5589999999999999,"15min":1.383},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.184},"max":{"1min":2.8490000000000002,"5min":5.2629999999999999,"15min":8.0559999999999992},"last":2.2749999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.7589999999999999,"15min":1.5740000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.189},"max":{"1min":2.944,"5min":22.984999999999999,"15min":22.984999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.3009999999999999,"15min":1.2769999999999999},"min":{"1min":0.35299999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":2.3199999999999998,"5min":16.884,"15min":16.884},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.1870000000000001,"15min":1.173},"min":{"1min":0.39200000000000002,"5min":0.30499999999999999,"15min":0.25700000000000001},"max":{"1min":2.2480000000000002,"5min":9.6890000000000001,"15min":10.208},"last":0.32200000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2529999999999999,"15min":1.1919999999999999},"min":{"1min":0.39800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":3.9670000000000001,"5min":4.5,"15min":7.9139999999999997},"last":2.2509999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.3029999999999999,"15min":1.3440000000000001},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":2.718,"5min":16.853999999999999,"15min":16.853999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Sep 17 08:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.389,"15min":1.254},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":2.7389999999999999,"5min":9.7159999999999993,"15min":10.253},"last":0.35399999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.262,"15min":1.24},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.27200000000000002},"max":{"1min":2.855,"5min":5.5469999999999997,"15min":7.9459999999999997},"last":2.2130000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.9670000000000001,"15min":1.5429999999999999},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.33100000000000002},"max":{"1min":2.7519999999999998,"5min":47.442,"15min":47.442},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.7509999999999999,"15min":1.45},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.378},"max":{"1min":2.6970000000000001,"5min":47.475000000000001,"15min":47.475000000000001},"last":0.376}]}]},{"osd":0,"up_from":8,"seq":34359738605,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3624760,"kb_used_data":3020204,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":90120392,"statfs":{"total":95995035648,"available":92283281408,"internally_reserved":0,"allocated":3092688896,"data_stored":3092379742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":619051462},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":2.0329999999999999,"5min":2.105,"15min":1.917},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":10.848000000000001,"5min":41.247,"15min":61.649000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":2.1749999999999998,"5min":2.387,"15min":2.125},"min":{"1min":0.182,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1669999999999998,"5min":40.436,"15min":61.281999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:26:33 2024","interfaces":[{"interface":"back","average":{"1min":8.5210000000000008,"5min":4.0810000000000004,"15min":2.1160000000000001},"min":{"1min":0.32600000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":114.771,"5min":114.771,"15min":114.771},"last":2.613},{"interface":"front","average":{"1min":8.4700000000000006,"5min":4.2670000000000003,"15min":2.0110000000000001},"min":{"1min":0.27900000000000003,"5min":0.193,"15min":0.193},"max":{"1min":114.732,"5min":114.732,"15min":114.732},"last":3.1859999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:25:46 2024","interfaces":[{"interface":"back","average":{"1min":2.1160000000000001,"5min":2.0299999999999998,"15min":2.069},"min":{"1min":0.39500000000000002,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":5.968,"5min":38.697000000000003,"15min":61.926000000000002},"last":2.6840000000000002},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.6659999999999999,"15min":1.819},"min":{"1min":0.35099999999999998,"5min":0.223,"15min":0.223},"max":{"1min":10.868,"5min":38.950000000000003,"15min":61.616},"last":1.99}]},{"osd":4,"last update":"Tue Sep 17 08:26:03 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.627,"15min":1.413},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26300000000000001},"max":{"1min":5.242,"5min":41.158000000000001,"15min":41.158000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.923,"5min":1.542,"15min":1.423},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":6.8920000000000003,"5min":38.978000000000002,"15min":38.978000000000002},"last":2.5529999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.214,"15min":1.3440000000000001},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.6230000000000002,"5min":10.805999999999999,"15min":39.058},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.359,"15min":1.5269999999999999},"min":{"1min":0.39000000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":5.1260000000000003,"5min":5.1399999999999997,"15min":43.485999999999997},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:25:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.3680000000000001,"15min":1.3009999999999999},"min":{"1min":0.377,"5min":0.317,"15min":0.27700000000000002},"max":{"1min":10.827999999999999,"5min":38.917999999999999,"15min":38.917999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.7709999999999999,"15min":1.518},"min":{"1min":0.374,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.2400000000000002,"5min":41.189,"15min":41.189},"last":0.76500000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":2.294,"5min":1.8,"15min":1.651},"min":{"1min":0.42599999999999999,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":6.8680000000000003,"5min":39.045999999999999,"15min":39.045999999999999},"last":2.5899999999999999},{"interface":"front","average":{"1min":1.786,"5min":1.5569999999999999,"15min":1.5049999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.307},"max":{"1min":10.769,"5min":39.082999999999998,"15min":39.082999999999998},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3077443584,"data_stored":3077439512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":956039168,"data_stored":956039168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1879769088,"data_stored":1879769088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1626013696,"data_stored":1626013696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2622427136,"data_stored":2622423064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1640562688,"data_stored":1640562688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2138836992,"data_stored":2138832920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1329725440,"data_stored":1329725440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:26:40.893 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-17T08:26:40.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-17T08:26:40.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:40.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:40.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:40.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:41.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:41.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:41.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:41.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:40 smithi079 ceph-mon[29565]: pgmap v742: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:26:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:40 smithi079 ceph-mon[40649]: pgmap v742: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:26:41.171 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:40 smithi173 ceph-mon[33187]: pgmap v742: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-17T08:26:41.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:41.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.223 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:41.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:41.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:41.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:41.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:41.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:41.606 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:41.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:26:41.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:41.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:41.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:41.768 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:41.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:41.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:41.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:41.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:41.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:41.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:41.957 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:42.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:42.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:42.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:42.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:42.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:42.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:42.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:42.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:26:42.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:42.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:42.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:42.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:42.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:42.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:42.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:42.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:42.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:42.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:42.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:42.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:42.941 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:42.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:42.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:43.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:43.079 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:43.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:43.181 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:43.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:43.237 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:43.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:43.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.334 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:43.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:43.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:43.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.473 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:43.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:43.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:43.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:43.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:43.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:43.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:43.746 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:43.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:43.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:43 smithi079 ceph-mon[40649]: pgmap v743: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 929 op/s 2024-09-17T08:26:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:43 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/768471280' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:43 smithi079 ceph-mon[29565]: pgmap v743: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 929 op/s 2024-09-17T08:26:43.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:43 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/768471280' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:43.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:43.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:43.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:43.942 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:43 smithi173 ceph-mon[33187]: pgmap v743: 148 pgs: 148 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 929 op/s 2024-09-17T08:26:43.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:43 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/768471280' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:43.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:44.018 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:44.120 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:44.152 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:44.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:44.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.289 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:44.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:44.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.421 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:44.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:44.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:44.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.558 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:44.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.695 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:44.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:44.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:44.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:44.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:44.967 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:44.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:45.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:45.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.101 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:45.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:45.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:45.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:45.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.358 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:45.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:45.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:26:45.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:26:45.491 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:26:45.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:26:45.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:26:45.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.494 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:45.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:45.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.634 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:45.735 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:45.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:45.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:45.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:45 smithi079 ceph-mon[40649]: pgmap v744: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:45.854 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:45 smithi079 ceph-mon[29565]: pgmap v744: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:45.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:45.897 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:45.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:45.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:45.930 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:45 smithi173 ceph-mon[33187]: pgmap v744: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-17T08:26:45.999 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:46.029 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:46.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:46.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:46.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:46.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:46.272 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:46.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:46.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.325 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:46.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:46.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:46.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.469 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:46.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:46.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:46.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:46.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:46.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.743 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:46.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:46.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:46.892 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:46.892 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:46.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:46.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:46.997 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:47.031 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:47.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:47.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:47.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.160 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:47.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:47.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:47.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:47.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:47.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:47.434 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:47.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:47.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:47.565 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:47.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:47.566 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:47 smithi173 ceph-mon[33187]: pgmap v745: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:47.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:26:47.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:47.717 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:47.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:47.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:47 smithi079 ceph-mon[29565]: pgmap v745: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:47.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:47 smithi079 ceph-mon[40649]: pgmap v745: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:47.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:47.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:47.856 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:47.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:47.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:47.958 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:26:48.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.002 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:48.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:48.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:48.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.146 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:48.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:48.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:48.287 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:48.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:48.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:48.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.428 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:48.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:48.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:26:48.569 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/148482188' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.570 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.571 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:48.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:set pool 1 pg_num to 11 2024-09-17T08:26:48.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:48.707 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:48.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.708 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/148482188' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:48.849 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:48.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:48.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:48.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:26:48.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/148482188' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:48.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:49.020 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:49.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:49.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:49.163 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:49.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:49.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:49.303 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:49.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:49.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:49.436 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:49.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:49.538 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:49.572 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:49.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:49.578 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:49.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:49.714 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:49.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:49.816 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: pgmap v746: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 948 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:26:49.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:49 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:49.852 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:49.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:49.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: pgmap v746: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 948 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: pgmap v746: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 948 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:26:49.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:26:49.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:49 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:26:49.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:49.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:49.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:49.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:50.048 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:26:50.048 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:26:50.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:50.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:50.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:50.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.267 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:50.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:50.409 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:50.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:50.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:50.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:50.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:50.577 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:26:50.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:50.690 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:50.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:50.720 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:50 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:50.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4255417845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.721 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:50 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:50.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.840 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:50.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:50.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:26:50.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:50.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4255417845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:50.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4255417845' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:50 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:26:50.941 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:50.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:50.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:50.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:51.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:51.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:51.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:51.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:51.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.264 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:51.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:51.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:51.402 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:51.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:51.504 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:51.537 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:51.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:51.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:51.671 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:51.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:51.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:51.819 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:51.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:51.864 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[29565]: pgmap v748: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:51.864 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:51.864 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[29565]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[40649]: pgmap v748: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-17T08:26:51.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:51.866 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:51 smithi079 ceph-mon[40649]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T08:26:51.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:51 smithi173 ceph-mon[33187]: pgmap v748: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-17T08:26:51.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:51 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:26:51.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:51 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-17T08:26:51.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:26:51.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:51 smithi173 ceph-mon[33187]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T08:26:51.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:51.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:51.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:51.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:52.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:52.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.093 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:52.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:52.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:52.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:52.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:52.264 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:52.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:52.362 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:52.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:52.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:52.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:52.508 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:52.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:52.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:52.751 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:52.782 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:52.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:52.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[40649]: pgmap v750: 158 pgs: 10 unknown, 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 932 op/s 2024-09-17T08:26:52.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:52.784 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[40649]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T08:26:52.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[29565]: pgmap v750: 158 pgs: 10 unknown, 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 932 op/s 2024-09-17T08:26:52.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:52.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:52 smithi079 ceph-mon[29565]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T08:26:52.884 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:52.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:52.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:52.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:52 smithi173 ceph-mon[33187]: pgmap v750: 158 pgs: 10 unknown, 148 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 932 op/s 2024-09-17T08:26:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:52 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-17T08:26:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:52 smithi173 ceph-mon[33187]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T08:26:53.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:53.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:53.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:26:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:26:53.154 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:53.191 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:53.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:53.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:53.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:53.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:53.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:53.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:53.624 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:26:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:26:53.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:53.728 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:53.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:53.860 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:53.935 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-17T08:26:53.935 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[29565]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T08:26:53.935 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:53.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-17T08:26:53.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[40649]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T08:26:53.936 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:53.961 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:53 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-17T08:26:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:53 smithi173 ceph-mon[33187]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T08:26:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:26:53.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:53.993 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:53.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:54.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:54.129 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:54.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:54.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:54.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:26:54.389 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:54.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:54.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:54.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.548 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:54.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:54.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:54.679 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:54.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:54.699 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:26:54.699 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:26:54.733 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:26:54.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:26:54.766 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:26:54.767 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:26:54.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:54.807 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:26:54.808 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:26:54.810 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:54.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:54.842 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:26:54.843 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:26:54.871 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:26:54.871 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:26:54.903 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:26:54.903 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:26:54.904 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:54 smithi079 ceph-mon[29565]: pgmap v753: 158 pgs: 10 unknown, 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-17T08:26:54.904 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:54 smithi079 ceph-mon[29565]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T08:26:54.904 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:54 smithi079 ceph-mon[40649]: pgmap v753: 158 pgs: 10 unknown, 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-17T08:26:54.904 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:54 smithi079 ceph-mon[40649]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T08:26:54.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:54.936 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:26:54.936 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:26:54.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:54.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:54.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:54 smithi173 ceph-mon[33187]: pgmap v753: 158 pgs: 10 unknown, 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-17T08:26:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:54 smithi173 ceph-mon[33187]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T08:26:55.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:55.058 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:26:55.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:26:55.058 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T08:26:55.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:26:55.087 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:55.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:55.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:55.219 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:55.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:55.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:55.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:55.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:55.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:55.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.513 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:55.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:55.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:55.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:55.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:55.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:55.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:55 smithi079 ceph-mon[29565]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T08:26:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2445713333' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:55 smithi079 ceph-mon[40649]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T08:26:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2445713333' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:55.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:55.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:55.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:55.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:55 smithi173 ceph-mon[33187]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T08:26:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2445713333' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:26:56.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:56.084 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:56.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:212302 10.9 10:94aac3e3:::benchmark_data_smithi079_172150_object79770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-17T08:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.156029+0000", 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0099269170000000004, 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099581989999999992, 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 212302 2024-09-17T08:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156029+0000", 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156029+0000", 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156027+0000", 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156033+0000", 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8590000000000001e-06 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156043+0000", 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0258000000000001e-05 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156049+0000", 2024-09-17T08:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4090000000000002e-06 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156098+0000", 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-05 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156207+0000", 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108281 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.156315+0000", 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108729 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163676+0000", 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073604600000000001 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.164081+0000", 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040470799999999999 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:212306 10.3 10:c05504a0:::benchmark_data_smithi079_172150_object79774:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.157910+0000", 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0080450160000000003, 2024-09-17T08:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081638860000000004, 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 212306 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157910+0000", 2024-09-17T08:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157910+0000", 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157909+0000", 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157915+0000", 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1489999999999999e-06 2024-09-17T08:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157928+0000", 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2805e-05 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157936+0000", 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4390000000000003e-06 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.157982+0000", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5843000000000001e-05 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.158089+0000", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107744 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.158194+0000", 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010456200000000001 2024-09-17T08:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:212304 10.f e337/287 10:f95277c9:::benchmark_data_smithi079_172150_object79772:head v 337'13291, mlcod=337'13289)", 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.159397+0000", 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0065586999999999998, 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067452659999999998, 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159397+0000", 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159397+0000", 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159396+0000", 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159403+0000", 2024-09-17T08:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.221e-06 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159414+0000", 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1045e-05 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.159421+0000", 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6930000000000001e-06 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160106+0000", 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068537199999999996 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160125+0000", 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8413000000000001e-05 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:212308 10.3 10:ce9a6569:::benchmark_data_smithi079_172150_object79776:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-17T08:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.160048+0000", 2024-09-17T08:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0059078359999999996, 2024-09-17T08:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061490859999999998, 2024-09-17T08:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 212308 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160048+0000", 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160048+0000", 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160046+0000", 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160052+0000", 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5820000000000001e-06 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160062+0000", 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0536999999999999e-05 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.160069+0000", 2024-09-17T08:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6320000000000002e-06 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163564+0000", 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034952989999999999 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163682+0000", 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011808500000000001 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163902+0000", 2024-09-17T08:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021916000000000001 2024-09-17T08:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:212298 10.e e337/287 10:7f4ef197:::benchmark_data_smithi079_172150_object79766:head v 337'13304, mlcod=337'13301)", 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.161777+0000", 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0041780910000000001, 2024-09-17T08:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044859360000000003, 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161777+0000", 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161777+0000", 2024-09-17T08:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161777+0000", 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161791+0000", 2024-09-17T08:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4123e-05 2024-09-17T08:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161803+0000", 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.161810+0000", 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3459999999999996e-06 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162159+0000", 2024-09-17T08:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034836600000000002 2024-09-17T08:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162251+0000", 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2023999999999998e-05 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:212306 10.3 e337/287)", 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.162060+0000", 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0038955869999999998, 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042534340000000004, 2024-09-17T08:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162060+0000", 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162060+0000", 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162060+0000", 2024-09-17T08:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162062+0000", 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.069e-06 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162067+0000", 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999998e-06 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.162071+0000", 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2400000000000001e-06 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:212306 10.3 e337/287)", 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.163146+0000", 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0028096589999999999, 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032078010000000001, 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163146+0000", 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163146+0000", 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163145+0000", 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163150+0000", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0119999999999996e-06 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163162+0000", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1579e-05 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163168+0000", 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7710000000000001e-06 2024-09-17T08:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:212313 10.e e337/287 10:75aa4c0b:::benchmark_data_smithi079_172150_object79781:head v 337'13305, mlcod=337'13301)", 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:26:56.163772+0000", 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0021831939999999998, 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026245489999999999, 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163772+0000", 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163772+0000", 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163772+0000", 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163789+0000", 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7184e-05 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163799+0000", 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0548e-05 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.163806+0000", 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.164982+0000", 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001176453 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:26:56.164999+0000", 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7416999999999999e-05 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 8 2024-09-17T08:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:26:56.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:56.218 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:56.259 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:56.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:56.361 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:56.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:56.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:56.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:56.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.542 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:56.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:56.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:26:56.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:56.687 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:26:56.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:56.817 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:56.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[29565]: pgmap v756: 158 pgs: 10 unknown, 148 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.46k op/s 2024-09-17T08:26:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[29565]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T08:26:56.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[29565]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T08:26:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[40649]: pgmap v756: 158 pgs: 10 unknown, 148 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.46k op/s 2024-09-17T08:26:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[40649]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T08:26:56.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:56 smithi079 ceph-mon[40649]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T08:26:56.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:56.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:56.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:56.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:56.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:56 smithi173 ceph-mon[33187]: pgmap v756: 158 pgs: 10 unknown, 148 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.46k op/s 2024-09-17T08:26:56.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:56 smithi173 ceph-mon[33187]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T08:26:56.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:56 smithi173 ceph-mon[33187]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T08:26:57.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:57.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.099 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:57.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:57.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:26:57.187 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:26:57.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:57.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.229 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:57.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:57.330 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:57.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.361 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:57.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:57.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:57.487 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:57.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:57.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:57.623 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:57.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:57.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:57.771 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:57.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:57.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:57.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:57.913 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:57.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:57.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:26:58.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:58.059 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:58.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.060 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:58.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:58.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.226 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:58.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:58.327 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:26:58.360 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:58.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:26:58.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:58.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:58.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:58.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:58.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:58.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:58.761 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:58.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:58.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:58.904 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:58.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:58.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:26:58 smithi173 ceph-mon[33187]: pgmap v759: 158 pgs: 6 unknown, 152 active+clean; 3.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 28 KiB/s rd, 219 KiB/s wr, 1.24k op/s; 157 KiB/s, 0 objects/s recovering 2024-09-17T08:26:59.006 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:26:59.039 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:59.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:26:59.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:26:58 smithi079 ceph-mon[29565]: pgmap v759: 158 pgs: 6 unknown, 152 active+clean; 3.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 28 KiB/s rd, 219 KiB/s wr, 1.24k op/s; 157 KiB/s, 0 objects/s recovering 2024-09-17T08:26:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:26:58 smithi079 ceph-mon[40649]: pgmap v759: 158 pgs: 6 unknown, 152 active+clean; 3.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 28 KiB/s rd, 219 KiB/s wr, 1.24k op/s; 157 KiB/s, 0 objects/s recovering 2024-09-17T08:26:59.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:59.177 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:59.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:59.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:26:59.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:59.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:26:59.412 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:26:59.446 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:59.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:26:59.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:26:59.584 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:59.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:26:59.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:26:59.711 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:26:59.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:26:59.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:59.837 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:59.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:26:59.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:26:59.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:26:59.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:26:59.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:00.080 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:00.112 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:00.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:00.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:00.245 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:00.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:00.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:00.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.378 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:00.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:00.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:00.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:00.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:00.617 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:00.656 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:00.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:00.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:00.783 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:00.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:00 smithi079 ceph-mon[40649]: pgmap v760: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 38 KiB/s rd, 294 KiB/s wr, 1.45k op/s; 264 KiB/s, 1 objects/s recovering 2024-09-17T08:27:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:00 smithi079 ceph-mon[29565]: pgmap v760: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 38 KiB/s rd, 294 KiB/s wr, 1.45k op/s; 264 KiB/s, 1 objects/s recovering 2024-09-17T08:27:00.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:00.919 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:00.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:00 smithi173 ceph-mon[33187]: pgmap v760: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 38 KiB/s rd, 294 KiB/s wr, 1.45k op/s; 264 KiB/s, 1 objects/s recovering 2024-09-17T08:27:01.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:01.043 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:01.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:01.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:01.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:01.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:01.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:01.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:01.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:01.418 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:01.450 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:01.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:01.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:01.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:01.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:01.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:01.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:01.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:01.822 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:01.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:01.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.855 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:01.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:01.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:01.985 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:01.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:02.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:02.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:02.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:02.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.267 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.267 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:02.368 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:02.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.405 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:02.405 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:02.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:02.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:02.673 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:02.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:02.820 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:02.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:02.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:02.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:02.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:02.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:03.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:03.111 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:03.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:03.147 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:03.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:03.246 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.247 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:03.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:03.385 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:03.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:03.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:03.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:03.505 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:03 smithi173 ceph-mon[33187]: pgmap v761: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 32 KiB/s rd, 252 KiB/s wr, 962 op/s; 226 KiB/s, 0 objects/s recovering 2024-09-17T08:27:03.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.521 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:03.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:03.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:03.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:03.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:03.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:03.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:03.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:03.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:03 smithi079 ceph-mon[40649]: pgmap v761: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 32 KiB/s rd, 252 KiB/s wr, 962 op/s; 226 KiB/s, 0 objects/s recovering 2024-09-17T08:27:03.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:03 smithi079 ceph-mon[29565]: pgmap v761: 158 pgs: 158 active+clean; 3.1 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 32 KiB/s rd, 252 KiB/s wr, 962 op/s; 226 KiB/s, 0 objects/s recovering 2024-09-17T08:27:03.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:03.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:03.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:03.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:03.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:04.050 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:04.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:04.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:04.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:04.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.237 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:04.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:04.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:04.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:04.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:04.474 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:04.506 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:04.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:04.531 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:27:04.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:04.645 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:04.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/117757290' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/117757290' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:04.764 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:04.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:04.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:04.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:04.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:04.930 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:04.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:04 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/117757290' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:27:05.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:05.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:05.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:05.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 339, 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:27:04.505016+0000", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:20:22.465407+0000", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:15:28.332713+0000", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5804032, 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-17T08:27:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 19, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 10, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:27:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 11, 2024-09-17T08:27:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 11, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 11, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 11, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 11, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "334", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "332", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:27:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:27:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.6399999856948853, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.6399999856948853, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:27:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:27:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9900000095367432, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9900000095367432, 2024-09-17T08:27:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1699999570846558, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:27:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.0699999332427979, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.0699999332427979, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.75, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.75, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:27:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:27:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 10, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_3", 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:24:02.853479+0000", 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:27:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:27:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "290", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:27:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:27:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:27:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.3300000429153442, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:27:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.75, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.75 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:27:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:27:05.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.226 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:05.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:27:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 18, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 204, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 227, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 205, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/359954782", 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/359954782", 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/359954782", 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/359954782", 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:27:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:27:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:27:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:27:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:27:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 205, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 227, 2024-09-17T08:27:05.257 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.258 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.258 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:27:05.262 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:27:05.263 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 337, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:27:05.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:03.856617+0000", 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:09.522096+0000", 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.271 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:34.167737+0000", 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 204 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:25:46.154786+0000", 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:27.165173+0000", 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:25:43.193024+0000", 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.274 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:11.432976+0000", 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:26:47.636815+0000", 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:27:05.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: 4, 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: 6 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.276 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:27:05.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "6.b", 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:27:05.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 6 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.280 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:27:05.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:05.283 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:27:05.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:05.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.369 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:05.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:05.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:05.514 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:05.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:05.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:05.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.663 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:05.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:05.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:05.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.795 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:05.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: pgmap v762: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 27 KiB/s rd, 220 KiB/s wr, 1.40k op/s; 197 KiB/s, 0 objects/s recovering 2024-09-17T08:27:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2699115003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: pgmap v762: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 27 KiB/s rd, 220 KiB/s wr, 1.40k op/s; 197 KiB/s, 0 objects/s recovering 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T08:27:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:27:05.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:27:05.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:05 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2699115003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:27:05.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:05.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:05.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:05.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: pgmap v762: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 27 KiB/s rd, 220 KiB/s wr, 1.40k op/s; 197 KiB/s, 0 objects/s recovering 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:27:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:05 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2699115003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:27:06.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:06.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.071 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:06.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:06.172 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:06.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:06.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:06.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:06.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.341 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:06.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:06.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:06.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:06.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:06.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:06.578 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:06.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:06.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:06.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:06.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:06.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:27:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:27:06.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:27:06.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:27:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:27:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:27:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:27:06.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:06 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:27:06.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:06.910 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:06.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:06.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:06 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:27:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:06 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:27:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:06 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:27:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:06 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:27:07.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:07.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:07.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:07.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:07.179 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:07.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:07.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:07.315 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:07.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:07.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:07.451 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:07.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:07.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:07.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:07.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:07.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:27:07.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:07.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:07.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:07.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:07.843 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:07.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:07 smithi079 ceph-mon[40649]: pgmap v764: 158 pgs: 158 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 25 KiB/s rd, 196 KiB/s wr, 1.47k op/s; 176 KiB/s, 0 objects/s recovering 2024-09-17T08:27:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:07 smithi079 ceph-mon[40649]: 10.b scrub starts 2024-09-17T08:27:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:07 smithi079 ceph-mon[29565]: pgmap v764: 158 pgs: 158 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 25 KiB/s rd, 196 KiB/s wr, 1.47k op/s; 176 KiB/s, 0 objects/s recovering 2024-09-17T08:27:07.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:07 smithi079 ceph-mon[29565]: 10.b scrub starts 2024-09-17T08:27:07.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:07 smithi173 ceph-mon[33187]: pgmap v764: 158 pgs: 158 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 25 KiB/s rd, 196 KiB/s wr, 1.47k op/s; 176 KiB/s, 0 objects/s recovering 2024-09-17T08:27:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:07 smithi173 ceph-mon[33187]: 10.b scrub starts 2024-09-17T08:27:07.981 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:07.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:07.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:08.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:08.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:08.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:08.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:08.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.255 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:08.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:08.356 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:08.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:08.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:08.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:08.522 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:08.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:08.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:08.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:08.774 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:08.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.814 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:08.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 10.b scrub ok 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 10.6 scrub starts 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:27:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 10.b scrub ok 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 10.6 scrub starts 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:27:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:27:08.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:08 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:27:08.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:08.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:08.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:08.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 10.b scrub ok 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 10.6 scrub starts 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:27:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:08 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:27:09.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:09.092 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:09.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:09.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:09.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:09.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:09.364 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:09.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:09.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:09.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.506 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:09.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:09.607 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:09.638 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:09.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:09.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:09.782 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:09.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[40649]: pgmap v765: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 12 KiB/s rd, 88 KiB/s wr, 1.30k op/s; 95 KiB/s, 0 objects/s recovering 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[40649]: 10.6 scrub ok 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[29565]: pgmap v765: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 12 KiB/s rd, 88 KiB/s wr, 1.30k op/s; 95 KiB/s, 0 objects/s recovering 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[29565]: 10.6 scrub ok 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:27:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:09 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:27:09.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:09.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:09.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:09.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:09 smithi173 ceph-mon[33187]: pgmap v765: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 12 KiB/s rd, 88 KiB/s wr, 1.30k op/s; 95 KiB/s, 0 objects/s recovering 2024-09-17T08:27:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:09 smithi173 ceph-mon[33187]: 10.6 scrub ok 2024-09-17T08:27:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:09 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:27:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:09 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:27:10.019 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:10.053 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:10.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:10.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:10.194 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:10.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:10.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:10.326 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:10.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.327 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:10.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:10.456 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:10.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:10.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:10.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.591 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:10.591 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:10.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:10.734 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:10.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:10.836 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:10.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:10 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:27:10.861 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:10 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:27:10.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:10 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:27:10.861 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:10 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:27:10.869 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:10.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:10.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:10.894 INFO:tasks.thrashosds.thrasher:j is {'epoch': 339, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:27:04.505016+0000', 'last_up_change': '2024-09-17T08:20:22.465407+0000', 'last_in_change': '2024-09-17T08:15:28.332713+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '334', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '332', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 1.6399999856948853, 'score_stable': 1.6399999856948853, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 1.1699999570846558, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.069999933242798, 'score_stable': 2.069999933242798, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-17T08:24:02.853479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '290', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 337, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 337, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 18, 'last_clean_end': 204, 'up_from': 227, 'up_thru': 337, 'down_at': 205, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 359954782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 359954782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 359954782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 359954782}]}, 'public_addr': '172.21.15.79:6819/359954782', 'cluster_addr': '172.21.15.79:6821/359954782', 'heartbeat_back_addr': '172.21.15.79:6825/359954782', 'heartbeat_front_addr': '172.21.15.79:6823/359954782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 337, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 337, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 205, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 227, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 337, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:03.856617+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:09.522096+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:34.167737+0000', 'dead_epoch': 204}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:25:46.154786+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:27.165173+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:25:43.193024+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:11.432976+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:26:47.636815+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.10', 'osds': [4, 5, 6]}], 'pg_upmap_items': [{'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 2}, {'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:27:10.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:27:10.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:10 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:27:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:10 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:27:11.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.013 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:11.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:11.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:11.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:11.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:11.268 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:11.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.310 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:11.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:11.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:11.436 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:11.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:11.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:11.576 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:11.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:11.677 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:11.709 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:11.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:11.811 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:11.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: pgmap v766: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:27:11.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: pgmap v766: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:27:11.843 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:11 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:27:11.845 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:11.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:11.947 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: pgmap v766: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:27:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:27:11.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:27:11.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:11 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:27:11.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:11.982 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:11.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:12.058 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:12.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:12.113 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:12.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:12.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:12.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.258 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:12.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:12.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:12.364 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:27:12.364 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:27:12.364 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:27:12.364 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:27:12.364 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:27:12.410 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:12.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:12.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:12.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:12.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:12.644 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:12.680 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:12.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:12.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:12.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:12.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 10.7 scrub starts 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 10.7 scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T08:27:12.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 10.7 scrub starts 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 10.7 scrub ok 2024-09-17T08:27:12.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:12.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:12.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T08:27:12.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:27:12.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:12 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:27:12.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:12.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:12.956 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:12.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 10.7 scrub starts 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 10.7 scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T08:27:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:27:12.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:12 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:27:13.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:13.107 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:13.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:13.142 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:13.207 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:13.210 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:13.237 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:13.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:13.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:13.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:13.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:13.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:13.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.499 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:13.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:13.586 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:13.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:13.645 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:13.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 10.f scrub starts 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:27:13.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 10.f scrub ok 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 10.f scrub starts 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:13.727 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 10.f scrub ok 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:27:13.728 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:13 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:27:13.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:13.775 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:13.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.777 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:13.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:13.922 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:13.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:13.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:13.970 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:27:13.970 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:27:13.970 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:27:13.970 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:27:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: pgmap v768: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 10.f scrub starts 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 10.f scrub ok 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:27:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:13 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:27:13.980 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:27:13.981 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:27:14.009 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:27:14.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:27:14.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:14.043 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:27:14.043 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:27:14.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:14.071 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:27:14.072 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:27:14.105 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:27:14.106 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:27:14.136 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:27:14.136 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:27:14.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:14.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:14.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:14.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:14.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:14.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:14.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:14.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:14.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:14.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:14.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:14.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:14.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:14.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:14.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:27:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: 10.4 scrub starts 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4111123826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: 10.4 scrub starts 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:27:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:27:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:14.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:14 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4111123826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:14.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:14.929 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:14.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:14.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: 10.4 scrub starts 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:14 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4111123826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:15.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:15.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:15.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:15.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:15.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:15.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:15.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:15.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:15.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:15.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:15.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:15.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:15.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:15.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:15.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:15.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:15.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: pgmap v770: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: 10.4 scrub ok 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:27:15.803 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: pgmap v770: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: 10.4 scrub ok 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T08:27:15.804 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:27:15.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:15 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:27:15.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:15.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:15.886 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:15.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: pgmap v770: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: 10.4 scrub ok 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:27:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:15 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:27:15.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:16.014 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:27:16.017 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:27:16.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:16.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:16.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:16.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:16.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:16.259 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:16.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:16.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:16.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:16.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.433 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:16.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:16.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:16.568 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:16.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:16.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:16.706 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:16.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:16.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:16.847 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[29565]: 10.1 scrub starts 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[29565]: pgmap v772: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[29565]: 10.1 scrub ok 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[29565]: from='client.25673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[29565]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[40649]: 10.1 scrub starts 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[40649]: pgmap v772: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:16.848 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[40649]: 10.1 scrub ok 2024-09-17T08:27:16.849 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[40649]: from='client.25673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:27:16.849 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:16 smithi079 ceph-mon[40649]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T08:27:16.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:16.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:16.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:16 smithi173 ceph-mon[33187]: 10.1 scrub starts 2024-09-17T08:27:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:16 smithi173 ceph-mon[33187]: pgmap v772: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-17T08:27:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:16 smithi173 ceph-mon[33187]: 10.1 scrub ok 2024-09-17T08:27:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:16 smithi173 ceph-mon[33187]: from='client.25673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:27:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:16 smithi173 ceph-mon[33187]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T08:27:16.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:16.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:16.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:17.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:17.121 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:17.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:17.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:17.261 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:17.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:17.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:17.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:17.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:17.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:17.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:17.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:17.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:17.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:17.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:17.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:17.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:17.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.829 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:27:17.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:27:17.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:27:17.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:27:17.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:27:17.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:17 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:27:17.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:17.967 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:17.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:17.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:27:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:27:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:27:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:27:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:27:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:17 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:27:18.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:18.100 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:18.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:18.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:18.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:18.338 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:18.379 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:18.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:18.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:18.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:27:18.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:18.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:18.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:18.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:18.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:18.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:18.790 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:18.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:18.812 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: pgmap v774: 158 pgs: 158 active+clean; 1.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.52k op/s 2024-09-17T08:27:18.813 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: pgmap v774: 158 pgs: 158 active+clean; 1.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.52k op/s 2024-09-17T08:27:18.814 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:18.846 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":772,"stamp":"2024-09-17T08:27:15.600259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2066749742,"num_objects":31724,"num_object_clones":0,"num_object_copies":95172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31724,"num_whiteouts":0,"num_read":2035,"num_read_kb":1883,"num_write":234427,"num_write_kb":8484316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":2103409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84674,"ondisk_log_size":84674,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10582852,"kb_used_data":6205800,"kb_used_omap":15,"kb_used_meta":4376816,"kb_avail":739378364,"statfs":{"total":767960285184,"available":757123444736,"internally_reserved":0,"allocated":6354739200,"data_stored":6351898102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":4481860373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,5,2,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-410189824,"num_objects":-6259,"num_object_clones":0,"num_object_copies":-18777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6259,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6259,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002024"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":363,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.669119+0000","last_change":"2024-09-17T08:27:12.669483+0000","last_active":"2024-09-17T08:27:12.669119+0000","last_peered":"2024-09-17T08:27:12.669119+0000","last_clean":"2024-09-17T08:27:12.669119+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:27:12.669119+0000","last_undegraded":"2024-09-17T08:27:12.669119+0000","last_fullsized":"2024-09-17T08:27:12.669119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:12.669002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.669002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:28.463354+0000","scrub_duration":0.0039241620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":381,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537168+0000","last_change":"2024-09-17T08:26:23.241237+0000","last_active":"2024-09-17T08:27:04.537168+0000","last_peered":"2024-09-17T08:27:04.537168+0000","last_clean":"2024-09-17T08:27:04.537168+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:27:04.537168+0000","last_undegraded":"2024-09-17T08:27:04.537168+0000","last_fullsized":"2024-09-17T08:27:04.537168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.241125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.241125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:49.606056+0000","scrub_duration":0.002185354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":430,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585662+0000","last_change":"2024-09-17T08:26:21.027223+0000","last_active":"2024-09-17T08:27:11.585662+0000","last_peered":"2024-09-17T08:27:11.585662+0000","last_clean":"2024-09-17T08:27:11.585662+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:27:11.585662+0000","last_undegraded":"2024-09-17T08:27:11.585662+0000","last_fullsized":"2024-09-17T08:27:11.585662+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:26:21.027087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.027087+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:45.075711+0000","scrub_duration":0.004246901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":384,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582314+0000","last_change":"2024-09-17T08:26:30.172498+0000","last_active":"2024-09-17T08:27:11.582314+0000","last_peered":"2024-09-17T08:27:11.582314+0000","last_clean":"2024-09-17T08:27:11.582314+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:27:11.582314+0000","last_undegraded":"2024-09-17T08:27:11.582314+0000","last_fullsized":"2024-09-17T08:27:11.582314+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.172198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.172198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.460859+0000","scrub_duration":0.0047583649999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":360,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584083+0000","last_change":"2024-09-17T08:25:12.813837+0000","last_active":"2024-09-17T08:27:11.584083+0000","last_peered":"2024-09-17T08:27:11.584083+0000","last_clean":"2024-09-17T08:27:11.584083+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:27:11.584083+0000","last_undegraded":"2024-09-17T08:27:11.584083+0000","last_fullsized":"2024-09-17T08:27:11.584083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.813704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.813704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071490010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":486,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582392+0000","last_change":"2024-09-17T08:24:29.572754+0000","last_active":"2024-09-17T08:27:11.582392+0000","last_peered":"2024-09-17T08:27:11.582392+0000","last_clean":"2024-09-17T08:27:11.582392+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:27:11.582392+0000","last_undegraded":"2024-09-17T08:27:11.582392+0000","last_fullsized":"2024-09-17T08:27:11.582392+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:29.572400+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.572400+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041638179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584632+0000","last_change":"2024-09-17T08:25:29.240062+0000","last_active":"2024-09-17T08:27:11.584632+0000","last_peered":"2024-09-17T08:27:11.584632+0000","last_clean":"2024-09-17T08:27:11.584632+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:27:11.584632+0000","last_undegraded":"2024-09-17T08:27:11.584632+0000","last_fullsized":"2024-09-17T08:27:11.584632+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.239407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.239407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062858009999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":381,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582980+0000","last_change":"2024-09-17T08:25:28.285107+0000","last_active":"2024-09-17T08:27:11.582980+0000","last_peered":"2024-09-17T08:27:11.582980+0000","last_clean":"2024-09-17T08:27:11.582980+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:27:11.582980+0000","last_undegraded":"2024-09-17T08:27:11.582980+0000","last_fullsized":"2024-09-17T08:27:11.582980+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.284981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042235850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":379,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589496+0000","last_change":"2024-09-17T08:27:08.741308+0000","last_active":"2024-09-17T08:27:11.589496+0000","last_peered":"2024-09-17T08:27:11.589496+0000","last_clean":"2024-09-17T08:27:11.589496+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:27:11.589496+0000","last_undegraded":"2024-09-17T08:27:11.589496+0000","last_fullsized":"2024-09-17T08:27:11.589496+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:08.740721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:27:08.740721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:23.656037+0000","scrub_duration":0.0025761759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":401,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582031+0000","last_change":"2024-09-17T08:24:16.615788+0000","last_active":"2024-09-17T08:27:11.582031+0000","last_peered":"2024-09-17T08:27:11.582031+0000","last_clean":"2024-09-17T08:27:11.582031+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:27:11.582031+0000","last_undegraded":"2024-09-17T08:27:11.582031+0000","last_fullsized":"2024-09-17T08:27:11.582031+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:24:16.615683+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:24:16.615683+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001772476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":404,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588194+0000","last_change":"2024-09-17T08:26:34.818543+0000","last_active":"2024-09-17T08:27:11.588194+0000","last_peered":"2024-09-17T08:27:11.588194+0000","last_clean":"2024-09-17T08:27:11.588194+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:27:11.588194+0000","last_undegraded":"2024-09-17T08:27:11.588194+0000","last_fullsized":"2024-09-17T08:27:11.588194+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:34.818425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:26:34.818425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:39.478375+0000","scrub_duration":0.008877533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":259,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584498+0000","last_change":"2024-09-17T08:27:08.306361+0000","last_active":"2024-09-17T08:27:11.584498+0000","last_peered":"2024-09-17T08:27:11.584498+0000","last_clean":"2024-09-17T08:27:11.584498+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:27:11.584498+0000","last_undegraded":"2024-09-17T08:27:11.584498+0000","last_fullsized":"2024-09-17T08:27:11.584498+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:08.306241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:08.306241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:24.760534+0000","scrub_duration":0.0035413850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":394,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589702+0000","last_change":"2024-09-17T08:26:11.668739+0000","last_active":"2024-09-17T08:27:11.589702+0000","last_peered":"2024-09-17T08:27:11.589702+0000","last_clean":"2024-09-17T08:27:11.589702+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:27:11.589702+0000","last_undegraded":"2024-09-17T08:27:11.589702+0000","last_fullsized":"2024-09-17T08:27:11.589702+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.668207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.668207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.803189+0000","scrub_duration":0.0090449410000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":396,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583225+0000","last_change":"2024-09-17T08:25:22.208990+0000","last_active":"2024-09-17T08:27:11.583225+0000","last_peered":"2024-09-17T08:27:11.583225+0000","last_clean":"2024-09-17T08:27:11.583225+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:27:11.583225+0000","last_undegraded":"2024-09-17T08:27:11.583225+0000","last_fullsized":"2024-09-17T08:27:11.583225+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:25:22.208698+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:22.208698+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":267,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537824+0000","last_change":"2024-09-17T08:24:27.797180+0000","last_active":"2024-09-17T08:27:04.537824+0000","last_peered":"2024-09-17T08:27:04.537824+0000","last_clean":"2024-09-17T08:27:04.537824+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:27:04.537824+0000","last_undegraded":"2024-09-17T08:27:04.537824+0000","last_fullsized":"2024-09-17T08:27:04.537824+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:27.796880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:27.796880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049344319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":381,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.539208+0000","last_change":"2024-09-17T08:24:20.752436+0000","last_active":"2024-09-17T08:27:04.539208+0000","last_peered":"2024-09-17T08:27:04.539208+0000","last_clean":"2024-09-17T08:27:04.539208+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:27:04.539208+0000","last_undegraded":"2024-09-17T08:27:04.539208+0000","last_fullsized":"2024-09-17T08:27:04.539208+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:20.752313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:20.752313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027283759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"75'130","reported_seq":295,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.530395+0000","last_change":"2024-09-17T08:26:56.656799+0000","last_active":"2024-09-17T08:27:04.530395+0000","last_peered":"2024-09-17T08:27:04.530395+0000","last_clean":"2024-09-17T08:27:04.530395+0000","last_became_active":"2024-09-17T08:26:56.652585+0000","last_became_peered":"2024-09-17T08:26:56.652585+0000","last_unstale":"2024-09-17T08:27:04.530395+0000","last_undegraded":"2024-09-17T08:27:04.530395+0000","last_fullsized":"2024-09-17T08:27:04.530395+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"340'14621","reported_seq":14835,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.770391+0000","last_change":"2024-09-17T08:25:28.636224+0000","last_active":"2024-09-17T08:27:11.770391+0000","last_peered":"2024-09-17T08:27:11.770391+0000","last_clean":"2024-09-17T08:27:11.770391+0000","last_became_active":"2024-09-17T08:24:04.453791+0000","last_became_peered":"2024-09-17T08:24:04.453791+0000","last_unstale":"2024-09-17T08:27:11.770391+0000","last_undegraded":"2024-09-17T08:27:11.770391+0000","last_fullsized":"2024-09-17T08:27:11.770391+0000","mapping_epoch":287,"log_start":"321'8829","ondisk_log_start":"321'8829","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6965","last_scrub_stamp":"2024-09-17T08:25:28.635952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.635952+0000","objects_scrubbed":6875,"log_size":5792,"log_dups_size":0,"ondisk_log_size":5792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.5130719930000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135856128,"num_objects":2073,"num_object_clones":0,"num_object_copies":6219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14621,"num_write_kb":534208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":503,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584331+0000","last_change":"2024-09-17T08:27:11.266259+0000","last_active":"2024-09-17T08:27:11.584331+0000","last_peered":"2024-09-17T08:27:11.584331+0000","last_clean":"2024-09-17T08:27:11.584331+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:27:11.584331+0000","last_undegraded":"2024-09-17T08:27:11.584331+0000","last_fullsized":"2024-09-17T08:27:11.584331+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:27:11.266120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.266120+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:17.860854+0000","scrub_duration":0.002802545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":380,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.587749+0000","last_change":"2024-09-17T08:25:21.870284+0000","last_active":"2024-09-17T08:27:11.587749+0000","last_peered":"2024-09-17T08:27:11.587749+0000","last_clean":"2024-09-17T08:27:11.587749+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:27:11.587749+0000","last_undegraded":"2024-09-17T08:27:11.587749+0000","last_fullsized":"2024-09-17T08:27:11.587749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:21.870174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:21.870174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028963090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":260,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585431+0000","last_change":"2024-09-17T08:26:22.325911+0000","last_active":"2024-09-17T08:27:11.585431+0000","last_peered":"2024-09-17T08:27:11.585431+0000","last_clean":"2024-09-17T08:27:11.585431+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:27:11.585431+0000","last_undegraded":"2024-09-17T08:27:11.585431+0000","last_fullsized":"2024-09-17T08:27:11.585431+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:22.325816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.325816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.578172+0000","scrub_duration":0.0015290900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":359,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:07.785733+0000","last_change":"2024-09-17T08:27:07.785733+0000","last_active":"2024-09-17T08:27:07.785733+0000","last_peered":"2024-09-17T08:27:07.785733+0000","last_clean":"2024-09-17T08:27:07.785733+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:27:07.785733+0000","last_undegraded":"2024-09-17T08:27:07.785733+0000","last_fullsized":"2024-09-17T08:27:07.785733+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:07.785336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:07.785336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:08.429271+0000","scrub_duration":0.014916341,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'130","reported_seq":669,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589107+0000","last_change":"2024-09-17T08:26:56.652512+0000","last_active":"2024-09-17T08:27:11.589107+0000","last_peered":"2024-09-17T08:27:11.589107+0000","last_clean":"2024-09-17T08:27:11.589107+0000","last_became_active":"2024-09-17T08:26:56.652142+0000","last_became_peered":"2024-09-17T08:26:56.652142+0000","last_unstale":"2024-09-17T08:27:11.589107+0000","last_undegraded":"2024-09-17T08:27:11.589107+0000","last_fullsized":"2024-09-17T08:27:11.589107+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012846286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":28,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2","version":"341'14385","reported_seq":14561,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.854086+0000","last_change":"2024-09-17T08:26:33.951791+0000","last_active":"2024-09-17T08:27:13.854086+0000","last_peered":"2024-09-17T08:27:13.854086+0000","last_clean":"2024-09-17T08:27:13.854086+0000","last_became_active":"2024-09-17T08:24:04.455271+0000","last_became_peered":"2024-09-17T08:24:04.455271+0000","last_unstale":"2024-09-17T08:27:13.854086+0000","last_undegraded":"2024-09-17T08:27:13.854086+0000","last_fullsized":"2024-09-17T08:27:13.854086+0000","mapping_epoch":287,"log_start":"321'8983","ondisk_log_start":"321'8983","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'11389","last_scrub_stamp":"2024-09-17T08:26:33.951560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:33.951560+0000","objects_scrubbed":4918,"log_size":5402,"log_dups_size":0,"ondisk_log_size":5402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:01.065252+0000","scrub_duration":1.6680826209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122093568,"num_objects":1863,"num_object_clones":0,"num_object_copies":5589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14385,"num_write_kb":519936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"220'20","reported_seq":398,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588006+0000","last_change":"2024-09-17T08:25:23.209427+0000","last_active":"2024-09-17T08:27:11.588006+0000","last_peered":"2024-09-17T08:27:11.588006+0000","last_clean":"2024-09-17T08:27:11.588006+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:27:11.588006+0000","last_undegraded":"2024-09-17T08:27:11.588006+0000","last_fullsized":"2024-09-17T08:27:11.588006+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:25:23.209287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.209287+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047713479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":249,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586686+0000","last_change":"2024-09-17T08:25:33.132550+0000","last_active":"2024-09-17T08:27:11.586686+0000","last_peered":"2024-09-17T08:27:11.586686+0000","last_clean":"2024-09-17T08:27:11.586686+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:27:11.586686+0000","last_undegraded":"2024-09-17T08:27:11.586686+0000","last_fullsized":"2024-09-17T08:27:11.586686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:33.131878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.131878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008861519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":534,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:08.039270+0000","last_change":"2024-09-17T08:25:11.078189+0000","last_active":"2024-09-17T08:27:08.039270+0000","last_peered":"2024-09-17T08:27:08.039270+0000","last_clean":"2024-09-17T08:27:08.039270+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:27:08.039270+0000","last_undegraded":"2024-09-17T08:27:08.039270+0000","last_fullsized":"2024-09-17T08:27:08.039270+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:11.078003+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:25:11.078003+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057575789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":387,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583641+0000","last_change":"2024-09-17T08:26:32.232090+0000","last_active":"2024-09-17T08:27:11.583641+0000","last_peered":"2024-09-17T08:27:11.583641+0000","last_clean":"2024-09-17T08:27:11.583641+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:27:11.583641+0000","last_undegraded":"2024-09-17T08:27:11.583641+0000","last_fullsized":"2024-09-17T08:27:11.583641+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:32.231846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:26:32.231846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.225179+0000","scrub_duration":0.0087096559999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'130","reported_seq":296,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.529229+0000","last_change":"2024-09-17T08:26:56.658938+0000","last_active":"2024-09-17T08:27:04.529229+0000","last_peered":"2024-09-17T08:27:04.529229+0000","last_clean":"2024-09-17T08:27:04.529229+0000","last_became_active":"2024-09-17T08:26:56.658579+0000","last_became_peered":"2024-09-17T08:26:56.658579+0000","last_unstale":"2024-09-17T08:27:04.529229+0000","last_undegraded":"2024-09-17T08:27:04.529229+0000","last_fullsized":"2024-09-17T08:27:04.529229+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1","version":"341'14794","reported_seq":14858,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.731441+0000","last_change":"2024-09-17T08:24:04.459231+0000","last_active":"2024-09-17T08:27:12.731441+0000","last_peered":"2024-09-17T08:27:12.731441+0000","last_clean":"2024-09-17T08:27:12.731441+0000","last_became_active":"2024-09-17T08:24:04.458930+0000","last_became_peered":"2024-09-17T08:24:04.458930+0000","last_unstale":"2024-09-17T08:27:12.731441+0000","last_undegraded":"2024-09-17T08:27:12.731441+0000","last_fullsized":"2024-09-17T08:27:12.731441+0000","mapping_epoch":287,"log_start":"323'10035","ondisk_log_start":"323'10035","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134610944,"num_objects":2054,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14794,"num_write_kb":539136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":250,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:06.751387+0000","last_change":"2024-09-17T08:25:39.196006+0000","last_active":"2024-09-17T08:27:06.751387+0000","last_peered":"2024-09-17T08:27:06.751387+0000","last_clean":"2024-09-17T08:27:06.751387+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:27:06.751387+0000","last_undegraded":"2024-09-17T08:27:06.751387+0000","last_fullsized":"2024-09-17T08:27:06.751387+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:39.195587+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.195587+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050104839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":249,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584567+0000","last_change":"2024-09-17T08:25:12.167258+0000","last_active":"2024-09-17T08:27:11.584567+0000","last_peered":"2024-09-17T08:27:11.584567+0000","last_clean":"2024-09-17T08:27:11.584567+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:27:11.584567+0000","last_undegraded":"2024-09-17T08:27:11.584567+0000","last_fullsized":"2024-09-17T08:27:11.584567+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:12.167132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.167132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020782719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":582,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.269298+0000","last_change":"2024-09-17T08:27:12.269298+0000","last_active":"2024-09-17T08:27:12.269298+0000","last_peered":"2024-09-17T08:27:12.269298+0000","last_clean":"2024-09-17T08:27:12.269298+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:27:12.269298+0000","last_undegraded":"2024-09-17T08:27:12.269298+0000","last_fullsized":"2024-09-17T08:27:12.269298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:27:12.269195+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.269195+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:13.482267+0000","scrub_duration":0.017091408999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":249,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585073+0000","last_change":"2024-09-17T08:25:28.215581+0000","last_active":"2024-09-17T08:27:11.585073+0000","last_peered":"2024-09-17T08:27:11.585073+0000","last_clean":"2024-09-17T08:27:11.585073+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:27:11.585073+0000","last_undegraded":"2024-09-17T08:27:11.585073+0000","last_fullsized":"2024-09-17T08:27:11.585073+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:28.215457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:28.215457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070925789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"341'14730","reported_seq":14901,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.843525+0000","last_change":"2024-09-17T08:26:35.602206+0000","last_active":"2024-09-17T08:27:13.843525+0000","last_peered":"2024-09-17T08:27:13.843525+0000","last_clean":"2024-09-17T08:27:13.843525+0000","last_became_active":"2024-09-17T08:24:04.455098+0000","last_became_peered":"2024-09-17T08:24:04.455098+0000","last_unstale":"2024-09-17T08:27:13.843525+0000","last_undegraded":"2024-09-17T08:27:13.843525+0000","last_fullsized":"2024-09-17T08:27:13.843525+0000","mapping_epoch":287,"log_start":"321'9367","ondisk_log_start":"321'9367","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'11853","last_scrub_stamp":"2024-09-17T08:26:35.601918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:35.601918+0000","objects_scrubbed":4775,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:50.669911+0000","scrub_duration":1.327324146,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121896960,"num_objects":1860,"num_object_clones":0,"num_object_copies":5580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14730,"num_write_kb":530880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":436,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583767+0000","last_change":"2024-09-17T08:24:25.581898+0000","last_active":"2024-09-17T08:27:11.583767+0000","last_peered":"2024-09-17T08:27:11.583767+0000","last_clean":"2024-09-17T08:27:11.583767+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:27:11.583767+0000","last_undegraded":"2024-09-17T08:27:11.583767+0000","last_fullsized":"2024-09-17T08:27:11.583767+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:24:25.581740+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:24:25.581740+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004957894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":387,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588754+0000","last_change":"2024-09-17T08:26:12.688090+0000","last_active":"2024-09-17T08:27:11.588754+0000","last_peered":"2024-09-17T08:27:11.588754+0000","last_clean":"2024-09-17T08:27:11.588754+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:27:11.588754+0000","last_undegraded":"2024-09-17T08:27:11.588754+0000","last_fullsized":"2024-09-17T08:27:11.588754+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:12.687091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.687091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.608122+0000","scrub_duration":0.006317067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":259,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582946+0000","last_change":"2024-09-17T08:24:30.522675+0000","last_active":"2024-09-17T08:27:11.582946+0000","last_peered":"2024-09-17T08:27:11.582946+0000","last_clean":"2024-09-17T08:27:11.582946+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:27:11.582946+0000","last_undegraded":"2024-09-17T08:27:11.582946+0000","last_fullsized":"2024-09-17T08:27:11.582946+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:24:30.522581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001427191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":353,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583375+0000","last_change":"2024-09-17T08:25:40.909763+0000","last_active":"2024-09-17T08:27:11.583375+0000","last_peered":"2024-09-17T08:27:11.583375+0000","last_clean":"2024-09-17T08:27:11.583375+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:27:11.583375+0000","last_undegraded":"2024-09-17T08:27:11.583375+0000","last_fullsized":"2024-09-17T08:27:11.583375+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.909493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.909493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014995481999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'130","reported_seq":297,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583189+0000","last_change":"2024-09-17T08:26:56.649503+0000","last_active":"2024-09-17T08:27:11.583189+0000","last_peered":"2024-09-17T08:27:11.583189+0000","last_clean":"2024-09-17T08:27:11.583189+0000","last_became_active":"2024-09-17T08:26:56.649122+0000","last_became_peered":"2024-09-17T08:26:56.649122+0000","last_unstale":"2024-09-17T08:27:11.583189+0000","last_undegraded":"2024-09-17T08:27:11.583189+0000","last_fullsized":"2024-09-17T08:27:11.583189+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":488690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"341'14625","reported_seq":14804,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.742914+0000","last_change":"2024-09-17T08:27:12.331163+0000","last_active":"2024-09-17T08:27:13.742914+0000","last_peered":"2024-09-17T08:27:13.742914+0000","last_clean":"2024-09-17T08:27:13.742914+0000","last_became_active":"2024-09-17T08:24:04.454206+0000","last_became_peered":"2024-09-17T08:24:04.454206+0000","last_unstale":"2024-09-17T08:27:13.742914+0000","last_undegraded":"2024-09-17T08:27:13.742914+0000","last_fullsized":"2024-09-17T08:27:13.742914+0000","mapping_epoch":287,"log_start":"323'9812","ondisk_log_start":"323'9812","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'14526","last_scrub_stamp":"2024-09-17T08:27:12.330512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.330512+0000","objects_scrubbed":1991,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:35.982204+0000","scrub_duration":0.66116257899999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123273216,"num_objects":1881,"num_object_clones":0,"num_object_copies":5643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14625,"num_write_kb":528192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"220'28","reported_seq":400,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583250+0000","last_change":"2024-09-17T08:25:12.405766+0000","last_active":"2024-09-17T08:27:11.583250+0000","last_peered":"2024-09-17T08:27:11.583250+0000","last_clean":"2024-09-17T08:27:11.583250+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:27:11.583250+0000","last_undegraded":"2024-09-17T08:27:11.583250+0000","last_fullsized":"2024-09-17T08:27:11.583250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'28","last_scrub_stamp":"2024-09-17T08:25:12.405574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:12.405574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00623603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589405+0000","last_change":"2024-09-17T08:26:29.708674+0000","last_active":"2024-09-17T08:27:11.589405+0000","last_peered":"2024-09-17T08:27:11.589405+0000","last_clean":"2024-09-17T08:27:11.589405+0000","last_became_active":"2024-09-17T08:23:18.669266+0000","last_became_peered":"2024-09-17T08:23:18.669266+0000","last_unstale":"2024-09-17T08:27:11.589405+0000","last_undegraded":"2024-09-17T08:27:11.589405+0000","last_fullsized":"2024-09-17T08:27:11.589405+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:29.708555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:26:29.708555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:33.829571+0000","scrub_duration":0.0021163079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":604,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.598371+0000","last_change":"2024-09-17T08:24:12.872168+0000","last_active":"2024-09-17T08:27:12.598371+0000","last_peered":"2024-09-17T08:27:12.598371+0000","last_clean":"2024-09-17T08:27:12.598371+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:27:12.598371+0000","last_undegraded":"2024-09-17T08:27:12.598371+0000","last_fullsized":"2024-09-17T08:27:12.598371+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:24:12.872090+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:24:12.872090+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":380,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.587489+0000","last_change":"2024-09-17T08:25:29.822800+0000","last_active":"2024-09-17T08:27:11.587489+0000","last_peered":"2024-09-17T08:27:11.587489+0000","last_clean":"2024-09-17T08:27:11.587489+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:27:11.587489+0000","last_undegraded":"2024-09-17T08:27:11.587489+0000","last_fullsized":"2024-09-17T08:27:11.587489+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.822689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.822689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085561849999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'130","reported_seq":296,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582224+0000","last_change":"2024-09-17T08:26:56.650104+0000","last_active":"2024-09-17T08:27:11.582224+0000","last_peered":"2024-09-17T08:27:11.582224+0000","last_clean":"2024-09-17T08:27:11.582224+0000","last_became_active":"2024-09-17T08:26:56.649679+0000","last_became_peered":"2024-09-17T08:26:56.649679+0000","last_unstale":"2024-09-17T08:27:11.582224+0000","last_undegraded":"2024-09-17T08:27:11.582224+0000","last_fullsized":"2024-09-17T08:27:11.582224+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.e","version":"341'14650","reported_seq":14823,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.741022+0000","last_change":"2024-09-17T08:26:35.610159+0000","last_active":"2024-09-17T08:27:12.741022+0000","last_peered":"2024-09-17T08:27:12.741022+0000","last_clean":"2024-09-17T08:27:12.741022+0000","last_became_active":"2024-09-17T08:24:04.452780+0000","last_became_peered":"2024-09-17T08:24:04.452780+0000","last_unstale":"2024-09-17T08:27:12.741022+0000","last_undegraded":"2024-09-17T08:27:12.741022+0000","last_fullsized":"2024-09-17T08:27:12.741022+0000","mapping_epoch":287,"log_start":"323'9895","ondisk_log_start":"323'9895","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'11813","last_scrub_stamp":"2024-09-17T08:26:35.610004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:35.610004+0000","objects_scrubbed":4932,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:55.701693+0000","scrub_duration":1.3984958249999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134086656,"num_objects":2046,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14650,"num_write_kb":534272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":419,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583538+0000","last_change":"2024-09-17T08:26:33.812643+0000","last_active":"2024-09-17T08:27:11.583538+0000","last_peered":"2024-09-17T08:27:11.583538+0000","last_clean":"2024-09-17T08:27:11.583538+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:27:11.583538+0000","last_undegraded":"2024-09-17T08:27:11.583538+0000","last_fullsized":"2024-09-17T08:27:11.583538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:26:33.812404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:33.812404+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.364758+0000","scrub_duration":0.016046656999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":249,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585936+0000","last_change":"2024-09-17T08:25:31.165549+0000","last_active":"2024-09-17T08:27:11.585936+0000","last_peered":"2024-09-17T08:27:11.585936+0000","last_clean":"2024-09-17T08:27:11.585936+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:27:11.585936+0000","last_undegraded":"2024-09-17T08:27:11.585936+0000","last_fullsized":"2024-09-17T08:27:11.585936+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:31.165432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:25:31.165432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001729558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":367,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583110+0000","last_change":"2024-09-17T08:26:23.149322+0000","last_active":"2024-09-17T08:27:11.583110+0000","last_peered":"2024-09-17T08:27:11.583110+0000","last_clean":"2024-09-17T08:27:11.583110+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:27:11.583110+0000","last_undegraded":"2024-09-17T08:27:11.583110+0000","last_fullsized":"2024-09-17T08:27:11.583110+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.149070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.149070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.469461+0000","scrub_duration":0.0050165310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":380,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583151+0000","last_change":"2024-09-17T08:25:39.349334+0000","last_active":"2024-09-17T08:27:11.583151+0000","last_peered":"2024-09-17T08:27:11.583151+0000","last_clean":"2024-09-17T08:27:11.583151+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:27:11.583151+0000","last_undegraded":"2024-09-17T08:27:11.583151+0000","last_fullsized":"2024-09-17T08:27:11.583151+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:39.349120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:39.349120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005278652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"75'130","reported_seq":304,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537425+0000","last_change":"2024-09-17T08:26:56.662209+0000","last_active":"2024-09-17T08:27:04.537425+0000","last_peered":"2024-09-17T08:27:04.537425+0000","last_clean":"2024-09-17T08:27:04.537425+0000","last_became_active":"2024-09-17T08:26:56.661568+0000","last_became_peered":"2024-09-17T08:26:56.661568+0000","last_unstale":"2024-09-17T08:27:04.537425+0000","last_undegraded":"2024-09-17T08:27:04.537425+0000","last_fullsized":"2024-09-17T08:27:04.537425+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"341'14537","reported_seq":14707,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.729276+0000","last_change":"2024-09-17T08:26:36.434232+0000","last_active":"2024-09-17T08:27:12.729276+0000","last_peered":"2024-09-17T08:27:12.729276+0000","last_clean":"2024-09-17T08:27:12.729276+0000","last_became_active":"2024-09-17T08:24:04.456039+0000","last_became_peered":"2024-09-17T08:24:04.456039+0000","last_unstale":"2024-09-17T08:27:12.729276+0000","last_undegraded":"2024-09-17T08:27:12.729276+0000","last_fullsized":"2024-09-17T08:27:12.729276+0000","mapping_epoch":287,"log_start":"323'9804","ondisk_log_start":"323'9804","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"329'11755","last_scrub_stamp":"2024-09-17T08:26:36.434036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:36.434036+0000","objects_scrubbed":4765,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:53.416183+0000","scrub_duration":1.2157273630000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128057368,"num_objects":1955,"num_object_clones":0,"num_object_copies":5865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14537,"num_write_kb":527681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":430,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.538538+0000","last_change":"2024-09-17T08:26:12.052601+0000","last_active":"2024-09-17T08:27:04.538538+0000","last_peered":"2024-09-17T08:27:04.538538+0000","last_clean":"2024-09-17T08:27:04.538538+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:27:04.538538+0000","last_undegraded":"2024-09-17T08:27:04.538538+0000","last_fullsized":"2024-09-17T08:27:04.538538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:26:12.052367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:12.052367+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.321528+0000","scrub_duration":0.0032600939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":387,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589540+0000","last_change":"2024-09-17T08:26:27.789924+0000","last_active":"2024-09-17T08:27:11.589540+0000","last_peered":"2024-09-17T08:27:11.589540+0000","last_clean":"2024-09-17T08:27:11.589540+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:27:11.589540+0000","last_undegraded":"2024-09-17T08:27:11.589540+0000","last_fullsized":"2024-09-17T08:27:11.589540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:26:27.789796+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:27.956988+0000","scrub_duration":0.007393897,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":321,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537901+0000","last_change":"2024-09-17T08:24:15.691704+0000","last_active":"2024-09-17T08:27:04.537901+0000","last_peered":"2024-09-17T08:27:04.537901+0000","last_clean":"2024-09-17T08:27:04.537901+0000","last_became_active":"2024-09-17T08:12:46.184121+0000","last_became_peered":"2024-09-17T08:12:46.184121+0000","last_unstale":"2024-09-17T08:27:04.537901+0000","last_undegraded":"2024-09-17T08:27:04.537901+0000","last_fullsized":"2024-09-17T08:27:04.537901+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:15.691622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:24:15.691622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034962180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":275,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584703+0000","last_change":"2024-09-17T08:26:36.218759+0000","last_active":"2024-09-17T08:27:11.584703+0000","last_peered":"2024-09-17T08:27:11.584703+0000","last_clean":"2024-09-17T08:27:11.584703+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:27:11.584703+0000","last_undegraded":"2024-09-17T08:27:11.584703+0000","last_fullsized":"2024-09-17T08:27:11.584703+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:36.218614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:36.218614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.518242+0000","scrub_duration":0.0017513920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"75'130","reported_seq":297,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584310+0000","last_change":"2024-09-17T08:26:56.651612+0000","last_active":"2024-09-17T08:27:11.584310+0000","last_peered":"2024-09-17T08:27:11.584310+0000","last_clean":"2024-09-17T08:27:11.584310+0000","last_became_active":"2024-09-17T08:26:56.651279+0000","last_became_peered":"2024-09-17T08:26:56.651279+0000","last_unstale":"2024-09-17T08:27:11.584310+0000","last_undegraded":"2024-09-17T08:27:11.584310+0000","last_fullsized":"2024-09-17T08:27:11.584310+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"341'14708","reported_seq":14931,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.820298+0000","last_change":"2024-09-17T08:25:36.457603+0000","last_active":"2024-09-17T08:27:12.820298+0000","last_peered":"2024-09-17T08:27:12.820298+0000","last_clean":"2024-09-17T08:27:12.820298+0000","last_became_active":"2024-09-17T08:24:04.458943+0000","last_became_peered":"2024-09-17T08:24:04.458943+0000","last_unstale":"2024-09-17T08:27:12.820298+0000","last_undegraded":"2024-09-17T08:27:12.820298+0000","last_fullsized":"2024-09-17T08:27:12.820298+0000","mapping_epoch":287,"log_start":"325'10015","ondisk_log_start":"325'10015","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'7478","last_scrub_stamp":"2024-09-17T08:25:36.457326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:25:36.457326+0000","objects_scrubbed":7398,"log_size":4693,"log_dups_size":0,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.6174414289999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129499136,"num_objects":1976,"num_object_clones":0,"num_object_copies":5928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14708,"num_write_kb":533888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":292,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.259887+0000","last_change":"2024-09-17T08:27:12.259887+0000","last_active":"2024-09-17T08:27:12.259887+0000","last_peered":"2024-09-17T08:27:12.259887+0000","last_clean":"2024-09-17T08:27:12.259887+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:27:12.259887+0000","last_undegraded":"2024-09-17T08:27:12.259887+0000","last_fullsized":"2024-09-17T08:27:12.259887+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:27:12.259507+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.259507+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:22.936414+0000","scrub_duration":0.011044918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":249,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585645+0000","last_change":"2024-09-17T08:25:14.210636+0000","last_active":"2024-09-17T08:27:11.585645+0000","last_peered":"2024-09-17T08:27:11.585645+0000","last_clean":"2024-09-17T08:27:11.585645+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:27:11.585645+0000","last_undegraded":"2024-09-17T08:27:11.585645+0000","last_fullsized":"2024-09-17T08:27:11.585645+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:14.210528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:25:14.210528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019364110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":593,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:08.038939+0000","last_change":"2024-09-17T08:26:22.193822+0000","last_active":"2024-09-17T08:27:08.038939+0000","last_peered":"2024-09-17T08:27:08.038939+0000","last_clean":"2024-09-17T08:27:08.038939+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:27:08.038939+0000","last_undegraded":"2024-09-17T08:27:08.038939+0000","last_fullsized":"2024-09-17T08:27:08.038939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:26:22.193701+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.193701+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.638128+0000","scrub_duration":0.0030149489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":380,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.538904+0000","last_change":"2024-09-17T08:25:37.001934+0000","last_active":"2024-09-17T08:27:04.538904+0000","last_peered":"2024-09-17T08:27:04.538904+0000","last_clean":"2024-09-17T08:27:04.538904+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:27:04.538904+0000","last_undegraded":"2024-09-17T08:27:04.538904+0000","last_fullsized":"2024-09-17T08:27:04.538904+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:37.001438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.001438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093143449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'130","reported_seq":304,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.536846+0000","last_change":"2024-09-17T08:26:56.664302+0000","last_active":"2024-09-17T08:27:04.536846+0000","last_peered":"2024-09-17T08:27:04.536846+0000","last_clean":"2024-09-17T08:27:04.536846+0000","last_became_active":"2024-09-17T08:26:56.663984+0000","last_became_peered":"2024-09-17T08:26:56.663984+0000","last_unstale":"2024-09-17T08:27:04.536846+0000","last_undegraded":"2024-09-17T08:27:04.536846+0000","last_fullsized":"2024-09-17T08:27:04.536846+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"340'14458","reported_seq":14642,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.505426+0000","last_change":"2024-09-17T08:26:25.894830+0000","last_active":"2024-09-17T08:27:12.505426+0000","last_peered":"2024-09-17T08:27:12.505426+0000","last_clean":"2024-09-17T08:27:12.505426+0000","last_became_active":"2024-09-17T08:24:04.457614+0000","last_became_peered":"2024-09-17T08:24:04.457614+0000","last_unstale":"2024-09-17T08:27:12.505426+0000","last_undegraded":"2024-09-17T08:27:12.505426+0000","last_fullsized":"2024-09-17T08:27:12.505426+0000","mapping_epoch":287,"log_start":"321'9366","ondisk_log_start":"321'9366","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10898","last_scrub_stamp":"2024-09-17T08:26:25.894587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.894587+0000","objects_scrubbed":5529,"log_size":5092,"log_dups_size":0,"ondisk_log_size":5092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.232827+0000","scrub_duration":1.670279176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125304832,"num_objects":1912,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14458,"num_write_kb":523840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":462,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.591824+0000","last_change":"2024-09-17T08:26:27.354029+0000","last_active":"2024-09-17T08:27:12.591824+0000","last_peered":"2024-09-17T08:27:12.591824+0000","last_clean":"2024-09-17T08:27:12.591824+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:27:12.591824+0000","last_undegraded":"2024-09-17T08:27:12.591824+0000","last_fullsized":"2024-09-17T08:27:12.591824+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:26:27.353856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:27.353856+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:57.299917+0000","scrub_duration":0.0088356479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":260,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586587+0000","last_change":"2024-09-17T08:26:24.311721+0000","last_active":"2024-09-17T08:27:11.586587+0000","last_peered":"2024-09-17T08:27:11.586587+0000","last_clean":"2024-09-17T08:27:11.586587+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:27:11.586587+0000","last_undegraded":"2024-09-17T08:27:11.586587+0000","last_fullsized":"2024-09-17T08:27:11.586587+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:24.311602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:26:24.311602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:35.713175+0000","scrub_duration":0.0041065229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":395,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.540304+0000","last_change":"2024-09-17T08:26:30.174985+0000","last_active":"2024-09-17T08:27:04.540304+0000","last_peered":"2024-09-17T08:27:04.540304+0000","last_clean":"2024-09-17T08:27:04.540304+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:27:04.540304+0000","last_undegraded":"2024-09-17T08:27:04.540304+0000","last_fullsized":"2024-09-17T08:27:04.540304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.174489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.174489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:38.176927+0000","scrub_duration":0.0034970859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":364,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583818+0000","last_change":"2024-09-17T08:25:29.848581+0000","last_active":"2024-09-17T08:27:11.583818+0000","last_peered":"2024-09-17T08:27:11.583818+0000","last_clean":"2024-09-17T08:27:11.583818+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:27:11.583818+0000","last_undegraded":"2024-09-17T08:27:11.583818+0000","last_fullsized":"2024-09-17T08:27:11.583818+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:29.848449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.848449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072796190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'130","reported_seq":669,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588029+0000","last_change":"2024-09-17T08:26:56.653105+0000","last_active":"2024-09-17T08:27:11.588029+0000","last_peered":"2024-09-17T08:27:11.588029+0000","last_clean":"2024-09-17T08:27:11.588029+0000","last_became_active":"2024-09-17T08:26:56.652768+0000","last_became_peered":"2024-09-17T08:26:56.652768+0000","last_unstale":"2024-09-17T08:27:11.588029+0000","last_undegraded":"2024-09-17T08:27:11.588029+0000","last_fullsized":"2024-09-17T08:27:11.588029+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012846286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":28,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"340'14421","reported_seq":14621,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.708045+0000","last_change":"2024-09-17T08:26:18.800526+0000","last_active":"2024-09-17T08:27:11.708045+0000","last_peered":"2024-09-17T08:27:11.708045+0000","last_clean":"2024-09-17T08:27:11.708045+0000","last_became_active":"2024-09-17T08:24:04.453753+0000","last_became_peered":"2024-09-17T08:24:04.453753+0000","last_unstale":"2024-09-17T08:27:11.708045+0000","last_undegraded":"2024-09-17T08:27:11.708045+0000","last_fullsized":"2024-09-17T08:27:11.708045+0000","mapping_epoch":287,"log_start":"320'8566","ondisk_log_start":"320'8566","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10467","last_scrub_stamp":"2024-09-17T08:26:18.800219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.800219+0000","objects_scrubbed":6104,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:43.431424+0000","scrub_duration":2.2200420680000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136380416,"num_objects":2081,"num_object_clones":0,"num_object_copies":6243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14421,"num_write_kb":528064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":267,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586105+0000","last_change":"2024-09-17T08:25:19.161609+0000","last_active":"2024-09-17T08:27:11.586105+0000","last_peered":"2024-09-17T08:27:11.586105+0000","last_clean":"2024-09-17T08:27:11.586105+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:27:11.586105+0000","last_undegraded":"2024-09-17T08:27:11.586105+0000","last_fullsized":"2024-09-17T08:27:11.586105+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:25:19.161454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.161454+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015453119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":981,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.731226+0000","last_change":"2024-09-17T08:27:10.306884+0000","last_active":"2024-09-17T08:27:12.731226+0000","last_peered":"2024-09-17T08:27:12.731226+0000","last_clean":"2024-09-17T08:27:12.731226+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:27:12.731226+0000","last_undegraded":"2024-09-17T08:27:12.731226+0000","last_fullsized":"2024-09-17T08:27:12.731226+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:27:10.306650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:10.306650+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:24.949046+0000","scrub_duration":0.002725903,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":701,"num_read_kb":696,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":354,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.531157+0000","last_change":"2024-09-17T08:26:15.566206+0000","last_active":"2024-09-17T08:27:04.531157+0000","last_peered":"2024-09-17T08:27:04.531157+0000","last_clean":"2024-09-17T08:27:04.531157+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:27:04.531157+0000","last_undegraded":"2024-09-17T08:27:04.531157+0000","last_fullsized":"2024-09-17T08:27:04.531157+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.566090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.566090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:37.464727+0000","scrub_duration":0.0094002110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":362,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.530594+0000","last_change":"2024-09-17T08:26:36.581654+0000","last_active":"2024-09-17T08:27:04.530594+0000","last_peered":"2024-09-17T08:27:04.530594+0000","last_clean":"2024-09-17T08:27:04.530594+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:27:04.530594+0000","last_undegraded":"2024-09-17T08:27:04.530594+0000","last_fullsized":"2024-09-17T08:27:04.530594+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:36.577718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:26:36.577718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:04.804800+0000","scrub_duration":0.0075585510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":386,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.676307+0000","last_change":"2024-09-17T08:27:11.676307+0000","last_active":"2024-09-17T08:27:11.676307+0000","last_peered":"2024-09-17T08:27:11.676307+0000","last_clean":"2024-09-17T08:27:11.676307+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:27:11.676307+0000","last_undegraded":"2024-09-17T08:27:11.676307+0000","last_fullsized":"2024-09-17T08:27:11.676307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:11.676187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.676187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:14.980020+0000","scrub_duration":0.002632587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'130","reported_seq":297,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.587581+0000","last_change":"2024-09-17T08:26:56.646124+0000","last_active":"2024-09-17T08:27:11.587581+0000","last_peered":"2024-09-17T08:27:11.587581+0000","last_clean":"2024-09-17T08:27:11.587581+0000","last_became_active":"2024-09-17T08:26:56.645417+0000","last_became_peered":"2024-09-17T08:26:56.645417+0000","last_unstale":"2024-09-17T08:27:11.587581+0000","last_undegraded":"2024-09-17T08:27:11.587581+0000","last_fullsized":"2024-09-17T08:27:11.587581+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045154770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"340'14692","reported_seq":14876,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.503223+0000","last_change":"2024-09-17T08:26:30.014820+0000","last_active":"2024-09-17T08:27:12.503223+0000","last_peered":"2024-09-17T08:27:12.503223+0000","last_clean":"2024-09-17T08:27:12.503223+0000","last_became_active":"2024-09-17T08:24:04.458862+0000","last_became_peered":"2024-09-17T08:24:04.458862+0000","last_unstale":"2024-09-17T08:27:12.503223+0000","last_undegraded":"2024-09-17T08:27:12.503223+0000","last_fullsized":"2024-09-17T08:27:12.503223+0000","mapping_epoch":287,"log_start":"322'9675","ondisk_log_start":"322'9675","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'11417","last_scrub_stamp":"2024-09-17T08:26:30.014628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.014628+0000","objects_scrubbed":5405,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:51.272301+0000","scrub_duration":1.851197124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135528448,"num_objects":2068,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14692,"num_write_kb":536320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":485,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537979+0000","last_change":"2024-09-17T08:25:37.983837+0000","last_active":"2024-09-17T08:27:04.537979+0000","last_peered":"2024-09-17T08:27:04.537979+0000","last_clean":"2024-09-17T08:27:04.537979+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:27:04.537979+0000","last_undegraded":"2024-09-17T08:27:04.537979+0000","last_fullsized":"2024-09-17T08:27:04.537979+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:25:37.983609+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:25:37.983609+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081599319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":388,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.539936+0000","last_change":"2024-09-17T08:25:17.027645+0000","last_active":"2024-09-17T08:27:04.539936+0000","last_peered":"2024-09-17T08:27:04.539936+0000","last_clean":"2024-09-17T08:27:04.539936+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:27:04.539936+0000","last_undegraded":"2024-09-17T08:27:04.539936+0000","last_fullsized":"2024-09-17T08:27:04.539936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:25:17.027377+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:25:17.027377+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088147999999999994,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":399,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588843+0000","last_change":"2024-09-17T08:25:34.874991+0000","last_active":"2024-09-17T08:27:11.588843+0000","last_peered":"2024-09-17T08:27:11.588843+0000","last_clean":"2024-09-17T08:27:11.588843+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:27:11.588843+0000","last_undegraded":"2024-09-17T08:27:11.588843+0000","last_fullsized":"2024-09-17T08:27:11.588843+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.874848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:13:56.115447+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.874848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082107689999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":260,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586179+0000","last_change":"2024-09-17T08:26:23.359243+0000","last_active":"2024-09-17T08:27:11.586179+0000","last_peered":"2024-09-17T08:27:11.586179+0000","last_clean":"2024-09-17T08:27:11.586179+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:27:11.586179+0000","last_undegraded":"2024-09-17T08:27:11.586179+0000","last_fullsized":"2024-09-17T08:27:11.586179+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.359104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.359104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.887092+0000","scrub_duration":0.0035052579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":260,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585500+0000","last_change":"2024-09-17T08:27:11.262169+0000","last_active":"2024-09-17T08:27:11.585500+0000","last_peered":"2024-09-17T08:27:11.585500+0000","last_clean":"2024-09-17T08:27:11.585500+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:27:11.585500+0000","last_undegraded":"2024-09-17T08:27:11.585500+0000","last_fullsized":"2024-09-17T08:27:11.585500+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:11.262054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.262054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:20.372278+0000","scrub_duration":0.0036509699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"341'14584","reported_seq":14709,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.825657+0000","last_change":"2024-09-17T08:27:06.943175+0000","last_active":"2024-09-17T08:27:13.825657+0000","last_peered":"2024-09-17T08:27:13.825657+0000","last_clean":"2024-09-17T08:27:13.825657+0000","last_became_active":"2024-09-17T08:24:04.455726+0000","last_became_peered":"2024-09-17T08:24:04.455726+0000","last_unstale":"2024-09-17T08:27:13.825657+0000","last_undegraded":"2024-09-17T08:27:13.825657+0000","last_fullsized":"2024-09-17T08:27:13.825657+0000","mapping_epoch":287,"log_start":"321'9220","ondisk_log_start":"321'9220","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'14039","last_scrub_stamp":"2024-09-17T08:27:06.942968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:27:06.942968+0000","objects_scrubbed":2422,"log_size":5364,"log_dups_size":0,"ondisk_log_size":5364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:30.203506+0000","scrub_duration":0.71145537999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121110528,"num_objects":1848,"num_object_clones":0,"num_object_copies":5544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":525824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":370,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584209+0000","last_change":"2024-09-17T08:26:25.062996+0000","last_active":"2024-09-17T08:27:11.584209+0000","last_peered":"2024-09-17T08:27:11.584209+0000","last_clean":"2024-09-17T08:27:11.584209+0000","last_became_active":"2024-09-17T08:12:46.186643+0000","last_became_peered":"2024-09-17T08:12:46.186643+0000","last_unstale":"2024-09-17T08:27:11.584209+0000","last_undegraded":"2024-09-17T08:27:11.584209+0000","last_fullsized":"2024-09-17T08:27:11.584209+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:26:25.062850+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.062850+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:41.215968+0000","scrub_duration":0.0041814790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":367,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586928+0000","last_change":"2024-09-17T08:24:06.873073+0000","last_active":"2024-09-17T08:27:11.586928+0000","last_peered":"2024-09-17T08:27:11.586928+0000","last_clean":"2024-09-17T08:27:11.586928+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:27:11.586928+0000","last_undegraded":"2024-09-17T08:27:11.586928+0000","last_fullsized":"2024-09-17T08:27:11.586928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:24:06.872995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:06.872995+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019197509999999999,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":252,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.531658+0000","last_change":"2024-09-17T08:25:42.221990+0000","last_active":"2024-09-17T08:27:04.531658+0000","last_peered":"2024-09-17T08:27:04.531658+0000","last_clean":"2024-09-17T08:27:04.531658+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:27:04.531658+0000","last_undegraded":"2024-09-17T08:27:04.531658+0000","last_fullsized":"2024-09-17T08:27:04.531658+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:42.221590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:25:42.221590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005773388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":652,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589019+0000","last_change":"2024-09-17T08:26:51.613495+0000","last_active":"2024-09-17T08:27:11.589019+0000","last_peered":"2024-09-17T08:27:11.589019+0000","last_clean":"2024-09-17T08:27:11.589019+0000","last_became_active":"2024-09-17T08:26:51.613195+0000","last_became_peered":"2024-09-17T08:26:51.613195+0000","last_unstale":"2024-09-17T08:27:11.589019+0000","last_undegraded":"2024-09-17T08:27:11.589019+0000","last_fullsized":"2024-09-17T08:27:11.589019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:26:25.775461+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:26:25.775461+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012846286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":27,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":381,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589370+0000","last_change":"2024-09-17T08:25:38.817905+0000","last_active":"2024-09-17T08:27:11.589370+0000","last_peered":"2024-09-17T08:27:11.589370+0000","last_clean":"2024-09-17T08:27:11.589370+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:27:11.589370+0000","last_undegraded":"2024-09-17T08:27:11.589370+0000","last_fullsized":"2024-09-17T08:27:11.589370+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.817546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002079291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":269,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583932+0000","last_change":"2024-09-17T08:25:41.900666+0000","last_active":"2024-09-17T08:27:11.583932+0000","last_peered":"2024-09-17T08:27:11.583932+0000","last_clean":"2024-09-17T08:27:11.583932+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:27:11.583932+0000","last_undegraded":"2024-09-17T08:27:11.583932+0000","last_fullsized":"2024-09-17T08:27:11.583932+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.900483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.900483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013601894,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"340'14561","reported_seq":14624,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.522131+0000","last_change":"2024-09-17T08:24:04.458040+0000","last_active":"2024-09-17T08:27:12.522131+0000","last_peered":"2024-09-17T08:27:12.522131+0000","last_clean":"2024-09-17T08:27:12.522131+0000","last_became_active":"2024-09-17T08:24:04.457745+0000","last_became_peered":"2024-09-17T08:24:04.457745+0000","last_unstale":"2024-09-17T08:27:12.522131+0000","last_undegraded":"2024-09-17T08:27:12.522131+0000","last_fullsized":"2024-09-17T08:27:12.522131+0000","mapping_epoch":287,"log_start":"322'9515","ondisk_log_start":"322'9515","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:24:03.430037+0000","objects_scrubbed":0,"log_size":5046,"log_dups_size":0,"ondisk_log_size":5046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129957888,"num_objects":1983,"num_object_clones":0,"num_object_copies":5949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14561,"num_write_kb":529408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":207,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586616+0000","last_change":"2024-09-17T08:25:35.125962+0000","last_active":"2024-09-17T08:27:11.586616+0000","last_peered":"2024-09-17T08:27:11.586616+0000","last_clean":"2024-09-17T08:27:11.586616+0000","last_became_active":"2024-09-17T08:23:18.670977+0000","last_became_peered":"2024-09-17T08:23:18.670977+0000","last_unstale":"2024-09-17T08:27:11.586616+0000","last_undegraded":"2024-09-17T08:27:11.586616+0000","last_fullsized":"2024-09-17T08:27:11.586616+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:25:35.125100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:35.125100+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023912690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":343,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:06.746698+0000","last_change":"2024-09-17T08:26:29.545228+0000","last_active":"2024-09-17T08:27:06.746698+0000","last_peered":"2024-09-17T08:27:06.746698+0000","last_clean":"2024-09-17T08:27:06.746698+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:27:06.746698+0000","last_undegraded":"2024-09-17T08:27:06.746698+0000","last_fullsized":"2024-09-17T08:27:06.746698+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:29.545114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:29.545114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:32.014523+0000","scrub_duration":0.0047629450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":349,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:06.746656+0000","last_change":"2024-09-17T08:25:13.057707+0000","last_active":"2024-09-17T08:27:06.746656+0000","last_peered":"2024-09-17T08:27:06.746656+0000","last_clean":"2024-09-17T08:27:06.746656+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:27:06.746656+0000","last_undegraded":"2024-09-17T08:27:06.746656+0000","last_fullsized":"2024-09-17T08:27:06.746656+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:13.057534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:52.379921+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.057534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011530043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":378,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588933+0000","last_change":"2024-09-17T08:26:15.680685+0000","last_active":"2024-09-17T08:27:11.588933+0000","last_peered":"2024-09-17T08:27:11.588933+0000","last_clean":"2024-09-17T08:27:11.588933+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:27:11.588933+0000","last_undegraded":"2024-09-17T08:27:11.588933+0000","last_fullsized":"2024-09-17T08:27:11.588933+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:15.680565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:15.680565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:34.928434+0000","scrub_duration":0.0076375050000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"340'14534","reported_seq":14737,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.768166+0000","last_change":"2024-09-17T08:26:17.151968+0000","last_active":"2024-09-17T08:27:11.768166+0000","last_peered":"2024-09-17T08:27:11.768166+0000","last_clean":"2024-09-17T08:27:11.768166+0000","last_became_active":"2024-09-17T08:24:04.458851+0000","last_became_peered":"2024-09-17T08:24:04.458851+0000","last_unstale":"2024-09-17T08:27:11.768166+0000","last_undegraded":"2024-09-17T08:27:11.768166+0000","last_fullsized":"2024-09-17T08:27:11.768166+0000","mapping_epoch":287,"log_start":"320'8721","ondisk_log_start":"320'8721","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10426","last_scrub_stamp":"2024-09-17T08:26:17.151829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.151829+0000","objects_scrubbed":6271,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.276228+0000","scrub_duration":2.6491713260000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136314880,"num_objects":2080,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":531648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":430,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537540+0000","last_change":"2024-09-17T08:25:16.078092+0000","last_active":"2024-09-17T08:27:04.537540+0000","last_peered":"2024-09-17T08:27:04.537540+0000","last_clean":"2024-09-17T08:27:04.537540+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:27:04.537540+0000","last_undegraded":"2024-09-17T08:27:04.537540+0000","last_fullsized":"2024-09-17T08:27:04.537540+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:25:16.077960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:16.077960+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021317215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":260,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584941+0000","last_change":"2024-09-17T08:26:17.341373+0000","last_active":"2024-09-17T08:27:11.584941+0000","last_peered":"2024-09-17T08:27:11.584941+0000","last_clean":"2024-09-17T08:27:11.584941+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:27:11.584941+0000","last_undegraded":"2024-09-17T08:27:11.584941+0000","last_fullsized":"2024-09-17T08:27:11.584941+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.341260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.341260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.774271+0000","scrub_duration":0.0033668219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":363,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.529484+0000","last_change":"2024-09-17T08:26:31.574646+0000","last_active":"2024-09-17T08:27:04.529484+0000","last_peered":"2024-09-17T08:27:04.529484+0000","last_clean":"2024-09-17T08:27:04.529484+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:27:04.529484+0000","last_undegraded":"2024-09-17T08:27:04.529484+0000","last_fullsized":"2024-09-17T08:27:04.529484+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:31.574528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:26:31.574528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:40.768540+0000","scrub_duration":0.0037572360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":378,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.539858+0000","last_change":"2024-09-17T08:25:40.906564+0000","last_active":"2024-09-17T08:27:04.539858+0000","last_peered":"2024-09-17T08:27:04.539858+0000","last_clean":"2024-09-17T08:27:04.539858+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:27:04.539858+0000","last_undegraded":"2024-09-17T08:27:04.539858+0000","last_fullsized":"2024-09-17T08:27:04.539858+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:40.906437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:25:40.906437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014127516,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"340'14390","reported_seq":14513,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.527364+0000","last_change":"2024-09-17T08:27:08.132232+0000","last_active":"2024-09-17T08:27:12.527364+0000","last_peered":"2024-09-17T08:27:12.527364+0000","last_clean":"2024-09-17T08:27:12.527364+0000","last_became_active":"2024-09-17T08:24:04.458042+0000","last_became_peered":"2024-09-17T08:24:04.458042+0000","last_unstale":"2024-09-17T08:27:12.527364+0000","last_undegraded":"2024-09-17T08:27:12.527364+0000","last_fullsized":"2024-09-17T08:27:12.527364+0000","mapping_epoch":287,"log_start":"321'9333","ondisk_log_start":"321'9333","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'14029","last_scrub_stamp":"2024-09-17T08:27:08.132029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:27:08.132029+0000","objects_scrubbed":2356,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:21.060182+0000","scrub_duration":0.89171964199999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129105920,"num_objects":1970,"num_object_clones":0,"num_object_copies":5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14390,"num_write_kb":523520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":423,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588680+0000","last_change":"2024-09-17T08:25:13.823972+0000","last_active":"2024-09-17T08:27:11.588680+0000","last_peered":"2024-09-17T08:27:11.588680+0000","last_clean":"2024-09-17T08:27:11.588680+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:27:11.588680+0000","last_undegraded":"2024-09-17T08:27:11.588680+0000","last_fullsized":"2024-09-17T08:27:11.588680+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:25:13.823819+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:25:13.823819+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031292669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":364,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:06.751370+0000","last_change":"2024-09-17T08:25:27.161139+0000","last_active":"2024-09-17T08:27:06.751370+0000","last_peered":"2024-09-17T08:27:06.751370+0000","last_clean":"2024-09-17T08:27:06.751370+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:27:06.751370+0000","last_undegraded":"2024-09-17T08:27:06.751370+0000","last_fullsized":"2024-09-17T08:27:06.751370+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:27.160999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:27.160999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058695600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":592,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.677557+0000","last_change":"2024-09-17T08:27:12.677760+0000","last_active":"2024-09-17T08:27:12.677557+0000","last_peered":"2024-09-17T08:27:12.677557+0000","last_clean":"2024-09-17T08:27:12.677557+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:27:12.677557+0000","last_undegraded":"2024-09-17T08:27:12.677557+0000","last_fullsized":"2024-09-17T08:27:12.677557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:27:12.677535+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:22:33.388161+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.677535+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:23.736887+0000","scrub_duration":0.0060670139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":216,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537830+0000","last_change":"2024-09-17T08:26:32.132061+0000","last_active":"2024-09-17T08:27:04.537830+0000","last_peered":"2024-09-17T08:27:04.537830+0000","last_clean":"2024-09-17T08:27:04.537830+0000","last_became_active":"2024-09-17T08:23:18.662935+0000","last_became_peered":"2024-09-17T08:23:18.662935+0000","last_unstale":"2024-09-17T08:27:04.537830+0000","last_undegraded":"2024-09-17T08:27:04.537830+0000","last_fullsized":"2024-09-17T08:27:04.537830+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:32.131951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:26:32.131951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:45.669382+0000","scrub_duration":0.0022176209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"341'14842","reported_seq":14959,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.757471+0000","last_change":"2024-09-17T08:27:11.326147+0000","last_active":"2024-09-17T08:27:13.757471+0000","last_peered":"2024-09-17T08:27:13.757471+0000","last_clean":"2024-09-17T08:27:13.757471+0000","last_became_active":"2024-09-17T08:24:04.453268+0000","last_became_peered":"2024-09-17T08:24:04.453268+0000","last_unstale":"2024-09-17T08:27:13.757471+0000","last_undegraded":"2024-09-17T08:27:13.757471+0000","last_fullsized":"2024-09-17T08:27:13.757471+0000","mapping_epoch":287,"log_start":"324'9993","ondisk_log_start":"324'9993","created":287,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'14658","last_scrub_stamp":"2024-09-17T08:27:11.325908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:03.430037+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.325908+0000","objects_scrubbed":2082,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:28.835096+0000","scrub_duration":0.65869144999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123076608,"num_objects":1878,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14842,"num_write_kb":535040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":442,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583181+0000","last_change":"2024-09-17T08:25:29.258817+0000","last_active":"2024-09-17T08:27:11.583181+0000","last_peered":"2024-09-17T08:27:11.583181+0000","last_clean":"2024-09-17T08:27:11.583181+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:27:11.583181+0000","last_undegraded":"2024-09-17T08:27:11.583181+0000","last_fullsized":"2024-09-17T08:27:11.583181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:25:29.258616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:29.258616+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011223878,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":377,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583152+0000","last_change":"2024-09-17T08:24:32.450419+0000","last_active":"2024-09-17T08:27:11.583152+0000","last_peered":"2024-09-17T08:27:11.583152+0000","last_clean":"2024-09-17T08:27:11.583152+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:27:11.583152+0000","last_undegraded":"2024-09-17T08:27:11.583152+0000","last_fullsized":"2024-09-17T08:27:11.583152+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:24:32.450300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:32.450300+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058717939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":368,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.530522+0000","last_change":"2024-09-17T08:25:25.116593+0000","last_active":"2024-09-17T08:27:04.530522+0000","last_peered":"2024-09-17T08:27:04.530522+0000","last_clean":"2024-09-17T08:27:04.530522+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:27:04.530522+0000","last_undegraded":"2024-09-17T08:27:04.530522+0000","last_fullsized":"2024-09-17T08:27:04.530522+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:25.116441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:25:25.116441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008148832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":183,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:06.746637+0000","last_change":"2024-09-17T08:26:30.599024+0000","last_active":"2024-09-17T08:27:06.746637+0000","last_peered":"2024-09-17T08:27:06.746637+0000","last_clean":"2024-09-17T08:27:06.746637+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:27:06.746637+0000","last_undegraded":"2024-09-17T08:27:06.746637+0000","last_fullsized":"2024-09-17T08:27:06.746637+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.598835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.598835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.854877+0000","scrub_duration":0.0038840160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":396,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537897+0000","last_change":"2024-09-17T08:25:33.033204+0000","last_active":"2024-09-17T08:27:04.537897+0000","last_peered":"2024-09-17T08:27:04.537897+0000","last_clean":"2024-09-17T08:27:04.537897+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:27:04.537897+0000","last_undegraded":"2024-09-17T08:27:04.537897+0000","last_fullsized":"2024-09-17T08:27:04.537897+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.032722+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085113229999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":392,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:10.815183+0000","last_change":"2024-09-17T08:27:10.815183+0000","last_active":"2024-09-17T08:27:10.815183+0000","last_peered":"2024-09-17T08:27:10.815183+0000","last_clean":"2024-09-17T08:27:10.815183+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:27:10.815183+0000","last_undegraded":"2024-09-17T08:27:10.815183+0000","last_fullsized":"2024-09-17T08:27:10.815183+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:10.811793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:10.811793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:12.682296+0000","scrub_duration":0.014640149,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":373,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584186+0000","last_change":"2024-09-17T08:25:32.809559+0000","last_active":"2024-09-17T08:27:11.584186+0000","last_peered":"2024-09-17T08:27:11.584186+0000","last_clean":"2024-09-17T08:27:11.584186+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:27:11.584186+0000","last_undegraded":"2024-09-17T08:27:11.584186+0000","last_fullsized":"2024-09-17T08:27:11.584186+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:32.809305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:25:32.809305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024200179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":354,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.529568+0000","last_change":"2024-09-17T08:26:11.533286+0000","last_active":"2024-09-17T08:27:04.529568+0000","last_peered":"2024-09-17T08:27:04.529568+0000","last_clean":"2024-09-17T08:27:04.529568+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:27:04.529568+0000","last_undegraded":"2024-09-17T08:27:04.529568+0000","last_fullsized":"2024-09-17T08:27:04.529568+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.533066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.533066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:20.390854+0000","scrub_duration":0.0073762410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":266,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583314+0000","last_change":"2024-09-17T08:25:18.296524+0000","last_active":"2024-09-17T08:27:11.583314+0000","last_peered":"2024-09-17T08:27:11.583314+0000","last_clean":"2024-09-17T08:27:11.583314+0000","last_became_active":"2024-09-17T08:23:18.666495+0000","last_became_peered":"2024-09-17T08:23:18.666495+0000","last_unstale":"2024-09-17T08:27:11.583314+0000","last_undegraded":"2024-09-17T08:27:11.583314+0000","last_fullsized":"2024-09-17T08:27:11.583314+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:18.296432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:18.296432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041815000000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":420,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584382+0000","last_change":"2024-09-17T08:26:17.976881+0000","last_active":"2024-09-17T08:27:11.584382+0000","last_peered":"2024-09-17T08:27:11.584382+0000","last_clean":"2024-09-17T08:27:11.584382+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:27:11.584382+0000","last_undegraded":"2024-09-17T08:27:11.584382+0000","last_fullsized":"2024-09-17T08:27:11.584382+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:26:17.976320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.976320+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.774813+0000","scrub_duration":0.0092893760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":276,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583738+0000","last_change":"2024-09-17T08:24:19.785545+0000","last_active":"2024-09-17T08:27:11.583738+0000","last_peered":"2024-09-17T08:27:11.583738+0000","last_clean":"2024-09-17T08:27:11.583738+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:27:11.583738+0000","last_undegraded":"2024-09-17T08:27:11.583738+0000","last_fullsized":"2024-09-17T08:27:11.583738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:19.785421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.621350+0000","last_clean_scrub_stamp":"2024-09-17T08:24:19.785421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005223391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":385,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589146+0000","last_change":"2024-09-17T08:26:35.867945+0000","last_active":"2024-09-17T08:27:11.589146+0000","last_peered":"2024-09-17T08:27:11.589146+0000","last_clean":"2024-09-17T08:27:11.589146+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:27:11.589146+0000","last_undegraded":"2024-09-17T08:27:11.589146+0000","last_fullsized":"2024-09-17T08:27:11.589146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:35.867811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:35.867811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:03.387177+0000","scrub_duration":0.011541553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":429,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583697+0000","last_change":"2024-09-17T08:24:09.894035+0000","last_active":"2024-09-17T08:27:11.583697+0000","last_peered":"2024-09-17T08:27:11.583697+0000","last_clean":"2024-09-17T08:27:11.583697+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:27:11.583697+0000","last_undegraded":"2024-09-17T08:27:11.583697+0000","last_fullsized":"2024-09-17T08:27:11.583697+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:24:09.893951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:09.893951+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019165969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":316,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586311+0000","last_change":"2024-09-17T08:26:16.948193+0000","last_active":"2024-09-17T08:27:11.586311+0000","last_peered":"2024-09-17T08:27:11.586311+0000","last_clean":"2024-09-17T08:27:11.586311+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:27:11.586311+0000","last_undegraded":"2024-09-17T08:27:11.586311+0000","last_fullsized":"2024-09-17T08:27:11.586311+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.948072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.948072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:26.892623+0000","scrub_duration":0.0075166590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":406,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:13.210095+0000","last_change":"2024-09-17T08:27:13.210198+0000","last_active":"2024-09-17T08:27:13.210095+0000","last_peered":"2024-09-17T08:27:13.210095+0000","last_clean":"2024-09-17T08:27:13.210095+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:27:13.210095+0000","last_undegraded":"2024-09-17T08:27:13.210095+0000","last_fullsized":"2024-09-17T08:27:13.210095+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:27:13.210073+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:27:13.210073+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:26.343796+0000","scrub_duration":0.0025890539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":363,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537598+0000","last_change":"2024-09-17T08:24:18.708208+0000","last_active":"2024-09-17T08:27:04.537598+0000","last_peered":"2024-09-17T08:27:04.537598+0000","last_clean":"2024-09-17T08:27:04.537598+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:27:04.537598+0000","last_undegraded":"2024-09-17T08:27:04.537598+0000","last_fullsized":"2024-09-17T08:27:04.537598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:18.708083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:18.708083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010815099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":321,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589428+0000","last_change":"2024-09-17T08:26:13.666735+0000","last_active":"2024-09-17T08:27:11.589428+0000","last_peered":"2024-09-17T08:27:11.589428+0000","last_clean":"2024-09-17T08:27:11.589428+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:27:11.589428+0000","last_undegraded":"2024-09-17T08:27:11.589428+0000","last_fullsized":"2024-09-17T08:27:11.589428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:13.666621+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.666621+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:16.247569+0000","scrub_duration":0.0059880139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":369,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537907+0000","last_change":"2024-09-17T08:25:15.032652+0000","last_active":"2024-09-17T08:27:04.537907+0000","last_peered":"2024-09-17T08:27:04.537907+0000","last_clean":"2024-09-17T08:27:04.537907+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:27:04.537907+0000","last_undegraded":"2024-09-17T08:27:04.537907+0000","last_fullsized":"2024-09-17T08:27:04.537907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:15.032525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:25:15.032525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013962988000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":589,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.599800+0000","last_change":"2024-09-17T08:25:23.783490+0000","last_active":"2024-09-17T08:27:12.599800+0000","last_peered":"2024-09-17T08:27:12.599800+0000","last_clean":"2024-09-17T08:27:12.599800+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:27:12.599800+0000","last_undegraded":"2024-09-17T08:27:12.599800+0000","last_fullsized":"2024-09-17T08:27:12.599800+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:25:23.783353+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:25:23.783353+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064248070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":369,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582898+0000","last_change":"2024-09-17T08:26:31.188294+0000","last_active":"2024-09-17T08:27:11.582898+0000","last_peered":"2024-09-17T08:27:11.582898+0000","last_clean":"2024-09-17T08:27:11.582898+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:27:11.582898+0000","last_undegraded":"2024-09-17T08:27:11.582898+0000","last_fullsized":"2024-09-17T08:27:11.582898+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:31.188153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:31.188153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:57.198487+0000","scrub_duration":0.0090076900000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":292,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.596181+0000","last_change":"2024-09-17T08:27:07.276176+0000","last_active":"2024-09-17T08:27:12.596181+0000","last_peered":"2024-09-17T08:27:12.596181+0000","last_clean":"2024-09-17T08:27:12.596181+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:27:12.596181+0000","last_undegraded":"2024-09-17T08:27:12.596181+0000","last_fullsized":"2024-09-17T08:27:12.596181+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:27:07.275632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:07.275632+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:34.204738+0000","scrub_duration":0.0073795509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":247,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583764+0000","last_change":"2024-09-17T08:24:13.873686+0000","last_active":"2024-09-17T08:27:11.583764+0000","last_peered":"2024-09-17T08:27:11.583764+0000","last_clean":"2024-09-17T08:27:11.583764+0000","last_became_active":"2024-09-17T08:23:18.671502+0000","last_became_peered":"2024-09-17T08:23:18.671502+0000","last_unstale":"2024-09-17T08:27:11.583764+0000","last_undegraded":"2024-09-17T08:27:11.583764+0000","last_fullsized":"2024-09-17T08:27:11.583764+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:13.873601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:24:13.873601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016156409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":373,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586550+0000","last_change":"2024-09-17T08:24:10.894915+0000","last_active":"2024-09-17T08:27:11.586550+0000","last_peered":"2024-09-17T08:27:11.586550+0000","last_clean":"2024-09-17T08:27:11.586550+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:27:11.586550+0000","last_undegraded":"2024-09-17T08:27:11.586550+0000","last_fullsized":"2024-09-17T08:27:11.586550+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:10.894828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:24:10.894828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001103185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":381,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.538540+0000","last_change":"2024-09-17T08:24:08.507584+0000","last_active":"2024-09-17T08:27:04.538540+0000","last_peered":"2024-09-17T08:27:04.538540+0000","last_clean":"2024-09-17T08:27:04.538540+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:27:04.538540+0000","last_undegraded":"2024-09-17T08:27:04.538540+0000","last_fullsized":"2024-09-17T08:27:04.538540+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:08.507505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:08.507505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014868570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":442,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582088+0000","last_change":"2024-09-17T08:25:33.258559+0000","last_active":"2024-09-17T08:27:11.582088+0000","last_peered":"2024-09-17T08:27:11.582088+0000","last_clean":"2024-09-17T08:27:11.582088+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:27:11.582088+0000","last_undegraded":"2024-09-17T08:27:11.582088+0000","last_fullsized":"2024-09-17T08:27:11.582088+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:25:33.258469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:25:33.258469+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016288769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585281+0000","last_change":"2024-09-17T08:25:30.191078+0000","last_active":"2024-09-17T08:27:11.585281+0000","last_peered":"2024-09-17T08:27:11.585281+0000","last_clean":"2024-09-17T08:27:11.585281+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:27:11.585281+0000","last_undegraded":"2024-09-17T08:27:11.585281+0000","last_fullsized":"2024-09-17T08:27:11.585281+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:30.190970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:30.190970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016272669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":249,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-17T08:27:12.596896+0000","last_change":"2024-09-17T08:26:26.350432+0000","last_active":"2024-09-17T08:27:12.596896+0000","last_peered":"2024-09-17T08:27:12.596896+0000","last_clean":"2024-09-17T08:27:12.596896+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:27:12.596896+0000","last_undegraded":"2024-09-17T08:27:12.596896+0000","last_fullsized":"2024-09-17T08:27:12.596896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:26.349968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:26.349968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:29.549108+0000","scrub_duration":0.004346824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":364,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585456+0000","last_change":"2024-09-17T08:27:09.714075+0000","last_active":"2024-09-17T08:27:11.585456+0000","last_peered":"2024-09-17T08:27:11.585456+0000","last_clean":"2024-09-17T08:27:11.585456+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:27:11.585456+0000","last_undegraded":"2024-09-17T08:27:11.585456+0000","last_fullsized":"2024-09-17T08:27:11.585456+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:09.713944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:09.713944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:23.064993+0000","scrub_duration":0.0064145390000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":82,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.581665+0000","last_change":"2024-09-17T08:24:26.616991+0000","last_active":"2024-09-17T08:27:11.581665+0000","last_peered":"2024-09-17T08:27:11.581665+0000","last_clean":"2024-09-17T08:27:11.581665+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:27:11.581665+0000","last_undegraded":"2024-09-17T08:27:11.581665+0000","last_fullsized":"2024-09-17T08:27:11.581665+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:24:26.616854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:26.616854+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002706263,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":252,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:05.718238+0000","last_change":"2024-09-17T08:27:05.718238+0000","last_active":"2024-09-17T08:27:05.718238+0000","last_peered":"2024-09-17T08:27:05.718238+0000","last_clean":"2024-09-17T08:27:05.718238+0000","last_became_active":"2024-09-17T08:20:23.483207+0000","last_became_peered":"2024-09-17T08:20:23.483207+0000","last_unstale":"2024-09-17T08:27:05.718238+0000","last_undegraded":"2024-09-17T08:27:05.718238+0000","last_fullsized":"2024-09-17T08:27:05.718238+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:05.718134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:05.718134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:24.460793+0000","scrub_duration":0.0024338390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583047+0000","last_change":"2024-09-17T08:25:41.337008+0000","last_active":"2024-09-17T08:27:11.583047+0000","last_peered":"2024-09-17T08:27:11.583047+0000","last_clean":"2024-09-17T08:27:11.583047+0000","last_became_active":"2024-09-17T08:23:18.671271+0000","last_became_peered":"2024-09-17T08:23:18.671271+0000","last_unstale":"2024-09-17T08:27:11.583047+0000","last_undegraded":"2024-09-17T08:27:11.583047+0000","last_fullsized":"2024-09-17T08:27:11.583047+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:41.336771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:25:41.336771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004352692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":389,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588117+0000","last_change":"2024-09-17T08:27:09.746008+0000","last_active":"2024-09-17T08:27:11.588117+0000","last_peered":"2024-09-17T08:27:11.588117+0000","last_clean":"2024-09-17T08:27:11.588117+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:27:11.588117+0000","last_undegraded":"2024-09-17T08:27:11.588117+0000","last_fullsized":"2024-09-17T08:27:11.588117+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:09.745900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:27:09.745900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:21.243107+0000","scrub_duration":0.0048935610000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":388,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585921+0000","last_change":"2024-09-17T08:24:21.848684+0000","last_active":"2024-09-17T08:27:11.585921+0000","last_peered":"2024-09-17T08:27:11.585921+0000","last_clean":"2024-09-17T08:27:11.585921+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:27:11.585921+0000","last_undegraded":"2024-09-17T08:27:11.585921+0000","last_fullsized":"2024-09-17T08:27:11.585921+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:24:21.848510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:24:21.848510+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030240649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":224,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.529003+0000","last_change":"2024-09-17T08:26:22.498965+0000","last_active":"2024-09-17T08:27:04.529003+0000","last_peered":"2024-09-17T08:27:04.529003+0000","last_clean":"2024-09-17T08:27:04.529003+0000","last_became_active":"2024-09-17T08:23:18.669127+0000","last_became_peered":"2024-09-17T08:23:18.669127+0000","last_unstale":"2024-09-17T08:27:04.529003+0000","last_undegraded":"2024-09-17T08:27:04.529003+0000","last_fullsized":"2024-09-17T08:27:04.529003+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:26:22.498827+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.498827+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:38.115530+0000","scrub_duration":0.011342636999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":376,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.588600+0000","last_change":"2024-09-17T08:26:17.717152+0000","last_active":"2024-09-17T08:27:11.588600+0000","last_peered":"2024-09-17T08:27:11.588600+0000","last_clean":"2024-09-17T08:27:11.588600+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:27:11.588600+0000","last_undegraded":"2024-09-17T08:27:11.588600+0000","last_fullsized":"2024-09-17T08:27:11.588600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:17.717066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:17.717066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:31.062923+0000","scrub_duration":0.0015956360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":353,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585302+0000","last_change":"2024-09-17T08:24:29.919412+0000","last_active":"2024-09-17T08:27:11.585302+0000","last_peered":"2024-09-17T08:27:11.585302+0000","last_clean":"2024-09-17T08:27:11.585302+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:27:11.585302+0000","last_undegraded":"2024-09-17T08:27:11.585302+0000","last_fullsized":"2024-09-17T08:27:11.585302+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:24:29.919294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:24:29.919294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048696069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":390,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.589269+0000","last_change":"2024-09-17T08:27:10.716378+0000","last_active":"2024-09-17T08:27:11.589269+0000","last_peered":"2024-09-17T08:27:11.589269+0000","last_clean":"2024-09-17T08:27:11.589269+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:27:11.589269+0000","last_undegraded":"2024-09-17T08:27:11.589269+0000","last_fullsized":"2024-09-17T08:27:11.589269+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:27:10.716109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:10.716109+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:24.456723+0000","scrub_duration":0.010987006000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":270,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.582465+0000","last_change":"2024-09-17T08:26:11.160161+0000","last_active":"2024-09-17T08:27:11.582465+0000","last_peered":"2024-09-17T08:27:11.582465+0000","last_clean":"2024-09-17T08:27:11.582465+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:27:11.582465+0000","last_undegraded":"2024-09-17T08:27:11.582465+0000","last_fullsized":"2024-09-17T08:27:11.582465+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:11.159902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:26:11.159902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:24.996888+0000","scrub_duration":0.011043470999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":409,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.144999+0000","last_change":"2024-09-17T08:27:11.144999+0000","last_active":"2024-09-17T08:27:11.144999+0000","last_peered":"2024-09-17T08:27:11.144999+0000","last_clean":"2024-09-17T08:27:11.144999+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:27:11.144999+0000","last_undegraded":"2024-09-17T08:27:11.144999+0000","last_fullsized":"2024-09-17T08:27:11.144999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:27:11.144705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.144705+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:28.460935+0000","scrub_duration":0.013738378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"104'9","reported_seq":506,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583075+0000","last_change":"2024-09-17T08:26:21.131000+0000","last_active":"2024-09-17T08:27:11.583075+0000","last_peered":"2024-09-17T08:27:11.583075+0000","last_clean":"2024-09-17T08:27:11.583075+0000","last_became_active":"2024-09-17T08:12:46.187927+0000","last_became_peered":"2024-09-17T08:12:46.187927+0000","last_unstale":"2024-09-17T08:27:11.583075+0000","last_undegraded":"2024-09-17T08:27:11.583075+0000","last_fullsized":"2024-09-17T08:27:11.583075+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'9","last_scrub_stamp":"2024-09-17T08:26:21.130873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.130873+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:23.349562+0000","scrub_duration":0.012556618,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":259,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585009+0000","last_change":"2024-09-17T08:26:21.302147+0000","last_active":"2024-09-17T08:27:11.585009+0000","last_peered":"2024-09-17T08:27:11.585009+0000","last_clean":"2024-09-17T08:27:11.585009+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:27:11.585009+0000","last_undegraded":"2024-09-17T08:27:11.585009+0000","last_fullsized":"2024-09-17T08:27:11.585009+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:21.302035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.302035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:50.505197+0000","scrub_duration":0.014448862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":370,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585985+0000","last_change":"2024-09-17T08:27:04.746649+0000","last_active":"2024-09-17T08:27:11.585985+0000","last_peered":"2024-09-17T08:27:11.585985+0000","last_clean":"2024-09-17T08:27:11.585985+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:27:11.585985+0000","last_undegraded":"2024-09-17T08:27:11.585985+0000","last_fullsized":"2024-09-17T08:27:11.585985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:04.746540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:27:04.746540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:10.885787+0000","scrub_duration":0.0024775520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":183,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.586025+0000","last_change":"2024-09-17T08:26:14.336368+0000","last_active":"2024-09-17T08:27:11.586025+0000","last_peered":"2024-09-17T08:27:11.586025+0000","last_clean":"2024-09-17T08:27:11.586025+0000","last_became_active":"2024-09-17T08:23:18.676257+0000","last_became_peered":"2024-09-17T08:23:18.676257+0000","last_unstale":"2024-09-17T08:27:11.586025+0000","last_undegraded":"2024-09-17T08:27:11.586025+0000","last_fullsized":"2024-09-17T08:27:11.586025+0000","mapping_epoch":278,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":279,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:26:14.336205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:14.336205+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:21.735226+0000","scrub_duration":0.0072317960000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":394,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.537726+0000","last_change":"2024-09-17T08:26:16.157165+0000","last_active":"2024-09-17T08:27:04.537726+0000","last_peered":"2024-09-17T08:27:04.537726+0000","last_clean":"2024-09-17T08:27:04.537726+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:27:04.537726+0000","last_undegraded":"2024-09-17T08:27:04.537726+0000","last_fullsized":"2024-09-17T08:27:04.537726+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.157047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.157047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:18.892586+0000","scrub_duration":0.052471214000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":354,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583694+0000","last_change":"2024-09-17T08:25:19.293077+0000","last_active":"2024-09-17T08:27:11.583694+0000","last_peered":"2024-09-17T08:27:11.583694+0000","last_clean":"2024-09-17T08:27:11.583694+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:27:11.583694+0000","last_undegraded":"2024-09-17T08:27:11.583694+0000","last_fullsized":"2024-09-17T08:27:11.583694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:19.292858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:25:19.292858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013986980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":321,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.583296+0000","last_change":"2024-09-17T08:26:20.113140+0000","last_active":"2024-09-17T08:27:11.583296+0000","last_peered":"2024-09-17T08:27:11.583296+0000","last_clean":"2024-09-17T08:27:11.583296+0000","last_became_active":"2024-09-17T08:12:46.185984+0000","last_became_peered":"2024-09-17T08:12:46.185984+0000","last_unstale":"2024-09-17T08:27:11.583296+0000","last_undegraded":"2024-09-17T08:27:11.583296+0000","last_fullsized":"2024-09-17T08:27:11.583296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:20.113014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:26:20.113014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:29.947022+0000","scrub_duration":0.0052786329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":485,"reported_epoch":339,"state":"active+clean","last_fresh":"2024-09-17T08:27:04.760356+0000","last_change":"2024-09-17T08:27:04.760356+0000","last_active":"2024-09-17T08:27:04.760356+0000","last_peered":"2024-09-17T08:27:04.760356+0000","last_clean":"2024-09-17T08:27:04.760356+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:27:04.760356+0000","last_undegraded":"2024-09-17T08:27:04.760356+0000","last_fullsized":"2024-09-17T08:27:04.760356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:27:04.760216+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:27:04.760216+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:05.239438+0000","scrub_duration":0.011259389,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.585136+0000","last_change":"2024-09-17T08:25:34.089169+0000","last_active":"2024-09-17T08:27:11.585136+0000","last_peered":"2024-09-17T08:27:11.585136+0000","last_clean":"2024-09-17T08:27:11.585136+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:27:11.585136+0000","last_undegraded":"2024-09-17T08:27:11.585136+0000","last_fullsized":"2024-09-17T08:27:11.585136+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:25:34.089010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037643389999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":354,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584049+0000","last_change":"2024-09-17T08:26:13.980481+0000","last_active":"2024-09-17T08:27:11.584049+0000","last_peered":"2024-09-17T08:27:11.584049+0000","last_clean":"2024-09-17T08:27:11.584049+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:27:11.584049+0000","last_undegraded":"2024-09-17T08:27:11.584049+0000","last_fullsized":"2024-09-17T08:27:11.584049+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:13.980289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:13.980289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:15.581630+0000","scrub_duration":0.014435847,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":369,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.587335+0000","last_change":"2024-09-17T08:25:38.883969+0000","last_active":"2024-09-17T08:27:11.587335+0000","last_peered":"2024-09-17T08:27:11.587335+0000","last_clean":"2024-09-17T08:27:11.587335+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:27:11.587335+0000","last_undegraded":"2024-09-17T08:27:11.587335+0000","last_fullsized":"2024-09-17T08:27:11.587335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:25:38.883814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:25:38.883814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012313661,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":441,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.584421+0000","last_change":"2024-09-17T08:27:06.697550+0000","last_active":"2024-09-17T08:27:11.584421+0000","last_peered":"2024-09-17T08:27:11.584421+0000","last_clean":"2024-09-17T08:27:11.584421+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:27:11.584421+0000","last_undegraded":"2024-09-17T08:27:11.584421+0000","last_fullsized":"2024-09-17T08:27:11.584421+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:27:06.697462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:06.697462+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:08.351982+0000","scrub_duration":0.0054958810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":381,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-17T08:27:11.587928+0000","last_change":"2024-09-17T08:26:18.169361+0000","last_active":"2024-09-17T08:27:11.587928+0000","last_peered":"2024-09-17T08:27:11.587928+0000","last_clean":"2024-09-17T08:27:11.587928+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:27:11.587928+0000","last_undegraded":"2024-09-17T08:27:11.587928+0000","last_fullsized":"2024-09-17T08:27:11.587928+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:18.169258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:26:18.169258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:39.643737+0000","scrub_duration":0.042744938000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2066153496,"num_objects":31528,"num_object_clones":0,"num_object_copies":94584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233532,"num_write_kb":8481857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6223179776,"data_stored":6223167560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82403,"ondisk_log_size":82403,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1119,"num_read_kb":876,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":701,"num_read_kb":696,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":298,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2098784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1430,"ondisk_log_size":1430,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070677,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1110656,"kb_used_data":520052,"kb_used_omap":1,"kb_used_meta":590590,"kb_avail":92634496,"statfs":{"total":95995035648,"available":94857723904,"internally_reserved":0,"allocated":532533248,"data_stored":532177318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":604764619},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.4630000000000001,"15min":1.516},"min":{"1min":0.24399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":3.4470000000000001,"5min":13.446,"15min":62.018000000000001},"last":2.4399999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.456,"15min":1.5860000000000001},"min":{"1min":0.22900000000000001,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":3.1680000000000001,"5min":13.417,"15min":61.896999999999998},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.0820000000000001,"15min":1.5129999999999999},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.23100000000000001},"max":{"1min":2.7400000000000002,"5min":14.731,"15min":62.067},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.876,"5min":1.3779999999999999,"15min":1.504},"min":{"1min":0.30599999999999999,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":2.7869999999999999,"5min":14.401999999999999,"15min":62.194000000000003},"last":0.375}]},{"osd":2,"last update":"Tue Sep 17 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.081,"15min":0.92900000000000005},"min":{"1min":0.26600000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.3109999999999999,"5min":9.3109999999999999,"15min":9.3109999999999999},"last":2.4580000000000002},{"interface":"front","average":{"1min":1.268,"5min":1.0660000000000001,"15min":0.72699999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.72299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":1.732,"5min":2.343,"15min":1.7090000000000001},"min":{"1min":0.36899999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":4.9850000000000003,"5min":74.376000000000005,"15min":74.376000000000005},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.194,"5min":2.177,"15min":1.774},"min":{"1min":0.307,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.7850000000000001,"5min":75.183000000000007,"15min":75.183000000000007},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.92900000000000005,"15min":0.73699999999999999},"min":{"1min":0.27400000000000002,"5min":0.224,"15min":0.224},"max":{"1min":2.3500000000000001,"5min":4.391,"15min":5.335},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.78500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23899999999999999},"max":{"1min":2.798,"5min":3.798,"15min":5.9669999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":2.3220000000000001,"5min":1.101,"15min":0.89100000000000001},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":34.436,"5min":34.436,"15min":34.436},"last":0.32600000000000001},{"interface":"front","average":{"1min":2.423,"5min":1.151,"15min":0.88200000000000001},"min":{"1min":0.317,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":34.965000000000003,"5min":34.965000000000003,"15min":34.965000000000003},"last":0.34699999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89000000000000001,"15min":0.81200000000000006},"min":{"1min":0.33500000000000002,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":2.4199999999999999,"5min":4.4640000000000004,"15min":7.2359999999999998},"last":0.315},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.80800000000000005,"15min":0.68700000000000006},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":2.7669999999999999,"5min":3.3849999999999998,"15min":5.9400000000000004},"last":0.69799999999999995}]}]},{"osd":6,"up_from":50,"seq":214748365018,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1395496,"kb_used_data":909084,"kb_used_omap":1,"kb_used_meta":486398,"kb_avail":92349656,"statfs":{"total":95995035648,"available":94566047744,"internally_reserved":0,"allocated":930902016,"data_stored":930561692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":498072007},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3220000000000001,"5min":1.996,"15min":2.4849999999999999},"min":{"1min":0.30599999999999999,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":9.4789999999999992,"5min":55.625,"15min":81.007000000000005},"last":6.4039999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.9590000000000001,"15min":2.5299999999999998},"min":{"1min":0.245,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":10.057,"5min":55.597000000000001,"15min":80.310000000000002},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Sep 17 08:26:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.006,"15min":1.657},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.22500000000000001},"max":{"1min":2.7320000000000002,"5min":9.8789999999999996,"15min":80.813000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0289999999999999,"15min":1.712},"min":{"1min":0.314,"5min":0.26000000000000001,"15min":0.24299999999999999},"max":{"1min":4.8890000000000002,"5min":10.01,"15min":80.281000000000006},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:26:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":2.0259999999999998,"15min":1.0900000000000001},"min":{"1min":0.28799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.5410000000000004,"5min":70.451999999999998,"15min":70.451999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.4139999999999999,"5min":2.0419999999999998,"15min":1.034},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.9950000000000001,"5min":70.488,"15min":70.488},"last":0.98099999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:26:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":1.018,"15min":1.9510000000000001},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8260000000000001,"5min":9.9309999999999992,"15min":80.427000000000007},"last":1.3029999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0660000000000001,"15min":1.8979999999999999},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26000000000000001},"max":{"1min":2.3220000000000001,"5min":11.859,"15min":80.379999999999995},"last":0.30499999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:26:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":0.96399999999999997,"15min":0.74199999999999999},"min":{"1min":0.26200000000000001,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":25.289999999999999,"5min":25.289999999999999,"15min":25.289999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.996,"5min":0.95999999999999996,"15min":0.68700000000000006},"min":{"1min":0.28699999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":25.184999999999999,"5min":25.184999999999999,"15min":25.184999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:26:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82599999999999996,"15min":0.72099999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.245},"max":{"1min":4.8570000000000002,"5min":5.5659999999999998,"15min":6.1859999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65200000000000002,"15min":0.61699999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26200000000000001},"max":{"1min":2.4129999999999998,"5min":3.1869999999999998,"15min":4.9969999999999999},"last":0.39800000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:26:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.70799999999999996},"min":{"1min":0.29199999999999998,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":2.948,"5min":2.948,"15min":4.774},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.749,"15min":0.64800000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":3.516,"5min":3.5590000000000002,"15min":5.0599999999999996},"last":0.45900000000000002}]}]},{"osd":5,"up_from":41,"seq":176093659359,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1241384,"kb_used_data":642592,"kb_used_omap":1,"kb_used_meta":598782,"kb_avail":92503768,"statfs":{"total":95995035648,"available":94723858432,"internally_reserved":0,"allocated":658014208,"data_stored":657670257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":613153222},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":3.5129999999999999,"5min":3.4409999999999998,"15min":2.3010000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.20300000000000001},"max":{"1min":26.158000000000001,"5min":65.941000000000003,"15min":65.941000000000003},"last":0.23899999999999999},{"interface":"front","average":{"1min":3.0539999999999998,"5min":3.46,"15min":2.2709999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":26.984000000000002,"5min":66.135000000000005,"15min":66.135000000000005},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":2.109,"15min":1.417},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":4.0469999999999997,"5min":66.981999999999999,"15min":66.981999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":1.0249999999999999,"5min":2.181,"15min":1.474},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":4.0599999999999996,"5min":66.316000000000003,"15min":66.316000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:26:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7280000000000002,"5min":3.4329999999999998,"15min":1.631},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":26.582999999999998,"5min":131.727,"15min":131.727},"last":0.66000000000000003},{"interface":"front","average":{"1min":2.359,"5min":3.4359999999999999,"15min":1.883},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":26.745999999999999,"5min":131.50399999999999,"15min":131.50399999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":2.3279999999999998,"15min":1.8},"min":{"1min":0.32600000000000001,"5min":0.26200000000000001,"15min":0.253},"max":{"1min":6.6879999999999997,"5min":67.676000000000002,"15min":75.286000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.175,"5min":2.1909999999999998,"15min":1.722},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27100000000000002},"max":{"1min":3.4100000000000001,"5min":67.691999999999993,"15min":77.206999999999994},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.83599999999999997,"15min":0.81599999999999995},"min":{"1min":0.28899999999999998,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":6.3600000000000003,"5min":6.3600000000000003,"15min":42.421999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.52,"5min":0.83599999999999997,"15min":0.81299999999999994},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":6.0899999999999999,"5min":6.0899999999999999,"15min":42.191000000000003},"last":0.84199999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:26:55 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":0.81699999999999995,"15min":0.68999999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.8399999999999999,"5min":7.8399999999999999,"15min":7.8399999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.204,"5min":0.85199999999999998,"15min":0.72199999999999998},"min":{"1min":0.30299999999999999,"5min":0.29499999999999998,"15min":0.26700000000000002},"max":{"1min":5.883,"5min":5.883,"15min":5.883},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.80100000000000005,"15min":0.755},"min":{"1min":0.33900000000000002,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":3.2989999999999999,"5min":4.9409999999999998,"15min":7.5270000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.26900000000000002},"max":{"1min":5.4980000000000002,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":0.34899999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953700,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1530616,"kb_used_data":1034348,"kb_used_omap":1,"kb_used_meta":496254,"kb_avail":92214536,"statfs":{"total":95995035648,"available":94427684864,"internally_reserved":0,"allocated":1059172352,"data_stored":1058816222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":508164550},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.968,"5min":1.1399999999999999,"15min":1.1060000000000001},"min":{"1min":0.222,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":20.271000000000001,"5min":20.271000000000001,"15min":23.786000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":2.1269999999999998,"5min":1.206,"15min":1.163},"min":{"1min":0.34200000000000003,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":19.798999999999999,"5min":19.798999999999999,"15min":23.245000000000001},"last":0.26700000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:27:00 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.1819999999999999,"15min":1.1870000000000001},"min":{"1min":0.28699999999999998,"5min":0.26000000000000001,"15min":0.23200000000000001},"max":{"1min":3.4460000000000002,"5min":9.3789999999999996,"15min":23.135000000000002},"last":2.7629999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.181,"15min":1.21},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.24099999999999999},"max":{"1min":8.1959999999999997,"5min":9.0719999999999992,"15min":23.434999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.92200000000000004,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.25900000000000001,"15min":0.25},"max":{"1min":3.9910000000000001,"5min":3.9910000000000001,"15min":3.9910000000000001},"last":0.432},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.751},"min":{"1min":0.33200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.0920000000000001,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Tue Sep 17 08:27:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.2669999999999999,"15min":1.9790000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":5.3120000000000003,"5min":28.138000000000002,"15min":154.137},"last":0.42099999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.377,"15min":1.9199999999999999},"min":{"1min":0.313,"5min":0.29799999999999999,"15min":0.26700000000000002},"max":{"1min":4.9169999999999998,"5min":28.117000000000001,"15min":153.54400000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73299999999999998,"15min":0.64300000000000002},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.23000000000000001},"max":{"1min":4.093,"5min":4.093,"15min":6.2329999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.73899999999999999,"15min":0.67800000000000005},"min":{"1min":0.27400000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":3.9409999999999998,"5min":4.0229999999999997,"15min":4.5300000000000002},"last":0.34100000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66000000000000003,"15min":0.61699999999999999},"min":{"1min":0.27000000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":1.669,"5min":2.5529999999999999,"15min":4.827},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":2.044,"5min":3.6659999999999999,"15min":6.0010000000000003},"last":0.38600000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:27:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.309,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":1.3759999999999999,"5min":4.0259999999999998,"15min":4.875},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":1.504,"5min":2.7949999999999999,"15min":4.6950000000000003},"last":0.63700000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117224,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1236180,"kb_used_data":609348,"kb_used_omap":1,"kb_used_meta":626686,"kb_avail":92508972,"statfs":{"total":95995035648,"available":94729187328,"internally_reserved":0,"allocated":623972352,"data_stored":623599897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":641726920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:37 2024","interfaces":[{"interface":"back","average":{"1min":2.0089999999999999,"5min":3.6520000000000001,"15min":2.343},"min":{"1min":0.27100000000000002,"5min":0.156,"15min":0.156},"max":{"1min":8.3149999999999995,"5min":130.40299999999999,"15min":130.40299999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.9379999999999999,"5min":3.2519999999999998,"15min":2.1299999999999999},"min":{"1min":0.30099999999999999,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":8.4939999999999998,"5min":130.70099999999999,"15min":130.70099999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Sep 17 08:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":2.0190000000000001,"15min":1.724},"min":{"1min":0.214,"5min":0.214,"15min":0.186},"max":{"1min":2.8879999999999999,"5min":71.459999999999994,"15min":71.459999999999994},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":2.2229999999999999,"15min":1.756},"min":{"1min":0.218,"5min":0.218,"15min":0.188},"max":{"1min":3.1520000000000001,"5min":71.438999999999993,"15min":71.438999999999993},"last":0.97299999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:26:35 2024","interfaces":[{"interface":"back","average":{"1min":2.714,"5min":1.9850000000000001,"15min":1.377},"min":{"1min":0.314,"5min":0.215,"15min":0.215},"max":{"1min":17.097999999999999,"5min":60.423999999999999,"15min":60.423999999999999},"last":1.014},{"interface":"front","average":{"1min":2.778,"5min":2.1200000000000001,"15min":1.8009999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":14.925000000000001,"5min":60.368000000000002,"15min":60.368000000000002},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:27:02 2024","interfaces":[{"interface":"back","average":{"1min":3.4399999999999999,"5min":1.655,"15min":1.5349999999999999},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.252},"max":{"1min":30.722999999999999,"5min":30.722999999999999,"15min":35.841000000000001},"last":1.042},{"interface":"front","average":{"1min":3.0819999999999999,"5min":1.4370000000000001,"15min":1.4419999999999999},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.27200000000000002},"max":{"1min":30.884,"5min":30.884,"15min":35.857999999999997},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:26:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3440000000000001,"15min":1.2789999999999999},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.29799999999999999},"max":{"1min":3.2570000000000001,"5min":5.6790000000000003,"15min":8.2240000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29499999999999998},"max":{"1min":4.71,"5min":9.7289999999999992,"15min":9.7289999999999992},"last":1.3009999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.1259999999999999,"15min":1.24},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":8.4550000000000001,"5min":8.4550000000000001,"15min":10.945},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.427,"15min":1.5149999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.311},"max":{"1min":5.5430000000000001,"5min":9.6600000000000001,"15min":9.6600000000000001},"last":0.5}]},{"osd":7,"last update":"Tue Sep 17 08:27:09 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":2.8159999999999998,"15min":1.855},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":3.8420000000000001,"5min":163.006,"15min":163.006},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.8879999999999999,"5min":3.113,"15min":1.9299999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28999999999999998},"max":{"1min":4.444,"5min":162.83699999999999,"15min":162.83699999999999},"last":1.083}]}]},{"osd":2,"up_from":227,"seq":974957576275,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1108768,"kb_used_data":828312,"kb_used_omap":4,"kb_used_meta":280443,"kb_avail":92636384,"statfs":{"total":95995035648,"available":94859657216,"internally_reserved":0,"allocated":848191488,"data_stored":847847873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":287174307},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.9510000000000001,"5min":1.5229999999999999,"15min":1.1559999999999999},"min":{"1min":0.245,"5min":0.188,"15min":0.188},"max":{"1min":5.6509999999999998,"5min":7.7450000000000001,"15min":7.7450000000000001},"last":4.5060000000000002},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3240000000000001,"15min":1.4179999999999999},"min":{"1min":0.30599999999999999,"5min":0.184,"15min":0.184},"max":{"1min":5.9329999999999998,"5min":9.1940000000000008,"15min":9.1940000000000008},"last":2.7429999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.96,"15min":1.474},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":5.2789999999999999,"5min":41.405000000000001,"15min":41.405000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":1.423,"5min":1.8959999999999999,"15min":1.48},"min":{"1min":0.20999999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":5.1449999999999996,"5min":41.25,"15min":41.25},"last":4.4770000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.7130000000000001,"15min":1.5209999999999999},"min":{"1min":0.218,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.1280000000000001,"5min":44.185000000000002,"15min":44.185000000000002},"last":2.0950000000000002},{"interface":"front","average":{"1min":1.823,"5min":1.843,"15min":1.494},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":7.492,"5min":43.994,"15min":43.994},"last":0.20300000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.153,"15min":0.93400000000000005},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":5.2169999999999996,"5min":5.524,"15min":5.524},"last":4.4509999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":0.98399999999999999,"15min":0.875},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.4699999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":3.952}]},{"osd":5,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.0509999999999999,"15min":0.97699999999999998},"min":{"1min":0.38100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.1059999999999999,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":4.4240000000000004},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0229999999999999,"15min":0.95499999999999996},"min":{"1min":0.32100000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.1659999999999999,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":1.5369999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.212,"15min":0.85799999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.309},{"interface":"front","average":{"1min":1.258,"5min":1.131,"15min":0.96199999999999997},"min":{"1min":0.40500000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.181,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":4.3849999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.127,"15min":1.0249999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.966,"5min":1.288,"15min":1.006},"min":{"1min":0.53800000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.468,"5min":7.468,"15min":7.468},"last":0.33400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1033304,"kb_used_data":366408,"kb_used_omap":1,"kb_used_meta":666878,"kb_avail":92711848,"statfs":{"total":95995035648,"available":94936932352,"internally_reserved":0,"allocated":375201792,"data_stored":374825287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":682883526},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:27:02 2024","interfaces":[{"interface":"back","average":{"1min":2.7160000000000002,"5min":1.921,"15min":1.8560000000000001},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":17.244,"5min":27.670999999999999,"15min":31.585999999999999},"last":0.752},{"interface":"front","average":{"1min":2.8050000000000002,"5min":1.6899999999999999,"15min":1.968},"min":{"1min":0.254,"5min":0.222,"15min":0.17199999999999999},"max":{"1min":17.364999999999998,"5min":26.120999999999999,"15min":35.807000000000002},"last":2.8559999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.712,"15min":1.099},"min":{"1min":0.28399999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":15.044,"5min":26.297999999999998,"15min":26.297999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":2.3519999999999999,"5min":1.6659999999999999,"15min":1.5449999999999999},"min":{"1min":0.25800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":8.2899999999999991,"5min":26.602,"15min":26.602},"last":6.4349999999999996}]},{"osd":3,"last update":"Tue Sep 17 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4099999999999999,"15min":1.45},"min":{"1min":0.25600000000000001,"5min":0.23200000000000001,"15min":0.184},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":8.0559999999999992},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.748,"5min":1.5469999999999999,"15min":1.599},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.189},"max":{"1min":3.149,"5min":22.984999999999999,"15min":22.984999999999999},"last":2.9990000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:27:09 2024","interfaces":[{"interface":"back","average":{"1min":1.9510000000000001,"5min":1.2250000000000001,"15min":1.373},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.25800000000000001},"max":{"1min":14.109,"5min":14.109,"15min":16.884},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.766,"5min":1.2729999999999999,"15min":1.254},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.25700000000000001},"max":{"1min":14.151999999999999,"5min":14.151999999999999,"15min":14.151999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2529999999999999,"15min":1.1919999999999999},"min":{"1min":0.39800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":3.9670000000000001,"5min":4.5,"15min":7.9139999999999997},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.3029999999999999,"15min":1.3440000000000001},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":2.718,"5min":16.853999999999999,"15min":16.853999999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.163,"15min":1.2809999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":2.3069999999999999,"5min":9.7159999999999993,"15min":10.253},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1479999999999999,"15min":1.2849999999999999},"min":{"1min":0.40999999999999998,"5min":0.33800000000000002,"15min":0.27200000000000002},"max":{"1min":4.0090000000000003,"5min":4.5369999999999999,"15min":7.9459999999999997},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Sep 17 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.9670000000000001,"15min":1.5429999999999999},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.33100000000000002},"max":{"1min":2.7519999999999998,"5min":47.442,"15min":47.442},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.7509999999999999,"15min":1.45},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.378},"max":{"1min":2.6970000000000001,"5min":47.475000000000001,"15min":47.475000000000001},"last":0.54000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738612,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1926448,"kb_used_data":1295656,"kb_used_omap":1,"kb_used_meta":630782,"kb_avail":91818704,"statfs":{"total":95995035648,"available":94022352896,"internally_reserved":0,"allocated":1326751744,"data_stored":1326399556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":645921222},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.3400000000000001,"15min":1.9630000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.189},"max":{"1min":5.9370000000000003,"5min":20.497,"15min":61.649000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.5669999999999999,"15min":2.1709999999999998},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":3.415,"5min":20.088000000000001,"15min":61.281999999999996},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:26:33 2024","interfaces":[{"interface":"back","average":{"1min":8.5210000000000008,"5min":4.0810000000000004,"15min":2.1160000000000001},"min":{"1min":0.32600000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":114.771,"5min":114.771,"15min":114.771},"last":1.1479999999999999},{"interface":"front","average":{"1min":8.4700000000000006,"5min":4.2670000000000003,"15min":2.0110000000000001},"min":{"1min":0.27900000000000003,"5min":0.193,"15min":0.193},"max":{"1min":114.732,"5min":114.732,"15min":114.732},"last":1.032}]},{"osd":3,"last update":"Tue Sep 17 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":3.0390000000000001,"5min":1.8819999999999999,"15min":2.2480000000000002},"min":{"1min":0.20899999999999999,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":8.2129999999999992,"5min":8.2129999999999992,"15min":61.926000000000002},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.298,"15min":1.8839999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.3490000000000002,"5min":10.868,"15min":61.616},"last":0.89400000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:27:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.284,"15min":1.45},"min":{"1min":0.36699999999999999,"5min":0.29799999999999999,"15min":0.26300000000000001},"max":{"1min":4.6429999999999998,"5min":5.242,"15min":41.158000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.371,"15min":1.5029999999999999},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":5.1520000000000001,"5min":6.8920000000000003,"15min":38.978000000000002},"last":2.8519999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:26:33 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.214,"15min":1.3440000000000001},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.6230000000000002,"5min":10.805999999999999,"15min":39.058},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.359,"15min":1.5269999999999999},"min":{"1min":0.39000000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":5.1260000000000003,"5min":5.1399999999999997,"15min":43.485999999999997},"last":1.129}]},{"osd":6,"last update":"Tue Sep 17 08:26:52 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.1200000000000001,"15min":1.355},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27700000000000002},"max":{"1min":6.0199999999999996,"5min":10.827999999999999,"15min":38.917999999999999},"last":1.085},{"interface":"front","average":{"1min":1.379,"5min":1.425,"15min":1.5740000000000001},"min":{"1min":0.42599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.8319999999999999,"5min":5.2400000000000002,"15min":41.189},"last":1.1020000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:26:14 2024","interfaces":[{"interface":"back","average":{"1min":2.294,"5min":1.8,"15min":1.651},"min":{"1min":0.42599999999999999,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":6.8680000000000003,"5min":39.045999999999999,"15min":39.045999999999999},"last":6.5330000000000004},{"interface":"front","average":{"1min":1.786,"5min":1.5569999999999999,"15min":1.5049999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.307},"max":{"1min":10.769,"5min":39.082999999999998,"15min":39.082999999999998},"last":6.8239999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1304760320,"data_stored":1304756248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":364969984,"data_stored":364969984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":818741248,"data_stored":818741248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":619380736,"data_stored":619380736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1038749696,"data_stored":1038745624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":643432448,"data_stored":643432448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":905646080,"data_stored":905642008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":527499264,"data_stored":527499264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:27:18.856 INFO:tasks.thrashosds.thrasher:Setting 5.2 to [5, 6, 2] 2024-09-17T08:27:18.856 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '5.2', '5', '6', '2'] 2024-09-17T08:27:18.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 5.2 5 6 2 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: pgmap v774: 158 pgs: 158 active+clean; 1.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.52k op/s 2024-09-17T08:27:18.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:18.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:18.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:18.977 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:18.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:19.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:19.114 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:19.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:19.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:19.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.251 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:19.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:19.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:19.402 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:19.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:19.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:19.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:27:19.586 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:19.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:19.590 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:19.605 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:19.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:19.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:19.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:19.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:19.846 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:19.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:19.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:19.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:19.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:19.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:20.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:20.131 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:20.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:20.233 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:20.268 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:20.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:20.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:20.407 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:20.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:20.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:20.541 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:20.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:20.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:20.665 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:20.666 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: pgmap v775: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: pgmap v775: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:27:20.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T08:27:20.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:20 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:20.682 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:20.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:20.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:20.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:20.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:20.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:20.952 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:20.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:20.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: pgmap v775: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-17T08:27:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T08:27:20.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:20 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:21.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:21.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:21.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:21.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:21.230 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:21.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:21.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:21.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.369 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:21.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:21.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:21.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.507 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:21.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:21.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:21.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:21.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:21.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:21.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:21.877 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:21.899 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:21.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:21.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T08:27:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:21 smithi173 ceph-mon[33187]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T08:27:22.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:22.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:22.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T08:27:22.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[40649]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4164976656' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T08:27:22.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:21 smithi079 ceph-mon[29565]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T08:27:22.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:22.181 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:22.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:22.282 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:22.311 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:22.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:22.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:22.448 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:22.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:22.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:22.585 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:22.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:22.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:22.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:22.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:22.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:22.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.855 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:22.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:22.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:22.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:22 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:27:22.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:22 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:27:22.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:22 smithi173 ceph-mon[33187]: pgmap v779: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:27:22.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:22.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:22.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[40649]: pgmap v779: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:27:22.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:22 smithi079 ceph-mon[29565]: pgmap v779: 158 pgs: 158 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-17T08:27:23.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:23.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:23.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:23.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:23.267 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:23.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:23.275 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:23.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:23.401 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:23.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:23.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:23.538 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:23.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:23.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:23.647 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:23.676 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:23.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:23.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:23.803 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:23.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:23.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:23.906 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 2.14 deep-scrub starts 2024-09-17T08:27:23.906 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 2.14 deep-scrub ok 2024-09-17T08:27:23.906 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 2.14 deep-scrub starts 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 2.14 deep-scrub ok 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:27:23.907 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:27:23.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:27:23.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:27:23.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:23.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:23.953 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:23.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:23.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 2.14 deep-scrub starts 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 2.14 deep-scrub ok 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:27:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:24.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:24.091 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:24.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:24.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:24.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:24.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:24.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:24.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.382 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:24.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:24.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:24.519 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:24.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:24.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:24.653 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:24.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: pgmap v780: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:24.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:27:24.753 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: pgmap v780: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:24.754 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:27:24.754 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:24 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:27:24.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:24.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.785 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:24.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:24.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:27:24.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:24.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:24.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: pgmap v780: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:27:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:24 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:27:25.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:25.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:25.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:25.184 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:25.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:25.321 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:25.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:25.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:25.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:25.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:25.590 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:25.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.591 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:25.692 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:25.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:25.827 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:25.864 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:25.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:25 smithi173 ceph-mon[33187]: 10.3 scrub starts 2024-09-17T08:27:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:25 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:27:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:25 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:27:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:25 smithi173 ceph-mon[33187]: 10.3 scrub ok 2024-09-17T08:27:25.994 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:25.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:25.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:26.096 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:26.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[40649]: 10.3 scrub starts 2024-09-17T08:27:26.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:27:26.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:27:26.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[40649]: 10.3 scrub ok 2024-09-17T08:27:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[29565]: 10.3 scrub starts 2024-09-17T08:27:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:27:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:27:26.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:25 smithi079 ceph-mon[29565]: 10.3 scrub ok 2024-09-17T08:27:26.126 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:26.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:26.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:26.257 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:26.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:26.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:26.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.397 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:26.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:26.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:26.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.520 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:26.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:26.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:26.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.656 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:26.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:26.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:26.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:26.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:26.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:26.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:26.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:26.932 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: pgmap v781: 158 pgs: 158 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T08:27:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:26 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:27:27.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:27.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:27.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: pgmap v781: 158 pgs: 158 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:27:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: pgmap v781: 158 pgs: 158 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T08:27:27.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:26 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:27:27.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:27.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.206 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:27.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:27.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:27.341 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:27.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:27.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:27.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.477 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:27.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:27.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:27.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:27.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:27.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:27.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:27.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:27.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:27.882 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:27.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:27.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 1.8 scrub starts 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 1.8 scrub ok 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 6.18 deep-scrub starts 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 6.18 deep-scrub ok 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 5.0 scrub starts 2024-09-17T08:27:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:27 smithi173 ceph-mon[33187]: 5.0 scrub ok 2024-09-17T08:27:27.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:28.016 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:28.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 1.8 scrub starts 2024-09-17T08:27:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 1.8 scrub ok 2024-09-17T08:27:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 6.18 deep-scrub starts 2024-09-17T08:27:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 6.18 deep-scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 5.0 scrub starts 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[40649]: 5.0 scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 1.8 scrub starts 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 1.8 scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 6.18 deep-scrub starts 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 6.18 deep-scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:27:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 5.0 scrub starts 2024-09-17T08:27:28.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:27 smithi079 ceph-mon[29565]: 5.0 scrub ok 2024-09-17T08:27:28.153 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:28.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:28.255 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:28.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:28.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:28.424 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:28.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:28.558 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:28.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:28.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:28.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:28.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.820 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:28.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:28.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:28.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:28 smithi173 ceph-mon[33187]: pgmap v783: 158 pgs: 158 active+clean; 997 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.31k op/s 2024-09-17T08:27:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:28 smithi173 ceph-mon[33187]: 5.16 deep-scrub starts 2024-09-17T08:27:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:28 smithi173 ceph-mon[33187]: 5.16 deep-scrub ok 2024-09-17T08:27:29.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:29.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:29.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:29.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[29565]: pgmap v783: 158 pgs: 158 active+clean; 997 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.31k op/s 2024-09-17T08:27:29.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[29565]: 5.16 deep-scrub starts 2024-09-17T08:27:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[29565]: 5.16 deep-scrub ok 2024-09-17T08:27:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[40649]: pgmap v783: 158 pgs: 158 active+clean; 997 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.31k op/s 2024-09-17T08:27:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[40649]: 5.16 deep-scrub starts 2024-09-17T08:27:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:28 smithi079 ceph-mon[40649]: 5.16 deep-scrub ok 2024-09-17T08:27:29.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:29.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:29.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:29.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:29.351 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:29.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:29.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.489 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:29.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:29.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:29.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:29.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:29.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:29.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:29.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:29.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:29.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:29.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:29.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:29.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:29 smithi173 ceph-mon[33187]: 10.c scrub starts 2024-09-17T08:27:29.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:29 smithi173 ceph-mon[33187]: 10.c scrub ok 2024-09-17T08:27:29.924 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:29 smithi173 ceph-mon[33187]: 6.13 deep-scrub starts 2024-09-17T08:27:29.925 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:29 smithi173 ceph-mon[33187]: 6.13 deep-scrub ok 2024-09-17T08:27:29.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:30.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.030 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:30.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[29565]: 10.c scrub starts 2024-09-17T08:27:30.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[29565]: 10.c scrub ok 2024-09-17T08:27:30.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[29565]: 6.13 deep-scrub starts 2024-09-17T08:27:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[29565]: 6.13 deep-scrub ok 2024-09-17T08:27:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[40649]: 10.c scrub starts 2024-09-17T08:27:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[40649]: 10.c scrub ok 2024-09-17T08:27:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[40649]: 6.13 deep-scrub starts 2024-09-17T08:27:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:29 smithi079 ceph-mon[40649]: 6.13 deep-scrub ok 2024-09-17T08:27:30.131 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:30.161 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:30.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:30.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:30.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:30.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:30.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:30.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:30.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:30.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:30.564 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:30.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:30.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:30.715 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:30.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:30.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:30.850 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:30.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:30.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:30.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:30 smithi173 ceph-mon[33187]: pgmap v784: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-17T08:27:30.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:30.990 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:30.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:31.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:31.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:31.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:30 smithi079 ceph-mon[29565]: pgmap v784: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-17T08:27:31.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:30 smithi079 ceph-mon[40649]: pgmap v784: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-17T08:27:31.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:31.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:31.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:31.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:31.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:31.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:31.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:31.509 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:31.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:31.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:31.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:31.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:31.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:31.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.776 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:31.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:31.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:31.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:31.919 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:31.920 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:31 smithi173 ceph-mon[33187]: 1.a scrub starts 2024-09-17T08:27:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:31 smithi173 ceph-mon[33187]: 1.a scrub ok 2024-09-17T08:27:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:31 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:27:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:31 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:27:32.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:32.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.046 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:32.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[40649]: 1.a scrub starts 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[40649]: 1.a scrub ok 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[29565]: 1.a scrub starts 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[29565]: 1.a scrub ok 2024-09-17T08:27:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:27:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:31 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:27:32.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:32.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:32.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:32.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:32.314 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:32.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:32.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:32.443 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:32.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:32.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:32.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.581 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:32.581 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:32.682 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:32.716 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:32.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:32.761 INFO:teuthology.orchestra.run.smithi079.stderr:set 5.2 pg_upmap mapping to [5,6,2] 2024-09-17T08:27:32.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:32.853 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:32.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:32.955 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: pgmap v785: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]: dispatch 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:32.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:32.988 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:32.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: pgmap v785: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]: dispatch 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: pgmap v785: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 779 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:27:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:27:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]: dispatch 2024-09-17T08:27:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:33.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:33.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:33.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:33.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:33.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.266 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:33.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:33.319 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:33.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:33.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.407 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:33.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:33.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:33.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:33.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:33.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:33.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:33.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:33.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:33.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:33.824 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:33.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:33.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:33.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:33.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:33.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:27:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: 1.2 scrub starts 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: 1.2 scrub ok 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]': finished 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:33 smithi173 ceph-mon[33187]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: 1.2 scrub starts 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: 1.2 scrub ok 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]': finished 2024-09-17T08:27:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T08:27:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[29565]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T08:27:34.056 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:27:34.056 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:27:34.056 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: 1.2 scrub starts 2024-09-17T08:27:34.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: 1.2 scrub ok 2024-09-17T08:27:34.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2844901668' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.2", "id": [5, 6, 2]}]': finished 2024-09-17T08:27:34.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T08:27:34.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:34.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:33 smithi079 ceph-mon[40649]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T08:27:34.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:34.105 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:34.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:34.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:34.241 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:27:34.241 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:27:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:34.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:34.271 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:27:34.271 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:27:34.306 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:27:34.306 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:27:34.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:27:34.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:27:34.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:34.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:27:34.372 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:27:34.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:34.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:34.403 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:27:34.403 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:27:34.437 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:27:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:27:34.470 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:27:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:27:34.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:34.508 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:34.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:34.610 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:34.638 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:34.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:34.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:34.778 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:34.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:34.833 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:27:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:27:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:27:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11680 9.6 9.a107be96 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.33846921200001, 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98902988199999997, 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11680 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492095+0000", 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492093+0000", 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492153+0000", 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0398e-05 2024-09-17T08:27:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492169+0000", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5747e-05 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492178+0000", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9889999999999998e-06 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.500907+0000", 2024-09-17T08:27:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0087288000000000001 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.500917+0000", 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0227999999999999e-05 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481114+0000", 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98019719299999997 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481125+0000", 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0677e-05 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11685 9.6 9.95710a26 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.33836435299997, 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98896512400000003, 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11685 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:27:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492200+0000", 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492199+0000", 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492221+0000", 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1826000000000001e-05 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492229+0000", 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5189999999999992e-06 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492236+0000", 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1709999999999999e-06 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501314+0000", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090785840000000007 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.501321+0000", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8669999999999996e-06 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481157+0000", 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97983637499999998 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481165+0000", 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4070000000000004e-06 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11687 9.6 9.1931af16 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.33832274399998, 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98896536099999999, 2024-09-17T08:27:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11687 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492241+0000", 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492264+0000", 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3332e-05 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492271+0000", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.233e-06 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.492276+0000", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6779999999999999e-06 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.502199+0000", 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099226100000000001 2024-09-17T08:27:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.502206+0000", 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2259999999999999e-06 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481200+0000", 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97899387199999999 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481207+0000", 2024-09-17T08:27:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8940000000000004e-06 2024-09-17T08:27:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11722 9.6 9.1e34edf6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.32139238600001, 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97205906399999997, 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11722 2024-09-17T08:27:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509172+0000", 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509169+0000", 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509202+0000", 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2230000000000001e-05 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509219+0000", 2024-09-17T08:27:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7748e-05 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509229+0000", 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.73e-06 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509283+0000", 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4252000000000002e-05 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.509290+0000", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6680000000000004e-06 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481224+0000", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97193378699999999 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481231+0000", 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11734 9.6 9.e7adb46 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.31504307199998, 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.96575857899999995, 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11734 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515521+0000", 2024-09-17T08:27:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515519+0000", 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515551+0000", 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2396e-05 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515573+0000", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1594999999999999e-05 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515582+0000", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4490000000000005e-06 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515650+0000", 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7528999999999997e-05 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.515660+0000", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0559e-05 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481272+0000", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.96561206600000005 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481280+0000", 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0960000000000003e-06 2024-09-17T08:27:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11755 9.6 9.44f58cd6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.30081125999999, 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95155133000000003, 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11755 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529753+0000", 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529750+0000", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999971 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529787+0000", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6903999999999998e-05 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529807+0000", 2024-09-17T08:27:34.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9723000000000001e-05 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529816+0000", 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.516e-06 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529876+0000", 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9246999999999997e-05 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.529884+0000", 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6770000000000001e-06 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481297+0000", 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95141305200000004 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481304+0000", 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:11758 9.6 9.46988236 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 432.29784258000001, 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.948607588, 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11758 2024-09-17T08:27:34.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532721+0000", 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532720+0000", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532745+0000", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4783999999999999e-05 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532760+0000", 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4521999999999999e-05 2024-09-17T08:27:34.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532769+0000", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4909999999999994e-06 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532827+0000", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.783e-05 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.532834+0000", 2024-09-17T08:27:34.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3180000000000001e-06 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481322+0000", 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.94848791200000004 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.481329+0000", 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7900000000000002e-06 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.24329.0:1632 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94434111868160 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e227)", 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 431.877182549, 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.53571094799999996, 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:27:34.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.24329", 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1632 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953382+0000", 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953379+0000", 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953387+0000", 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7030000000000006e-06 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953400+0000", 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953412+0000", 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2456999999999999e-05 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953473+0000", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0698000000000001e-05 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:22.953483+0000", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4730000000000006e-06 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.479366+0000", 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.52588388500000005 2024-09-17T08:27:34.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.479544+0000", 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017710100000000001 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:20:23.489092+0000", 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0095489370000000004 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210605 9.0 e272/230 9:018ae7c4:::benchmark_data_smithi079_143589_object79406:head v 272'12909, mlcod=272'12906)", 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.89067877700001, 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37561313400000002, 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939884+0000", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939892+0000", 2024-09-17T08:27:34.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.187e-06 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939902+0000", 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0917000000000001e-05 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939910+0000", 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3420000000000002e-06 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939949+0000", 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9501e-05 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939970+0000", 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0380000000000001e-05 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315468+0000", 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37549839200000001 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315498+0000", 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0312999999999999e-05 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210614 9.2 e272/230 9:448427a4:::benchmark_data_smithi079_143589_object79415:head v 272'13602, mlcod=272'13600)", 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.88440133900002, 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36429002500000002, 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946161+0000", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946171+0000", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303862+0000", 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35769168000000001 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303871+0000", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8189999999999998e-06 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304030+0000", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158679 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304053+0000", 2024-09-17T08:27:34.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3189000000000001e-05 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310428+0000", 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063754440000000001 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310453+0000", 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4318e-05 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210615 9.6 9:6555ccd0:::benchmark_data_smithi079_143589_object79416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.88391383499999, 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36942082399999998, 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210615 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.886 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946648+0000", 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946656+0000", 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303849+0000", 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35719317699999997 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303864+0000", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5011000000000001e-05 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303913+0000", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9240999999999998e-05 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304058+0000", 2024-09-17T08:27:34.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014518800000000001 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304185+0000", 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126952 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312414+0000", 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008228806 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312468+0000", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3854000000000003e-05 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315985+0000", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035164839999999998 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316026+0000", 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1301000000000003e-05 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316071+0000", 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5207999999999998e-05 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213094 9.6 9:6c0b11e2:::benchmark_data_smithi079_143589_object81895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.49319895299999, 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.31323531599999999, 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213094 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:27:34.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337363+0000", 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337371+0000", 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6359999999999994e-06 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337385+0000", 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4183e-05 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337392+0000", 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4719999999999998e-06 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337442+0000", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9378000000000001e-05 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337568+0000", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012602300000000001 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337689+0000", 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012135599999999999 2024-09-17T08:27:34.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341229+0000", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003539979 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.345283+0000", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040540510000000004 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650515+0000", 2024-09-17T08:27:34.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30523122800000002 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650560+0000", 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5611000000000001e-05 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650600+0000", 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0299000000000002e-05 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213100 9.6 9:65f8bbfc:::benchmark_data_smithi079_143589_object81901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.48934517599997, 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30948176700000002, 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213100 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341217+0000", 2024-09-17T08:27:34.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341225+0000", 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.328e-06 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341239+0000", 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4051e-05 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341247+0000", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3319999999999992e-06 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341301+0000", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3996000000000003e-05 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341442+0000", 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141328 2024-09-17T08:27:34.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341562+0000", 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011956 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.346195+0000", 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046330629999999998 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.347443+0000", 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001248292 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650634+0000", 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30319073099999999 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650668+0000", 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4360999999999998e-05 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650701+0000", 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2305999999999999e-05 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213109 9.6 9:690187cb:::benchmark_data_smithi079_143589_object81910:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.47974263200001, 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29998592099999999, 2024-09-17T08:27:34.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213109 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:27:34.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350820+0000", 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350826+0000", 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5889999999999999e-06 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350840+0000", 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3474e-05 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350848+0000", 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9829999999999996e-06 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350896+0000", 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8618000000000001e-05 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351004+0000", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108012 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351110+0000", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105951 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355185+0000", 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040743740000000004 2024-09-17T08:27:34.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355199+0000", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-05 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650732+0000", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29553320199999999 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650766+0000", 2024-09-17T08:27:34.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4260000000000001e-05 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650807+0000", 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0850999999999999e-05 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213119 9.0 e272/230 9:0a967850:::benchmark_data_smithi079_143589_object81920:head v 272'13077, mlcod=272'13075)", 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.46788639300001, 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29685224700000001, 2024-09-17T08:27:34.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362678+0000", 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362677+0000", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362684+0000", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1459999999999999e-06 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362696+0000", 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687999999999999e-05 2024-09-17T08:27:34.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362703+0000", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7259999999999996e-06 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362762+0000", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8424000000000002e-05 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.362787+0000", 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5799000000000002e-05 2024-09-17T08:27:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.659042+0000", 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29625426799999999 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.659530+0000", 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048823999999999997 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213120 9.0 e272/230 9:090e3aab:::benchmark_data_smithi079_143589_object81921:head v 272'13078, mlcod=272'13075)", 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.46449655399999, 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29576923700000002, 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366068+0000", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366067+0000", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.366076+0000", 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0210000000000005e-06 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650376+0000", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28430049099999999 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650392+0000", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6096999999999999e-05 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650464+0000", 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1775999999999999e-05 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650490+0000", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.552e-05 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.661805+0000", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011315725 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.661837+0000", 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1464000000000002e-05 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:213167 9.2 e272/230 9:4746aeca:::benchmark_data_smithi079_143589_object81968:head v 272'13735, mlcod=272'13733)", 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.42503814299999, 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.252615176, 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405526+0000", 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405524+0000", 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.405535+0000", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811e-05 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650412+0000", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.244876661 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650423+0000", 2024-09-17T08:27:34.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1259e-05 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650465+0000", 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1532000000000001e-05 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650490+0000", 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.516e-05 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.658002+0000", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075121060000000002 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.658141+0000", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013928899999999999 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121824 10.3 10:c7ef409d:::benchmark_data_smithi079_172150_object121823:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.008556+0000", 2024-09-17T08:27:34.925 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.82200826, 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.192729972, 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121824 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008556+0000", 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008556+0000", 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008555+0000", 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008580+0000", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5491999999999999e-05 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008591+0000", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0455000000000001e-05 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.008598+0000", 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2119999999999997e-06 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.011762+0000", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031639150000000002 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.011901+0000", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013886200000000001 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.012012+0000", 2024-09-17T08:27:34.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111055 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.014273+0000", 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022610659999999999 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.014722+0000", 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044920499999999999 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201156+0000", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18643401700000001 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201246+0000", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9952999999999997e-05 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201286+0000", 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9762000000000003e-05 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121830 10.3 10:c47db810:::benchmark_data_smithi079_172150_object121829:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.026714+0000", 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.803850195, 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17474314699999999, 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:27:34.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121830 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026714+0000", 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026714+0000", 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026713+0000", 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026734+0000", 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.175e-05 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026748+0000", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3405e-05 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026753+0000", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7320000000000003e-06 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026825+0000", 2024-09-17T08:27:34.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1662000000000006e-05 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.026966+0000", 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000140486 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.027506+0000", 2024-09-17T08:27:34.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054045999999999999 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.031402+0000", 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038961859999999998 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.032077+0000", 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000674908 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201334+0000", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16925675800000001 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201423+0000", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9290999999999996e-05 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201457+0000", 2024-09-17T08:27:34.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3865000000000002e-05 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121839 10.3 10:c3595298:::benchmark_data_smithi079_172150_object121838:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.040168+0000", 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.790395571, 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.161441634, 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:27:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121839 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040168+0000", 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040168+0000", 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:27:34.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040168+0000", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040190+0000", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2463000000000002e-05 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040206+0000", 2024-09-17T08:27:34.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5914999999999999e-05 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040213+0000", 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.272e-06 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040270+0000", 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6371e-05 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:27:34.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040416+0000", 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014626400000000001 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.040567+0000", 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000151526 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.043857+0000", 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032898179999999999 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.044283+0000", 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042564499999999999 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201499+0000", 2024-09-17T08:27:34.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.157216147 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201573+0000", 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4264999999999995e-05 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.201610+0000", 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6925999999999998e-05 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:27:34.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:27:34.946 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:27:34.981 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:34.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:34.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:35.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:35.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:27:35.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:27:35.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[29565]: pgmap v787: 158 pgs: 158 active+clean; 519 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-17T08:27:35.088 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1160297543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:35.089 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:27:35.089 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:27:35.089 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[40649]: pgmap v787: 158 pgs: 158 active+clean; 519 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-17T08:27:35.089 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:34 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1160297543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:35.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:35.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:35.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:34 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:27:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:34 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:27:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:34 smithi173 ceph-mon[33187]: pgmap v787: 158 pgs: 158 active+clean; 519 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-17T08:27:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:34 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1160297543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:35.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:35.340 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:35.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:35.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:35.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:35.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:35.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.612 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:35.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:35.713 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:35.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:35.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:35.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:35.883 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:35.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:35.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:35.985 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:36.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.016 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:36.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:36.117 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:36.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.156 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:36.156 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:36.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:36.291 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:36.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:36.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:36.428 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:36.529 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:36.530 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:36.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:36.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:36.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.705 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:36.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:36.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:36.843 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:36.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:36.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:36.974 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:36.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:37.018 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:36 smithi079 ceph-mon[40649]: pgmap v789: 158 pgs: 158 active+clean; 401 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.18k op/s 2024-09-17T08:27:37.018 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:36 smithi079 ceph-mon[29565]: pgmap v789: 158 pgs: 158 active+clean; 401 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.18k op/s 2024-09-17T08:27:37.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:27:37.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:37.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.105 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:37.207 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:37.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:36 smithi173 ceph-mon[33187]: pgmap v789: 158 pgs: 158 active+clean; 401 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.18k op/s 2024-09-17T08:27:37.240 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:37.342 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:37.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:37.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:37.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:37.601 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:37.633 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.634 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:37.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:37.775 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:37.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:37.816 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Removed 132529 objects 2024-09-17T08:27:37.816 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Clean up completed and total clean up time :109.415 2024-09-17T08:27:37.844 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:37.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:37.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:37.899 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:37.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:37.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:27:38.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:38.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.023 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:38.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:38.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:38.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:38.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:38.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:38.272 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:38.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:38.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:38.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:38.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:38.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.526 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:38.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:38.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:38.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:38.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:38.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:38.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:38.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:38.790 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:27:38.791 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:27:38.791 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:27:38.791 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.989011 2024-09-17T08:27:38.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 0.9890113460261052 2024-09-17T08:27:38.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[29565]: pgmap v790: 158 pgs: 158 active+clean; 226 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:27:38.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:38 smithi079 ceph-mon[40649]: pgmap v790: 158 pgs: 158 active+clean; 226 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-17T08:27:38.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:38.902 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:38.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:39.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:39.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.031 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:39.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:39.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:39.158 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:39.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:39.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:38 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:27:39.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:38 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:27:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:38 smithi173 ceph-mon[33187]: pgmap v790: 158 pgs: 158 active+clean; 226 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-17T08:27:39.260 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:39.284 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:39.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:39.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:39.411 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:39.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:39.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:39.535 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:39.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:39.553 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:39.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:39.659 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:39.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:39.725 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-17T08:27:39.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-17T08:27:39.742 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:27:39.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:39.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:39.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:39.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:39.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:39.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:39.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:39.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:27:39.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:27:39.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:27:39.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:27:39.974 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:27:40.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:40.039 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:40.140 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:40.167 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:40.213 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:39 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:27:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:39 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:27:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:39 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:27:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:39 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:27:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:27:40.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:40.294 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:40.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:40.418 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:40.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:40.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:40.541 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:40.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:40.643 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:40.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:40.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:40.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:40.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:40.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:40.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:40.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:41.024 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:41.046 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:41.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:41.080 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 4.1 deep-scrub starts 2024-09-17T08:27:41.080 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 4.1 deep-scrub ok 2024-09-17T08:27:41.080 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:27:41.080 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: pgmap v791: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[29565]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 4.1 deep-scrub starts 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 4.1 deep-scrub ok 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:27:41.081 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: pgmap v791: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:27:41.082 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:40 smithi079 ceph-mon[40649]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T08:27:41.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:41.169 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:41.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 4.1 deep-scrub starts 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 4.1 deep-scrub ok 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: pgmap v791: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1396914745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:27:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:40 smithi173 ceph-mon[33187]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T08:27:41.271 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:41.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:41.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:41.399 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:41.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:41.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:41.466 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:27:41.466 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:27:41.466 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-17T08:27:41.466 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:27:41.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:41.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:41.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:41.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:41.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:41.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:41.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:41.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:41.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:41.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:41.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:41.924 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:41.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:42.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:42.050 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:42.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:42.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:42.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.175 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:42.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:42.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:42.301 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:42.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:42.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:27:42.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:42.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:42.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:42.551 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:42.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:42.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:42 smithi079 ceph-mon[40649]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T08:27:42.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:42 smithi079 ceph-mon[29565]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T08:27:42.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:42.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:42.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:42.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:42.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:27:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:42.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:42.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:42.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:42.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:42.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:42.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:42 smithi173 ceph-mon[33187]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T08:27:43.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:43.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:43.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:43.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:43.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.187 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:43.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:43.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:43.312 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:43.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:43.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:43.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:43.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:43.563 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:43.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:43.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:43.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:43.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:43.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:43 smithi173 ceph-mon[33187]: pgmap v794: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.00k op/s 2024-09-17T08:27:43.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:43.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:43.818 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:43.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:43 smithi079 ceph-mon[40649]: pgmap v794: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.00k op/s 2024-09-17T08:27:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:43 smithi079 ceph-mon[29565]: pgmap v794: 158 pgs: 158 active+clean; 23 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.00k op/s 2024-09-17T08:27:43.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:43.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:43.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:43.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:44.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:44.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:44.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:44.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:44.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.193 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:44.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:44.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:44.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:44.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:44.438 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:44.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:44.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:44.566 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:44.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:44.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:44.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:44.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:44.690 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:44.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:44.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:44.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:44.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:44.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:44.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:44.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:45.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:45.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:45.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:45.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:45.191 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:45.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:45.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:45.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.316 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:45.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:45.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:45.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.440 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:45.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:45.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:45.567 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:45.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:45.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:45.679 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:45.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:45.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:45.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:45.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:45 smithi079 ceph-mon[40649]: pgmap v795: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 802 op/s 2024-09-17T08:27:45.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:45 smithi079 ceph-mon[29565]: pgmap v795: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 802 op/s 2024-09-17T08:27:45.815 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:45.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:45.916 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:45.942 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:45.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:45.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:45 smithi173 ceph-mon[33187]: pgmap v795: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 802 op/s 2024-09-17T08:27:46.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:46.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:46.168 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:46.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:46.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:46.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:46.402 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:27:46.402 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:27:46.402 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:46.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:46.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:27:46.428 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:27:46.445 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:27:46.446 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:27:46.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:46.461 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:27:46.461 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:27:46.478 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:27:46.479 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:27:46.494 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:27:46.494 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:27:46.512 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:27:46.513 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:27:46.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:27:46.529 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:27:46.550 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 0.989011 (8648152) 2024-09-17T08:27:46.551 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:46.576 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:46.678 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:46.687 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]: dispatch 2024-09-17T08:27:46.687 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3638091357' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:46.687 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]: dispatch 2024-09-17T08:27:46.688 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3638091357' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:46.700 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:46.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:46.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:46.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:46.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:46.949 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:46.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:46.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:46.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]: dispatch 2024-09-17T08:27:46.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3638091357' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:47.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:47.072 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:47.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:47.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:47.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:47.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:47.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:47.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.321 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:47.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:47.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:47.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:47.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:47.571 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:47.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[29565]: pgmap v796: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:27:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]': finished 2024-09-17T08:27:47.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[29565]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T08:27:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[40649]: pgmap v796: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:27:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]': finished 2024-09-17T08:27:47.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:47 smithi079 ceph-mon[40649]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T08:27:47.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:47.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.697 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:47.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:47.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:47.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:47.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:47.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:47.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:47.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:47.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:47.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:47 smithi173 ceph-mon[33187]: pgmap v796: 158 pgs: 158 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:27:47.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2253025046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9890113460261052}]': finished 2024-09-17T08:27:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:47 smithi173 ceph-mon[33187]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T08:27:48.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:48.071 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:48.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:48.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:48.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:48.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.320 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:48.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:48.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:48.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:48.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:48.571 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:48.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:48.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:48.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.697 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:48.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:48.822 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[40649]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[29565]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:48.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:48.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:48 smithi173 ceph-mon[33187]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T08:27:48.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:48.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:27:48.924 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:48.948 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:48.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:48.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:49.045 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:49.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:49.075 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:49.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:49.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.202 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:49.303 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:49.327 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:49.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:49.453 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:49.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.578 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:49.664 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[40649]: pgmap v799: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1004 B/s rd, 0 B/s wr, 59 op/s 2024-09-17T08:27:49.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 7 pgs inactive, 14 pgs peering (PG_AVAILABILITY) 2024-09-17T08:27:49.665 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:49.665 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[29565]: pgmap v799: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1004 B/s rd, 0 B/s wr, 59 op/s 2024-09-17T08:27:49.665 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 7 pgs inactive, 14 pgs peering (PG_AVAILABILITY) 2024-09-17T08:27:49.665 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:49 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:49.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:49.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:49.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:49.825 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:49.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:49.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:49.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:49.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:49.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:49 smithi173 ceph-mon[33187]: pgmap v799: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1004 B/s rd, 0 B/s wr, 59 op/s 2024-09-17T08:27:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:49 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 7 pgs inactive, 14 pgs peering (PG_AVAILABILITY) 2024-09-17T08:27:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:49 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:27:50.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:50.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:50.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:50.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:50.199 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:50.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:50.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:50.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:50.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:50.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:50.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:50.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:50.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:50.574 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:50.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:50.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:50.700 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:50.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:50.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:27:50.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:50.822 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:50.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:50.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:50.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:50.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:50.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:51.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:51.076 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:51.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:51.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:51.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:51.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:51.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:51.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:51.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:51.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:51.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.458 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:51.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_3' removed 2024-09-17T08:27:51.559 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:51.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:51.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:51.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:51.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:51.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: pgmap v800: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 46 op/s 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2452563457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:27:51.757 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[29565]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: pgmap v800: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 46 op/s 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2452563457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:27:51.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:27:51.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:51.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:27:51.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:51 smithi079 ceph-mon[40649]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T08:27:51.809 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:51.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:51.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:51.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:51.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:51.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:51.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: pgmap v800: 158 pgs: 21 peering, 137 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 46 op/s 2024-09-17T08:27:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2452563457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:27:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:51 smithi173 ceph-mon[33187]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T08:27:52.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:52.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.086 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:52.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:52.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:52.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:52.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:52.210 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:27:52.210 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:27:52.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:52.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:52.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:52.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:52.455 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:52.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:52.557 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:52.579 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:52.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:52.681 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:52.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.703 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:52.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:52.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:52.826 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:52.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:52.864 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:52 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:52.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:52 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:52.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:52 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:52.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:52 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:52.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:52.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:52.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:52.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:52 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:52 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:53.052 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:53.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:53.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:53.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:27:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:27:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:27:53.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:53.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.197 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:53.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:53.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:53.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.321 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:53.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:53.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:53.444 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:53.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:53.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:27:53.530 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:53.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:53.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.568 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:53.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:53.575 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:27:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:27:53.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:53 smithi173 ceph-mon[33187]: pgmap v803: 142 pgs: 15 peering, 127 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-17T08:27:53.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:53.586 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:27:53.670 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:53.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:53.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:53.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:53 smithi079 ceph-mon[40649]: pgmap v803: 142 pgs: 15 peering, 127 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-17T08:27:53.733 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:53.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:53 smithi079 ceph-mon[29565]: pgmap v803: 142 pgs: 15 peering, 127 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-17T08:27:53.736 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:27:53.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:53.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.817 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:53.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:53.918 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:53.934 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-17T08:27:53.935 INFO:tasks.radosbench:Beginning radosbench... 2024-09-17T08:27:53.935 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-17T08:27:53.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_4 16 2024-09-17T08:27:53.939 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:53.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:53.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:54.041 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:54.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:54.072 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:27:54.072 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:27:54.072 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T08:27:54.073 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:27:54.073 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:27:54.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-17T08:27:54.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:54.188 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:54.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:54.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:54.312 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:54.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.435 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:54.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:54.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.558 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:54.659 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:54.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:54.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:54.804 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:54.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:54.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:54.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:54.838 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[40649]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T08:27:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:54.838 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:54 smithi079 ceph-mon[29565]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T08:27:54.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:54.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:54.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:54.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:54.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:27:54.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:54 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:27:54.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1171322935' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:27:54.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:54 smithi173 ceph-mon[33187]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T08:27:55.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:55.052 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:55.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:55.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:55.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.177 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:55.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:55.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:55.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.304 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:55.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:55.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:55.421 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:55.427 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:55.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:55.456 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:55.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:55.550 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:55.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:55.674 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:55.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:55.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:55.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:55.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:55 smithi079 ceph-mon[40649]: pgmap v805: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:55.805 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:55 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 14 pgs peering) 2024-09-17T08:27:55.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:55 smithi079 ceph-mon[29565]: pgmap v805: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:55.805 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:55 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 14 pgs peering) 2024-09-17T08:27:55.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:55.919 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:55.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:55.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:55 smithi173 ceph-mon[33187]: pgmap v805: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:55 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 14 pgs peering) 2024-09-17T08:27:56.021 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:56.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:56.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:56.166 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:56.268 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:56.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:56.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:56.412 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:56.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:56.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.540 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:56.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:56.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:56.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:27:56.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:56.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.789 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:56.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:56.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:56.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:56.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:56.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:57.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:57.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:57.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:57.137 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:57.159 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:57.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:27:57.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.283 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:57.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:27:57.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:57.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:57.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:57.519 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:57 smithi079 ceph-mon[40649]: pgmap v806: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:57.520 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:57 smithi079 ceph-mon[40649]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T08:27:57.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:57 smithi079 ceph-mon[29565]: pgmap v806: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:57.520 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:57 smithi079 ceph-mon[29565]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T08:27:57.530 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:57.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:57.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.654 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.654 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:57.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:57.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:57.878 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:57.900 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:57.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:57.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:57 smithi173 ceph-mon[33187]: pgmap v806: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-17T08:27:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:57 smithi173 ceph-mon[33187]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T08:27:58.002 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:58.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.024 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:58.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:58.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:58.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:58.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:58.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:27:58.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:58.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:27:58.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:58.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:58.396 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:58.399 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:27:58.399 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:27:58.408 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:27:58.408 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:27:58.425 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:27:58.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:27:58.445 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:27:58.446 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:27:58.461 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:27:58.461 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:27:58.477 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:27:58.477 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:27:58.493 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:27:58.494 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:27:58.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:58.514 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:27:58.515 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:27:58.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:58.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:27:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:27:58.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1112171572' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:27:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:27:58.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:58 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1112171572' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:58.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:58.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:58.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:58.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:58.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.768 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:58.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:58.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:27:58.874 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:58.874 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:27:58.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:58.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:58.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:27:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:58 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:27:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:58 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:27:58.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:58 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1112171572' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:27:58.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:59.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.018 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:59.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:59.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:27:59.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:59.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.142 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:27:59.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:59.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:59.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:59.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:27:59.391 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:59.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:27:59.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:59.515 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_4' created 2024-09-17T08:27:59.523 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:59.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:59.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[29565]: pgmap v808: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[29565]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1013038924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[40649]: pgmap v808: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[40649]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1013038924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:27:59.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:27:59 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:27:59.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:27:59.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:59.651 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:27:59.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:27:59.776 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:27:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.777 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:27:59.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:27:59.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:27:59.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:27:59.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:27:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:59 smithi173 ceph-mon[33187]: pgmap v808: 142 pgs: 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:27:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:59 smithi173 ceph-mon[33187]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T08:27:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:59 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1013038924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:27:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:27:59 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-17T08:28:00.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:00.029 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:00.131 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:00.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:00.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:00.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:00.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:00.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:00.403 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:00.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:00.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:00.535 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:00.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:00.663 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:00.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-17T08:28:00.765 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:00.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:00 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-17T08:28:00.773 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:00 smithi079 ceph-mon[40649]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T08:28:00.773 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:00 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-17T08:28:00.773 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:00 smithi079 ceph-mon[29565]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T08:28:00.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:00.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:00.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:00.921 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:00.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:00 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-17T08:28:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:00 smithi173 ceph-mon[33187]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T08:28:01.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.049 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:01.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:01.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:01.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:01.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:01.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:01.301 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:01.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:01.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:01.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:01.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:01.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:01.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:01.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:01.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[40649]: pgmap v811: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:28:01.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[40649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:28:01.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[40649]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T08:28:01.618 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[40649]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T08:28:01.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[29565]: pgmap v811: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:28:01.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[29565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:28:01.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[29565]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T08:28:01.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:01 smithi079 ceph-mon[29565]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238263 9.9 9:970a2f6b:::benchmark_data_smithi079_143589_object107064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.65060184100003, 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55651207300000005, 2024-09-17T08:28:01.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238263 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961766+0000", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961772+0000", 2024-09-17T08:28:01.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9970000000000004e-06 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961796+0000", 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3638e-05 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961803+0000", 2024-09-17T08:28:01.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9949999999999999e-06 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961847+0000", 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3470999999999998e-05 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961955+0000", 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108051 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962124+0000", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016913200000000001 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.968042+0000", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059184399999999996 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033570+0000", 2024-09-17T08:28:01.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065527335000000006 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518190+0000", 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48461996699999998 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518239+0000", 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9529e-05 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518280+0000", 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0831000000000003e-05 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238266 9.9 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.64987419300002, 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55589687899999995, 2024-09-17T08:28:01.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238266 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962499+0000", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4819999999999996e-06 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962532+0000", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2979999999999999e-05 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962548+0000", 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6359e-05 2024-09-17T08:28:01.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965118+0000", 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025690779999999998 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965221+0000", 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103297 2024-09-17T08:28:01.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965327+0000", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106302 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.968153+0000", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028257759999999999 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.040235+0000", 2024-09-17T08:28:01.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072081920999999993 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518320+0000", 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47808557299999999 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518360+0000", 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9138999999999999e-05 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518392+0000", 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2768000000000002e-05 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238290 9.5 9:ac6fd467:::benchmark_data_smithi079_143589_object107091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.56550421700001, 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48563189400000001, 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238290 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046863+0000", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046872+0000", 2024-09-17T08:28:01.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9220000000000003e-06 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046890+0000", 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8059000000000001e-05 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046900+0000", 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8549999999999997e-06 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046953+0000", 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3090000000000002e-05 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.047110+0000", 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156219 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.047224+0000", 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114768 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.049865+0000", 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002640725 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.049952+0000", 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.666e-05 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532402+0000", 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48245032700000001 2024-09-17T08:28:01.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532445+0000", 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3058e-05 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532497+0000", 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203999999999997e-05 2024-09-17T08:28:01.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:119017 10.8 10:1bd4cd59:::benchmark_data_smithi079_172150_object119016:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.420108172, 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23653085400000001, 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119017 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192260+0000", 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192286+0000", 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6441e-05 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192313+0000", 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6330999999999999e-05 2024-09-17T08:28:01.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192321+0000", 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8010000000000004e-06 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192361+0000", 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0958999999999999e-05 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192488+0000", 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126334 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,7", 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192611+0000", 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123399 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198243+0000", 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056316229999999997 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198979+0000", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073625899999999996 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428687+0000", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.229707467 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428737+0000", 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0133999999999998e-05 2024-09-17T08:28:01.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428792+0000", 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5630000000000001e-05 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.655 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:119020 10.8 10:1af502c1:::benchmark_data_smithi079_172150_object119019:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.418780983, 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23606200299999999, 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:28:01.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119020 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193587+0000", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193609+0000", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2028999999999999e-05 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193620+0000", 2024-09-17T08:28:01.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0216e-05 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193626+0000", 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8040000000000004e-06 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195292+0000", 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016656710000000001 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195409+0000", 2024-09-17T08:28:01.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011663999999999999 2024-09-17T08:28:01.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,7", 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195540+0000", 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131409 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198351+0000", 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028104279999999998 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199234+0000", 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088331200000000003 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429570+0000", 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23033583699999999 2024-09-17T08:28:01.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429618+0000", 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7882e-05 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429651+0000", 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3118999999999999e-05 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119023 10.1 e315/287 10:87de14dd:::benchmark_data_smithi079_172150_object119022:head v 315'7535, mlcod=315'7531)", 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.41311609499999, 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22938773000000001, 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199253+0000", 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200796+0000", 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543083 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200806+0000", 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3700000000000001e-06 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200813+0000", 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0450000000000003e-06 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200874+0000", 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1484999999999998e-05 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200893+0000", 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8522e-05 2024-09-17T08:28:01.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428596+0000", 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22770304 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428641+0000", 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5673000000000001e-05 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:119017 10.8 e315/287 ondisk, result = 0)", 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.41180281999999, 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.228234619, 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200566+0000", 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200571+0000", 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2630000000000003e-06 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200582+0000", 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0951999999999999e-05 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200588+0000", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1800000000000001e-06 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428672+0000", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.228083016 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428680+0000", 2024-09-17T08:28:01.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428801+0000", 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121805 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119026 10.c e315/287 10:306dec2d:::benchmark_data_smithi079_172150_object119025:head v 315'7477, mlcod=315'7475)", 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:28:01.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.41153421000001, 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23368836000000001, 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200868+0000", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2904000000000002e-05 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200875+0000", 2024-09-17T08:28:01.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9450000000000004e-06 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200879+0000", 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6310000000000002e-06 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201055+0000", 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000176236 2024-09-17T08:28:01.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201072+0000", 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6368e-05 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.434273+0000", 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23320106600000001 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.434524+0000", 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000250967 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119027 10.2 e315/287 10:42c7ce22:::benchmark_data_smithi079_172150_object119026:head v 315'7297, mlcod=315'7295)", 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.41121232200001, 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23186778399999999, 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:28:01.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201180+0000", 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3482e-05 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428534+0000", 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22735417699999999 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428547+0000", 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.275e-05 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428618+0000", 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1295000000000001e-05 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428645+0000", 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6401e-05 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.432945+0000", 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043006119999999997 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.433025+0000", 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9678999999999994e-05 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:119020 10.8 e315/287 ondisk, result = 0)", 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.40996181899999, 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22724935700000001, 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202407+0000", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202413+0000", 2024-09-17T08:28:01.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428521+0000", 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.226107843 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428541+0000", 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9791000000000001e-05 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429557+0000", 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010156620000000001 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429564+0000", 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3980000000000002e-06 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429657+0000", 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2985000000000005e-05 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119030 10.c e315/287 10:3b44ad5c:::benchmark_data_smithi079_172150_object119029:head v 315'7478, mlcod=315'7475)", 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.40851067700001, 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.2340527, 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:28:01.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203856+0000", 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999967 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203960+0000", 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001042 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428837+0000", 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22487770800000001 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428845+0000", 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1629999999999999e-06 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.431334+0000", 2024-09-17T08:28:01.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:01.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024897270000000002 2024-09-17T08:28:01.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.431353+0000", 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8521000000000001e-05 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.437883+0000", 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065302750000000003 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.437912+0000", 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8523000000000001e-05 2024-09-17T08:28:01.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119029 10.1 e315/287 10:898b06f3:::benchmark_data_smithi079_172150_object119028:head v 315'7536, mlcod=315'7533)", 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.40848081799999, 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23247437100000001, 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203888+0000", 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:28:01.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428932+0000", 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22504348800000001 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428940+0000", 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2770000000000005e-06 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428945+0000", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.305e-06 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429160+0000", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000214477 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429179+0000", 2024-09-17T08:28:01.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9454999999999999e-05 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.436327+0000", 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071472150000000002 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.436363+0000", 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6649000000000001e-05 2024-09-17T08:28:01.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121829 10.4 10:2418dda7:::benchmark_data_smithi079_172150_object121828:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 142.59480920999999, 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17637900000000001, 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121829 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:28:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017559+0000", 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017581+0000", 2024-09-17T08:28:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1721000000000001e-05 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017596+0000", 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5184999999999999e-05 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017602+0000", 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017651+0000", 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8971000000000003e-05 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017808+0000", 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015683900000000001 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,4", 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017965+0000", 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156625 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.020680+0000", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027150059999999998 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.020812+0000", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013275400000000001 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193767+0000", 2024-09-17T08:28:01.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17295470399999999 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193898+0000", 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013092600000000001 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193939+0000", 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1458000000000002e-05 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121831 10.4 10:29d5055b:::benchmark_data_smithi079_172150_object121830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.030292+0000", 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 142.58207752999999, 2024-09-17T08:28:01.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16384456, 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121831 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030292+0000", 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030292+0000", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030290+0000", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030315+0000", 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.527e-05 2024-09-17T08:28:01.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030331+0000", 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6038000000000001e-05 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030338+0000", 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9929999999999998e-06 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030384+0000", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5940999999999999e-05 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030512+0000", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000127732 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,4", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.030657+0000", 2024-09-17T08:28:01.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144888 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.033387+0000", 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002730213 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.033494+0000", 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010678100000000001 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193981+0000", 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16048732800000001 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.194101+0000", 2024-09-17T08:28:01.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119469 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.194137+0000", 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5877999999999998e-05 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121841 10.4 10:2725599a:::benchmark_data_smithi079_172150_object121840:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:28:01.712 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.042196+0000", 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 142.57017388099999, 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15293122000000001, 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121841 2024-09-17T08:28:01.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042196+0000", 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042196+0000", 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042194+0000", 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042221+0000", 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-05 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042235+0000", 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3439e-05 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042243+0000", 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.49e-06 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042289+0000", 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5627999999999997e-05 2024-09-17T08:28:01.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042406+0000", 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117422 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,4", 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.042522+0000", 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115578 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.045415+0000", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028929289999999998 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.046062+0000", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064724599999999998 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.194876+0000", 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14881387800000001 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.195048+0000", 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017198300000000001 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.195127+0000", 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9010000000000004e-05 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:227104 10.5 e339/287 10:a9cd0999:::benchmark_data_smithi079_172150_object94572:head v 339'14216, mlcod=339'14212)", 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:27:07.699830+0000", 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.912540016000001, 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.1057703, 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699830+0000", 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699830+0000", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699828+0000", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699836+0000", 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2790000000000001e-06 2024-09-17T08:28:01.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699848+0000", 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2666e-05 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699855+0000", 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1570000000000003e-06 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699889+0000", 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3155999999999999e-05 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.699907+0000", 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8559999999999998e-05 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.805344+0000", 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.105436642 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.805600+0000", 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025611999999999998 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:227105 10.5 e339/287 10:adaaced1:::benchmark_data_smithi079_172150_object94573:head v 339'14217, mlcod=339'14212)", 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:27:07.705855+0000", 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.906514465999997, 2024-09-17T08:28:01.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.102939945, 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705855+0000", 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705855+0000", 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705854+0000", 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705862+0000", 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.763e-06 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705872+0000", 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0112000000000001e-05 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705878+0000", 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705938+0000", 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0046000000000001e-05 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.705957+0000", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8383999999999999e-05 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.808649+0000", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.102692565 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.808795+0000", 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145802 2024-09-17T08:28:01.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:227111 10.b e339/287 10:d97e7339:::benchmark_data_smithi079_172150_object94579:head v 339'14109, mlcod=339'14107)", 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:27:07.707130+0000", 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.905239332000001, 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.094614172999999996, 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.707130+0000", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.707130+0000", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.707129+0000", 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.707137+0000", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9899999999999997e-06 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.778706+0000", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.071568941999999997 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.778716+0000", 2024-09-17T08:28:01.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0063e-05 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.778799+0000", 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2782999999999997e-05 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.778817+0000", 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.791e-05 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.801714+0000", 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022897026000000001 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.801744+0000", 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0682000000000002e-05 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:227113 10.5 e339/287 10:aa0cd92c:::benchmark_data_smithi079_172150_object94581:head v 339'14218, mlcod=339'14212)", 2024-09-17T08:28:01.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:27:07.709724+0000", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.902645624999998, 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.099094470000000004, 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.709724+0000", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.709724+0000", 2024-09-17T08:28:01.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.709723+0000", 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.709729+0000", 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.75e-06 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777074+0000", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067344887000000006 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777079+0000", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2530000000000004e-06 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777487+0000", 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000407666 2024-09-17T08:28:01.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777508+0000", 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1251e-05 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.808798+0000", 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031290336000000002 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.808818+0000", 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0009000000000001e-05 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:227123 10.c e339/287 10:35d05910:::benchmark_data_smithi079_172150_object94591:head v 339'14327, mlcod=339'14324)", 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:27:07.718089+0000", 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.894280682000002, 2024-09-17T08:28:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.094104186000000006, 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.718089+0000", 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.718089+0000", 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:28:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.718087+0000", 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.718100+0000", 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.279e-05 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:28:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777049+0000", 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058948636999999998 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.777066+0000", 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7197000000000002e-05 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.780259+0000", 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031930790000000001 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.780279+0000", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9738999999999999e-05 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.812128+0000", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031849652999999999 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:27:07.812193+0000", 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4638e-05 2024-09-17T08:28:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:01.746 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:28:01.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.768 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:01.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:01.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:28:01.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:01.894 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:01.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:01.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:01 smithi173 ceph-mon[33187]: pgmap v811: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:28:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:01 smithi173 ceph-mon[33187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T08:28:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:01 smithi173 ceph-mon[33187]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T08:28:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:01 smithi173 ceph-mon[33187]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T08:28:01.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:02.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.020 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:02.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:02.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:02.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:02.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:02.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:02.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.277 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:02.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:02.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:02.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.406 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:02.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:02.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:02.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:02.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:02.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:02.657 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:02.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:02.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:02.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.781 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:02.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:02.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:02.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:02.903 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:02.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:03.004 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:03.031 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:03.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:03.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:03.154 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:03.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:03.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:03.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.283 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:03.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:03.327 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:03.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:03.410 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:03.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:03.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:03.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.536 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:03.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[29565]: pgmap v814: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:03.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:03.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[40649]: pgmap v814: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:03.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:03.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:03.663 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:03.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:03.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:03 smithi173 ceph-mon[33187]: pgmap v814: 158 pgs: 16 unknown, 142 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:03.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:03.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.790 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:03.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:03.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:03.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:03.918 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:03.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:04.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:04.041 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:04.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:04.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:04.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:04.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:04.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:28:04.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:04.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.304 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:04.304 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:04.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:04.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.432 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:04.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:04.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:04.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:04.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:04.623 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:04.624 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:04.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:04.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.683 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:04.683 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:04.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:04.805 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:04.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:04.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:04.907 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:04.933 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:04.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:04.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:05.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:05.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:05.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:05.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:05.184 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:05.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:05.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:05.311 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:05.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:05.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:05.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.441 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:05.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:05.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:05.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.565 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:05.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:05.666 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:05.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:05.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:05.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:05.819 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:05.821 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:05 smithi079 ceph-mon[29565]: pgmap v815: 158 pgs: 7 unknown, 151 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-09-17T08:28:05.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:05 smithi079 ceph-mon[40649]: pgmap v815: 158 pgs: 7 unknown, 151 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-09-17T08:28:05.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:05.943 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:05.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:05.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:05 smithi173 ceph-mon[33187]: pgmap v815: 158 pgs: 7 unknown, 151 active+clean; 629 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-09-17T08:28:06.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:06.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:06.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:06.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:06.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:06.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:06.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:06.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:06.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:06.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:06.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:06.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:06.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:06.573 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:06.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.574 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:06.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:06.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:06.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:06.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:06.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:06.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:06.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:06.945 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:06.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:07.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:07.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.068 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:07.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:07.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:07.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:07.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:07.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:07.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:07.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:07.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:07.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:07.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:07.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:07.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:07.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:07.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:07.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:07.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:07.745 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:07 smithi079 ceph-mon[40649]: pgmap v816: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 838 B/s rd, 0 op/s 2024-09-17T08:28:07.746 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:07 smithi079 ceph-mon[29565]: pgmap v816: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 838 B/s rd, 0 op/s 2024-09-17T08:28:07.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:07.818 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:07.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:07.920 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:07.945 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:07.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:07.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:07 smithi173 ceph-mon[33187]: pgmap v816: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 838 B/s rd, 0 op/s 2024-09-17T08:28:08.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:08.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.071 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:08.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:08.126 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:28:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:28:08.142 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:28:08.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:28:08.157 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:28:08.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:28:08.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:08.175 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:28:08.175 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:28:08.194 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:28:08.194 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:28:08.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:08.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:08.209 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:28:08.209 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:28:08.225 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:28:08.225 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:28:08.242 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:28:08.242 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:28:08.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:08.322 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:08.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.324 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:08.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:08.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.450 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:08.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:08.543 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:28:08.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:08.573 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:08.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:08.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:08.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:08.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:08.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:28:08.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3982064657' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:08.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:28:08.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3982064657' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:08.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:08.819 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:08.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:08.921 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:08.946 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:08.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:08.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:08.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:28:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3982064657' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:09.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:09.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:09.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:09.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:09.195 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.196 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:09.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:09.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:09.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:09.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:09.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:09.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:09.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:09.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:09.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:09.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:09.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.818 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.818 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:09.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[40649]: pgmap v817: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:28:09.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[40649]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[29565]: pgmap v817: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:28:09.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:09 smithi079 ceph-mon[29565]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T08:28:09.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:09.941 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:09.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:09.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:09 smithi173 ceph-mon[33187]: pgmap v817: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:28:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:09 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/335917148' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:28:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:09 smithi173 ceph-mon[33187]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T08:28:10.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:10.049 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:28:10.049 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T08:28:10.049 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:28:10.064 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:10.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:10.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:10.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:10.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:10.319 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:10.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:10.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:10.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:10.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.574 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:10.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:10.701 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:10.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:28:10.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:28:10.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:28:10.768 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:28:10.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:28:10.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:28:10.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:28:10.769 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:10 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:28:10.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:10.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:10.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:10.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:10.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:10.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:10 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:28:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:10 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:28:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:10 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:28:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:10 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:28:11.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:11.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.083 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:11.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:11.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:11.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:11.310 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:11.336 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:11.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:11.460 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:11.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:11.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.583 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:11.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:11.706 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:11.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:11.829 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:11.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:28:11.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: pgmap v819: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:28:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:28:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:28:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:28:11.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[40649]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T08:28:11.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: pgmap v819: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:11.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:28:11.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:28:11.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:28:11.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:28:11.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:11 smithi079 ceph-mon[29565]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T08:28:11.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:11.952 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:11.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:11.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: pgmap v819: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:28:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:11 smithi173 ceph-mon[33187]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T08:28:12.000 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:12.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:12.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:12.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:12.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:12.204 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:12.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:12.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:12.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:12.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:12.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:12.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.456 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:12.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:12.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:12.560 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-17T08:28:12.580 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:12.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:12.682 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:12.707 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:12.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.708 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:12.751 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:28:12.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:28:12.809 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:12.833 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:12.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.834 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:12.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:12.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:12.957 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:12.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:12 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:28:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:12 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:28:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:12 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:28:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:12 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:28:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T08:28:13.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:13.083 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:13.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:13.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:28:13.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:13.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.243 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:13.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:13.364 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:13.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:13.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:13.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:13.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:13.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:13.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:13.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:13.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:13.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:13.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:13.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-mon[33187]: pgmap v821: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:28:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-mon[33187]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T08:28:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:28:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:13 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:28:13.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.744 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:13.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[40649]: pgmap v821: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[40649]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[29565]: pgmap v821: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/951699497' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-17T08:28:13.766 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[29565]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T08:28:13.767 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:28:13.767 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:13 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:28:13.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:13.866 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:13.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:13.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:13.995 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:13.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:13.996 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:14.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:14.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:14.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:14.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:14.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:14.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:14.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:14.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.405 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:14.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:14.446 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:28:14.446 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:28:14.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:14.532 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:14.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:14.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:14.659 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:14.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 1.3 scrub starts 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 1.3 scrub ok 2024-09-17T08:28:14.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 1.9 scrub starts 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 1.9 scrub ok 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:14.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 1.3 scrub starts 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 1.3 scrub ok 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 1.9 scrub starts 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 1.9 scrub ok 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:28:14.749 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:14 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:28:14.761 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:14.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:14.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:14.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:14.912 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:14.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:14.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:14.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:28:14.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 1.3 scrub starts 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 1.3 scrub ok 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 1.9 scrub starts 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 1.9 scrub ok 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:28:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:14 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:28:15.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:15.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:15.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:15.157 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:15.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:28:15.222 DEBUG:teuthology.orchestra.run.smithi079:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-17T08:28:15.259 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-17T08:28:15.259 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-17T08:28:15.259 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:15.286 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:15.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:15.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:15.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:hints = 1 2024-09-17T08:28:15.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-17T08:28:15.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object prefix: benchmark_data_smithi079_199625 2024-09-17T08:28:15.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:28:15.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 0 0 0 0 0 0 - 0 2024-09-17T08:28:15.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:15.547 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:15.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:15.687 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: pgmap v824: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:28:15.747 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[40649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: pgmap v824: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:28:15.748 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:15 smithi079 ceph-mon[29565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:28:15.789 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:15.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:15.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:15.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:15.961 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:15.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:15.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: pgmap v824: 158 pgs: 158 active+clean; 629 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s 2024-09-17T08:28:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:28:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:28:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:28:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:28:15.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:15 smithi173 ceph-mon[33187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T08:28:16.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:16.100 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:16.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:16.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:16.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:16.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:16.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:16.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:16.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 1 15 3654 3639 227.425 227.438 0.00606091 0.00438438 2024-09-17T08:28:16.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:16.508 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:16.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:16.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:28:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:28:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:28:16.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:28:16.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:16 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:28:16.647 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:16.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:16.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:16.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:16.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:16.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:16.923 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:16.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:16.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:16.947 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:28:16.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:28:16.947 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T08:28:16.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:28:16.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:16 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:28:17.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:17.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.062 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:17.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:17.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:17.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:17.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:17.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:17.321 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:17.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.322 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:17.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:17.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:17.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:17.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 2 16 6723 6707 209.58 191.75 0.00121581 0.00459682 2024-09-17T08:28:17.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:17.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:17.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:17.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:17.721 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:17.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:17.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:17.861 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:17.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[40649]: pgmap v825: 158 pgs: 158 active+clean; 577 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:28:17.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[29565]: pgmap v825: 158 pgs: 158 active+clean; 577 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:28:17.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:17 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:28:17.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:17 smithi173 ceph-mon[33187]: pgmap v825: 158 pgs: 158 active+clean; 577 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-17T08:28:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:17 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:28:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:17 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:28:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:17 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:28:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:17 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:28:17.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:17.992 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:17.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:18.013 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:18.018 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:18.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:18.125 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:18.227 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:18.257 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:18.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:18.391 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:18.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 3 16 9781 9765 203.422 191.125 0.0028976 0.00490816 2024-09-17T08:28:18.493 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:18.524 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:18.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:18.657 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:18.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:18.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.798 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:18.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: pgmap v826: 158 pgs: 158 active+clean; 119 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s wr, 308 op/s 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:28:18.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: pgmap v826: 158 pgs: 158 active+clean; 119 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s wr, 308 op/s 2024-09-17T08:28:18.856 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:18.856 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:18.856 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:28:18.856 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:18 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:28:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:28:18.890 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:28:18.890 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:28:18.900 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:18.919 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:28:18.920 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:28:18.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:18.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:18.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:18.952 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:28:18.953 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: pgmap v826: 158 pgs: 158 active+clean; 119 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s wr, 308 op/s 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:28:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:18 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:28:18.989 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:28:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:28:19.025 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:28:19.025 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:28:19.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:19.059 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:28:19.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:28:19.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:19.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:19.110 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:28:19.110 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:28:19.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:19.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:19.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:19.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:19.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:19.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:19.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:19.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 4 16 12378 12362 193.141 162.312 0.00590898 0.00516937 2024-09-17T08:28:19.465 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:19.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:19.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:19.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:19.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:19.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:19.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:28:19.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:28:19.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:28:19.723 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:28:19.727 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:28:19.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:28:19.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1706689284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:19.728 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1706689284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:19.729 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:19.734 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:28:19.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:19.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:19.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:19.874 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:19.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:19.875 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:19.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1706689284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:19.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:20.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:20.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:20.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:20.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.150 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:20.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:20.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:20.283 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:20.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:20.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:20.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:20.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 5 15 14890 14875 185.923 157.062 0.00480907 0.00537232 2024-09-17T08:28:20.524 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:20.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:20.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.556 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:20.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:20.695 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:20.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:20.797 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:20.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:20.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: pgmap v827: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:28:20.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:28:20.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: pgmap v827: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:28:20.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:20 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:28:20.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:20.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:20.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:20.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: pgmap v827: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3022546822' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:20.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T08:28:20.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:28:20.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:28:20.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:20 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:28:21.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:21.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:21.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:21.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.264 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:21.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:21.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:21.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:21.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:21.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 6 16 17388 17372 180.944 156.062 0.00664947 0.00551972 2024-09-17T08:28:21.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:21.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.540 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:21.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:21.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:21.671 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:21.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:21.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:21.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:21.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:21.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:21.947 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:21.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:21.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:22.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:28:22.064 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[29565]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:28:22.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:21 smithi079 ceph-mon[40649]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T08:28:22.098 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:22.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:22.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:22.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:28:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:21 smithi173 ceph-mon[33187]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T08:28:22.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:22.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:22.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:22.372 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:22.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:22.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 7 16 20023 20007 178.619 164.688 0.00917748 0.00559298 2024-09-17T08:28:22.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:22.514 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:22.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:22.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:22.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:22.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:22.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:22.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:22.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:22.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:22.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:22.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:22.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:23.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:23.064 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:23.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:23.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:23.206 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:23.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:23.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:23.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:23.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:23.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:23.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 8 16 22480 22464 175.485 153.562 0.0024874 0.00569228 2024-09-17T08:28:23.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:23.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:23.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:23.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:23.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:23.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:23.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: pgmap v830: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 6.c deep-scrub starts 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 6.c deep-scrub ok 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:28:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:23 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:28:23.741 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:23.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:23.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: pgmap v830: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 6.c deep-scrub starts 2024-09-17T08:28:23.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 6.c deep-scrub ok 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: pgmap v830: 158 pgs: 158 active+clean; 543 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 68 MiB/s wr, 1.08k op/s 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 6.c deep-scrub starts 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 6.c deep-scrub ok 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:28:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:28:23.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:28:23.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:23 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:28:23.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:23.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:23.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:23.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:24.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:24.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:28:24.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:24.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.187 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:24.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:24.250 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:28:24.250 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:28:24.250 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 367, 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:28:21.507213+0000", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:20:22.465407+0000", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:15:28.332713+0000", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:28:24.251 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 19, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 11, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:28:24.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.253 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 11, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 11, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 11, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 11, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 11, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.254 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "334", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "332", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.255 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.256 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.257 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9099999666213989, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9099999666213989, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.258 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.259 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:28:24.260 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.261 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.262 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.263 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.264 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5399999618530273, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5399999618530273, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:28:24.265 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:28:24.266 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.267 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.268 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.269 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:28:24.270 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.3299999237060547, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.3299999237060547, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:28:24.273 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.274 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.275 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.276 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.277 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.190000057220459, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.190000057220459, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.5, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.5, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.280 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.281 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.282 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.283 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.5, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.5, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.284 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.285 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.286 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.287 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.288 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.289 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5399999618530273, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5399999618530273, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:28:24.290 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 11, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_4", 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:27:59.045325+0000", 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:28:24.291 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:28:24.292 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "364", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:28:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:28:24.294 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:28:24.295 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:28:24.296 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.75, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.75, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:28:24.297 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:28:24.298 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.299 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:28:24.300 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.301 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:28:24.302 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.303 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:28:24.304 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.305 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:28:24.306 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.308 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.309 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 18, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 204, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 227, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 205, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.310 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.312 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/359954782", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/359954782", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/359954782", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/359954782", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.316 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.320 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.321 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:28:24.322 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:28:24.323 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:28:24.324 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:28:24.328 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.329 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 205, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:28:24.334 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.9889984130859375, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.335 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:28:24.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:28:24.341 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:28:24.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:28:24.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:28:24.346 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:13.892952+0000", 2024-09-17T08:28:24.347 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:19.216457+0000", 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:57.760692+0000", 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 204 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.349 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:28:13.301028+0000", 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:51.017634+0000", 2024-09-17T08:28:24.350 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:11.532277+0000", 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.351 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:31.824114+0000", 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:28:24.352 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:27:49.772119+0000", 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: 4, 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: 6 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "5.2", 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: 6, 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:28:24.354 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "6.b", 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2024-09-17T08:28:24.357 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 6 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:28:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:28:24.359 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:28:24.361 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:28:24.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:24.396 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:24.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.397 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:24.445 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 9 16 24676 24660 171.236 137.25 0.00560029 0.00583387 2024-09-17T08:28:24.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:24.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:24.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:24.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:24.664 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:24.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:24.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:24.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:24.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 1.4 deep-scrub starts 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 1.4 deep-scrub ok 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/379231275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:28:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 1.4 deep-scrub starts 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 1.4 deep-scrub ok 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/379231275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:28:24.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:24 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:28:24.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:24.923 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:24.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:24.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 1.4 deep-scrub starts 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 1.4 deep-scrub ok 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/379231275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:28:24.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:24 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:28:25.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:25.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:25.155 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:25.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:25.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:25.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:25.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:25.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:25.438 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:25.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 10 16 26692 26676 166.711 126 0.00181431 0.00597486 2024-09-17T08:28:25.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:25.569 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:25.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:25.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.704 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:25.805 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:25.837 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[40649]: pgmap v831: 158 pgs: 158 active+clean; 1010 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 126 MiB/s wr, 2.02k op/s 2024-09-17T08:28:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:28:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:28:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:28:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[29565]: pgmap v831: 158 pgs: 158 active+clean; 1010 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 126 MiB/s wr, 2.02k op/s 2024-09-17T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:25 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:28:25.939 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:25.972 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:25.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:25.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:25 smithi173 ceph-mon[33187]: pgmap v831: 158 pgs: 158 active+clean; 1010 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 126 MiB/s wr, 2.02k op/s 2024-09-17T08:28:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:25 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:28:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:25 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:28:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:25 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:28:25.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:25 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:28:26.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:26.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:28:26.111 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:26.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:26.214 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:26.241 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:26.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:26.382 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:26.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:26.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 11 16 28504 28488 161.85 113.25 0.00642181 0.00617304 2024-09-17T08:28:26.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:26.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:26.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:26.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:26.653 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:26.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:26.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:26.786 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:26.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[29565]: 4.c deep-scrub starts 2024-09-17T08:28:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[29565]: 4.c deep-scrub ok 2024-09-17T08:28:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:28:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:28:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[40649]: 4.c deep-scrub starts 2024-09-17T08:28:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[40649]: 4.c deep-scrub ok 2024-09-17T08:28:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:28:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:26 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:28:26.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:26.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:26.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:26.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:26 smithi173 ceph-mon[33187]: 4.c deep-scrub starts 2024-09-17T08:28:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:26 smithi173 ceph-mon[33187]: 4.c deep-scrub ok 2024-09-17T08:28:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:26 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:28:26.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:26 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:28:27.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:27.059 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:27.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:27.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:27.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:27.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:27.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:27.329 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:27.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:27.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:27.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 12 16 30357 30341 158.013 115.812 0.0018077 0.00630739 2024-09-17T08:28:27.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.452 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:27.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:27.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:27.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.575 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:27.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:27.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:27.712 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:27.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:27.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:27.842 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:27.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: pgmap v832: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 153 MiB/s wr, 2.45k op/s 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 1.5 scrub starts 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 1.5 scrub ok 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: pgmap v832: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 153 MiB/s wr, 2.45k op/s 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:28:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:28:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 1.5 scrub starts 2024-09-17T08:28:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 1.5 scrub ok 2024-09-17T08:28:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:28:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:28:27.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:27 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:28:27.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: pgmap v832: 158 pgs: 158 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 153 MiB/s wr, 2.45k op/s 2024-09-17T08:28:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:28:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:28:27.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 1.5 scrub starts 2024-09-17T08:28:27.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 1.5 scrub ok 2024-09-17T08:28:27.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:28:27.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:28:27.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:27 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:28:27.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:27.976 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:27.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:28.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:28.115 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:28.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:28.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:28.246 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:28.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:28.348 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:28.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:28.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:28.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 13 16 32281 32265 155.107 120.25 0.00792402 0.00644057 2024-09-17T08:28:28.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:28.516 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:28.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:28.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:28.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.640 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:28.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:28.741 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:28.773 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:28.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:28.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[40649]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T08:28:28.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:28:28.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:28:28.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:28 smithi079 ceph-mon[29565]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T08:28:28.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:28.907 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:28.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:28.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:28:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:28 smithi173 ceph-mon[33187]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T08:28:29.010 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:29.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:29.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:29.166 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:29.268 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:29.289 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:29.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:29.423 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:29.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 14 16 33887 33871 151.197 100.375 0.00655144 0.00660844 2024-09-17T08:28:29.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:29.557 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:29.659 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:29.693 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:29.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:29.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:29.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:29.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: pgmap v834: 158 pgs: 158 active+clean; 1.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 128 MiB/s wr, 2.05k op/s 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 5.b deep-scrub starts 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 5.b deep-scrub ok 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 1.6 deep-scrub starts 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[29565]: 1.6 deep-scrub ok 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: pgmap v834: 158 pgs: 158 active+clean; 1.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 128 MiB/s wr, 2.05k op/s 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:28:29.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:28:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 5.b deep-scrub starts 2024-09-17T08:28:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 5.b deep-scrub ok 2024-09-17T08:28:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 1.6 deep-scrub starts 2024-09-17T08:28:29.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:29 smithi079 ceph-mon[40649]: 1.6 deep-scrub ok 2024-09-17T08:28:29.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:29.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:29.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: pgmap v834: 158 pgs: 158 active+clean; 1.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 128 MiB/s wr, 2.05k op/s 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 5.b deep-scrub starts 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 5.b deep-scrub ok 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 1.6 deep-scrub starts 2024-09-17T08:28:29.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:29 smithi173 ceph-mon[33187]: 1.6 deep-scrub ok 2024-09-17T08:28:30.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:30.104 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:30.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:30.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:30.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:30.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:30.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:30.377 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:30.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:30.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 15 16 35644 35628 148.438 109.812 0.00699549 0.00673145 2024-09-17T08:28:30.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:30.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:30.618 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:30.648 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:30.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:30.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:30.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:30.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 5.1c deep-scrub starts 2024-09-17T08:28:30.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 5.1c deep-scrub ok 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 5.1c deep-scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 5.1c deep-scrub ok 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:28:30.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:28:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:28:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:28:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:28:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:28:30.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:30 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:28:30.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:30.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:30.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:30.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:30.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 5.1c deep-scrub starts 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 5.1c deep-scrub ok 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:30 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:31.012 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:31.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:31.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:31.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:31.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:31.300 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:31.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:31.433 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:31.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 16 16 37118 37102 144.918 92.125 0.00350003 0.00689553 2024-09-17T08:28:31.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:31.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:31.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:31.692 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:31.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:31.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: pgmap v835: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 177 MiB/s wr, 2.82k op/s 2024-09-17T08:28:31.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[29565]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: pgmap v835: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 177 MiB/s wr, 2.82k op/s 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:28:31.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:28:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:28:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:31 smithi079 ceph-mon[40649]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T08:28:31.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:31.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:31.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:31.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:31.994 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: pgmap v835: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 177 MiB/s wr, 2.82k op/s 2024-09-17T08:28:31.994 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:28:31.994 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:28:31.994 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:28:31.995 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:28:31.995 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:28:31.995 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:31 smithi173 ceph-mon[33187]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T08:28:32.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:32.130 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:32.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:32.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:32.268 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:32.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:32.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:32.397 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:32.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:32.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 17 16 38810 38794 142.613 105.75 0.00868038 0.00700663 2024-09-17T08:28:32.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:32.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:32.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:32.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:32.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:32.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:32.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:32.804 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:32.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[40649]: 1.1 scrub starts 2024-09-17T08:28:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[40649]: 1.1 scrub ok 2024-09-17T08:28:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[40649]: 1.7 scrub starts 2024-09-17T08:28:32.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[40649]: 1.7 scrub ok 2024-09-17T08:28:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[29565]: 1.1 scrub starts 2024-09-17T08:28:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[29565]: 1.1 scrub ok 2024-09-17T08:28:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[29565]: 1.7 scrub starts 2024-09-17T08:28:32.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:32 smithi079 ceph-mon[29565]: 1.7 scrub ok 2024-09-17T08:28:32.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:32.938 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:32.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:32.939 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:32 smithi173 ceph-mon[33187]: 1.1 scrub starts 2024-09-17T08:28:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:32 smithi173 ceph-mon[33187]: 1.1 scrub ok 2024-09-17T08:28:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:32 smithi173 ceph-mon[33187]: 1.7 scrub starts 2024-09-17T08:28:32.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:32 smithi173 ceph-mon[33187]: 1.7 scrub ok 2024-09-17T08:28:33.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:33.078 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:33.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:33.180 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:33.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:33.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:33.302 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:33.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:33.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:33.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:33.446 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 18 16 40579 40563 140.832 110.562 0.00958557 0.00709462 2024-09-17T08:28:33.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:33.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:33.524 INFO:tasks.thrashosds.thrasher:j is {'epoch': 367, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:28:21.507213+0000', 'last_up_change': '2024-09-17T08:20:22.465407+0000', 'last_in_change': '2024-09-17T08:15:28.332713+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '334', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '332', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 1.909999966621399, 'score_stable': 1.909999966621399, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-17T08:27:59.045325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '364', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 18, 'last_clean_end': 204, 'up_from': 227, 'up_thru': 359, 'down_at': 205, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 359954782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 359954782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 359954782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 359954782}]}, 'public_addr': '172.21.15.79:6819/359954782', 'cluster_addr': '172.21.15.79:6821/359954782', 'heartbeat_back_addr': '172.21.15.79:6825/359954782', 'heartbeat_front_addr': '172.21.15.79:6823/359954782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 205, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9889984130859375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:13.892952+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:19.216457+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:57.760692+0000', 'dead_epoch': 204}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:28:13.301028+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:51.017634+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:11.532277+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:31.824114+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:27:49.772119+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '5.2', 'osds': [5, 6, 2]}], 'pg_upmap_items': [{'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 2}, {'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:28:33.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:28:33.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:33.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:33.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:33.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:33.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:33.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:33.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:33.849 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:33.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: pgmap v837: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 121 MiB/s wr, 1.93k op/s 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: pgmap v837: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 121 MiB/s wr, 1.93k op/s 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:28:33.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:28:33.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:28:33.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:33.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:33.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:33.872 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:33.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:33.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: pgmap v837: 158 pgs: 158 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 121 MiB/s wr, 1.93k op/s 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:34.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:34.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:34.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:34.145 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:34.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:34.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:34.282 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:34.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:34.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:34.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:34.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:34.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 19 16 41990 41974 138.061 88.1875 0.00807112 0.00723767 2024-09-17T08:28:34.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:34.547 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:34.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:34.649 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:34.674 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:34.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:34.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:28:34.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:34 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:28:34.820 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:34.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:34.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:34.949 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:28:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:28:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:28:34.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:28:34.949 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:28:34.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:34 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:28:34.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:34 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:28:34.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:34.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:34 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:28:34.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:34 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:28:34.987 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:34.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:34.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:35.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:35.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:35.231 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:35.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:35.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:35.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:35.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:35.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:28:35.447534+0000 min lat: 0.00119686 max lat: 0.194888 avg lat: 0.00730914 2024-09-17T08:28:35.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:28:35.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 20 16 43769 43753 136.717 111.188 0.0113806 0.00730914 2024-09-17T08:28:35.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:35.534 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:35.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:35.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:35.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:35.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:35.795 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:35.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[40649]: pgmap v838: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.83k op/s 2024-09-17T08:28:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:28:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[29565]: pgmap v838: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.83k op/s 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:28:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:28:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:35 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:28:35.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:35.925 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:35.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:35.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:35 smithi173 ceph-mon[33187]: pgmap v838: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.83k op/s 2024-09-17T08:28:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:35 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:28:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:35 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:28:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:35 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:28:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:35 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:28:36.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:36.066 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:36.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:36.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:36.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:36.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:36.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:36.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:36.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:36.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:36.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 21 16 45626 45610 135.732 116.062 0.00710879 0.00736229 2024-09-17T08:28:36.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:36.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:36.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:36.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:36.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:36.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:36.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:36.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:36.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:28:36.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:36.874 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:36.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:36.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:28:36.885 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 2.0 deep-scrub starts 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: 2.0 deep-scrub ok 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: pgmap v839: 158 pgs: 158 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 120 MiB/s wr, 1.92k op/s 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[29565]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:28:36.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 2.0 deep-scrub starts 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: 2.0 deep-scrub ok 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: pgmap v839: 158 pgs: 158 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 120 MiB/s wr, 1.92k op/s 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T08:28:36.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:36 smithi079 ceph-mon[40649]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 2.0 deep-scrub starts 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: 2.0 deep-scrub ok 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: pgmap v839: 158 pgs: 158 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 120 MiB/s wr, 1.92k op/s 2024-09-17T08:28:36.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T08:28:36.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:36 smithi173 ceph-mon[33187]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T08:28:36.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:37.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:37.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:37.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:37.162 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:37.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:37.264 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:37.294 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:37.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:37.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:37.438 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:37.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:37.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 22 16 47377 47361 134.537 109.438 0.009991 0.00742781 2024-09-17T08:28:37.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:37.566 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:37.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:37.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:37.695 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:37.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:37.797 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:37.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.832 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:37.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:28:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:28:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:28:37.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:28:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:28:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:28:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:28:37.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:37 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:28:37.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:37.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:37 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:28:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:37 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:28:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:37 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:28:37.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:37 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:28:37.988 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:37.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:37.993 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:37.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:38.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:38.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:38.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:38.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:38.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:38.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:38.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:38.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 23 16 49084 49068 133.325 106.688 0.0128094 0.00749566 2024-09-17T08:28:38.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:38.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:38.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:38.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:38.659 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:38.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:38.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:38.802 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:38.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:38.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:38.940 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:38.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:38.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:38.967 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:28:38.968 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:28:38.968 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[29565]: pgmap v842: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 96 MiB/s wr, 1.53k op/s 2024-09-17T08:28:38.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:28:38.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:28:38.968 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:38 smithi079 ceph-mon[40649]: pgmap v842: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 96 MiB/s wr, 1.53k op/s 2024-09-17T08:28:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:38 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:28:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:38 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:28:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:38 smithi173 ceph-mon[33187]: pgmap v842: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 96 MiB/s wr, 1.53k op/s 2024-09-17T08:28:39.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:39.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.073 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:39.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:39.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:39.205 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:39.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:39.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:39.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:39.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:39.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:39.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 24 16 50700 50684 131.978 101 0.00948961 0.00757157 2024-09-17T08:28:39.472 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:39.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:39.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:39.614 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:39.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:39.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:39.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.742 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:39.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:39.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:39.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:39.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:39.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 6.17 deep-scrub starts 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 6.17 deep-scrub ok 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:28:39.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3864545286' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:39.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:40.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:40.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:40.057 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:28:40.057 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:28:40.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:28:40.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 6.17 deep-scrub starts 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 6.17 deep-scrub ok 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:28:40.060 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3864545286' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 6.17 deep-scrub starts 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 6.17 deep-scrub ok 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:28:40.061 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3864545286' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:40.115 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:40.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:40.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:40.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:40.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:40.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:40.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:40.416 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:40.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:40.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 25 16 52240 52224 130.549 96.25 0.00427588 0.00765466 2024-09-17T08:28:40.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:40.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:40.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:40.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:40.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:40.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:40.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:40.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.816 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:40.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:40.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:40.938 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:40.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:40.939 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:40.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:40 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:28:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:40 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:28:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:40 smithi173 ceph-mon[33187]: 5.9 deep-scrub starts 2024-09-17T08:28:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:40 smithi173 ceph-mon[33187]: 5.9 deep-scrub ok 2024-09-17T08:28:40.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:40 smithi173 ceph-mon[33187]: pgmap v843: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 132 MiB/s wr, 2.11k op/s 2024-09-17T08:28:41.039 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:41.061 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:41.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:28:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:28:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[40649]: 5.9 deep-scrub starts 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[40649]: 5.9 deep-scrub ok 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[40649]: pgmap v843: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 132 MiB/s wr, 2.11k op/s 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[29565]: 5.9 deep-scrub starts 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[29565]: 5.9 deep-scrub ok 2024-09-17T08:28:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:40 smithi079 ceph-mon[29565]: pgmap v843: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 132 MiB/s wr, 2.11k op/s 2024-09-17T08:28:41.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:41.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.200 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:41.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:41.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:41.335 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:41.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:41.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:41.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 26 16 53242 53226 127.936 62.625 0.0148307 0.0078107 2024-09-17T08:28:41.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:41.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:41.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:41.615 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:41.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:41.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:41.745 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:41.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:41.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:41.882 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:41.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:41.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:42.012 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:42.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:42.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:42.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:42.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:42.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:42.447 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 27 16 54744 54728 126.674 93.875 0.0106568 0.00788876 2024-09-17T08:28:42.455 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:42.552 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:28:42.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:42.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:42.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:42.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:28:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:28:42.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3353592868' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:28:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:28:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3353592868' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:42 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.876 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:42.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:42.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:42.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:42 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:28:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:42 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:28:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3353592868' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:42 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:28:42.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:43.013 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:43.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:43.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:43.153 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:43.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:43.255 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:43.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.292 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:43.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:43.365 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:43.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:43.422 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:43.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:43.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 28 16 56340 56324 125.71 99.75 0.00675335 0.00794942 2024-09-17T08:28:43.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:43.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:43.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:43.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: pgmap v844: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[40649]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: pgmap v844: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:43.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:28:43.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:43 smithi079 ceph-mon[29565]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T08:28:43.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:43.694 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:43.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.695 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:43.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: pgmap v844: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:28:43.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:43 smithi173 ceph-mon[33187]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T08:28:43.796 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:43.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:43.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:43.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:43.973 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:43.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:43.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:44.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:44.116 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:44.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:44.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:44.246 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:44.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:44.348 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:44.384 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:44.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:44.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 29 16 57927 57911 124.796 99.1875 0.00696142 0.00800806 2024-09-17T08:28:44.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:44.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:44.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:44.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:44.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:28:44.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.664 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:44.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:44.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:44.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:44.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:44.839 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:44.839 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:44.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:44.946 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:44.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:44.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:45.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:45.079 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:45.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:45.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:45.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:45.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:45.357 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:45.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.358 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:45.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:28:45.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 30 16 59450 59434 123.808 95.1875 0.0176534 0.0080718 2024-09-17T08:28:45.459 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:45.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:45.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:45.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:45.646 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:45.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:45.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:45.777 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:45.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:45.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[29565]: pgmap v846: 158 pgs: 158 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 640 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:28:45.829 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:28:45.829 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:28:45.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[40649]: pgmap v846: 158 pgs: 158 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 640 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:28:45.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:28:45.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:45 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:28:45.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:45.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:45.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:45.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:45 smithi173 ceph-mon[33187]: pgmap v846: 158 pgs: 158 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 640 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:28:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:45 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:45 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:46.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:46.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.051 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:46.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:46.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:46.185 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:46.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.186 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:46.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:46.311 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:46.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.312 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:46.413 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:46.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:46.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:46.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 31 15 60911 60896 122.762 91.375 0.0120991 0.00814079 2024-09-17T08:28:46.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:46.579 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:46.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:46.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:46.720 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:46.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:46.822 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:46.849 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:46.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:46 smithi079 ceph-mon[40649]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T08:28:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:46 smithi079 ceph-mon[29565]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T08:28:46.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:46 smithi173 ceph-mon[33187]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T08:28:46.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:46.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:46.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:47.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:47.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:47.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:47.260 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:47.362 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:47.396 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:47.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:47.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 32 16 62204 62188 121.449 80.75 0.00810924 0.00822899 2024-09-17T08:28:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:28:47.466 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:28:47.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:47.529 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:47.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:47.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:47.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:47.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:47.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:47 smithi079 ceph-mon[40649]: pgmap v848: 158 pgs: 158 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:28:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:47 smithi079 ceph-mon[29565]: pgmap v848: 158 pgs: 158 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:28:47.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:47.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:47.956 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:47.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:47.962 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:47.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:47 smithi173 ceph-mon[33187]: pgmap v848: 158 pgs: 158 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-17T08:28:48.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:48.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:48.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:48.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:48.240 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:48.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:48.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:48.376 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:48.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:48.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 33 16 63663 63647 120.532 91.1875 0.00631151 0.00829155 2024-09-17T08:28:48.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:48.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:48.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:48.585 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":848,"stamp":"2024-09-17T08:28:45.629812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3657898262,"num_objects":56002,"num_object_clones":0,"num_object_copies":168006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56002,"num_whiteouts":0,"num_read":1922,"num_read_kb":1674,"num_write":56522,"num_write_kb":3571627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4625,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58090,"ondisk_log_size":58090,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15252260,"kb_used_data":10524252,"kb_used_omap":15,"kb_used_meta":4727920,"kb_avail":734708956,"statfs":{"total":767960285184,"available":752341970944,"internally_reserved":0,"allocated":10776834048,"data_stored":10772970268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":4841390869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":874381312,"num_objects":13342,"num_object_clones":0,"num_object_copies":40026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13342,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":13342,"num_write_kb":853888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003573"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":395,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528653+0000","last_change":"2024-09-17T08:27:12.669483+0000","last_active":"2024-09-17T08:28:36.528653+0000","last_peered":"2024-09-17T08:28:36.528653+0000","last_clean":"2024-09-17T08:28:36.528653+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:28:36.528653+0000","last_undegraded":"2024-09-17T08:28:36.528653+0000","last_fullsized":"2024-09-17T08:28:36.528653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:12.669002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.669002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039241620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":414,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.553285+0000","last_change":"2024-09-17T08:26:23.241237+0000","last_active":"2024-09-17T08:28:42.553285+0000","last_peered":"2024-09-17T08:28:42.553285+0000","last_clean":"2024-09-17T08:28:42.553285+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:28:42.553285+0000","last_undegraded":"2024-09-17T08:28:42.553285+0000","last_fullsized":"2024-09-17T08:28:42.553285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:23.241125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:26:23.241125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002185354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":472,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.530711+0000","last_change":"2024-09-17T08:28:29.665518+0000","last_active":"2024-09-17T08:28:36.530711+0000","last_peered":"2024-09-17T08:28:36.530711+0000","last_clean":"2024-09-17T08:28:36.530711+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:28:36.530711+0000","last_undegraded":"2024-09-17T08:28:36.530711+0000","last_fullsized":"2024-09-17T08:28:36.530711+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:28:29.665385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.665385+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.448791+0000","scrub_duration":0.0073568490000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":426,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:41.281153+0000","last_change":"2024-09-17T08:28:41.281153+0000","last_active":"2024-09-17T08:28:41.281153+0000","last_peered":"2024-09-17T08:28:41.281153+0000","last_clean":"2024-09-17T08:28:41.281153+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:28:41.281153+0000","last_undegraded":"2024-09-17T08:28:41.281153+0000","last_fullsized":"2024-09-17T08:28:41.281153+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:41.281073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:41.281073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.528289+0000","scrub_duration":0.001166683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":403,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529403+0000","last_change":"2024-09-17T08:28:09.842951+0000","last_active":"2024-09-17T08:28:36.529403+0000","last_peered":"2024-09-17T08:28:36.529403+0000","last_clean":"2024-09-17T08:28:36.529403+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:28:36.529403+0000","last_undegraded":"2024-09-17T08:28:36.529403+0000","last_fullsized":"2024-09-17T08:28:36.529403+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:09.842851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:28:09.842851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:13.788917+0000","scrub_duration":0.001617739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":528,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527732+0000","last_change":"2024-09-17T08:27:38.219718+0000","last_active":"2024-09-17T08:28:36.527732+0000","last_peered":"2024-09-17T08:28:36.527732+0000","last_clean":"2024-09-17T08:28:36.527732+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:28:36.527732+0000","last_undegraded":"2024-09-17T08:28:36.527732+0000","last_fullsized":"2024-09-17T08:28:36.527732+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:27:38.219617+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:27:38.219617+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021625220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521099+0000","last_change":"2024-09-17T08:28:16.352900+0000","last_active":"2024-09-17T08:28:36.521099+0000","last_peered":"2024-09-17T08:28:36.521099+0000","last_clean":"2024-09-17T08:28:36.521099+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:28:36.521099+0000","last_undegraded":"2024-09-17T08:28:36.521099+0000","last_fullsized":"2024-09-17T08:28:36.521099+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.352564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.352564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:26.336699+0000","scrub_duration":0.0029315639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":424,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527867+0000","last_change":"2024-09-17T08:28:16.236102+0000","last_active":"2024-09-17T08:28:36.527867+0000","last_peered":"2024-09-17T08:28:36.527867+0000","last_clean":"2024-09-17T08:28:36.527867+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:28:36.527867+0000","last_undegraded":"2024-09-17T08:28:36.527867+0000","last_fullsized":"2024-09-17T08:28:36.527867+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.235973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.235973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:22.183028+0000","scrub_duration":0.003218683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":422,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:38.384894+0000","last_change":"2024-09-17T08:28:38.384894+0000","last_active":"2024-09-17T08:28:38.384894+0000","last_peered":"2024-09-17T08:28:38.384894+0000","last_clean":"2024-09-17T08:28:38.384894+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:28:38.384894+0000","last_undegraded":"2024-09-17T08:28:38.384894+0000","last_fullsized":"2024-09-17T08:28:38.384894+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:38.384792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:28:38.384792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:48.679670+0000","scrub_duration":0.0022556690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":444,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527920+0000","last_change":"2024-09-17T08:27:17.205107+0000","last_active":"2024-09-17T08:28:36.527920+0000","last_peered":"2024-09-17T08:28:36.527920+0000","last_clean":"2024-09-17T08:28:36.527920+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:28:36.527920+0000","last_undegraded":"2024-09-17T08:28:36.527920+0000","last_fullsized":"2024-09-17T08:28:36.527920+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:27:17.205020+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:27:17.205020+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040914250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":447,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525170+0000","last_change":"2024-09-17T08:28:30.443367+0000","last_active":"2024-09-17T08:28:36.525170+0000","last_peered":"2024-09-17T08:28:36.525170+0000","last_clean":"2024-09-17T08:28:36.525170+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:28:36.525170+0000","last_undegraded":"2024-09-17T08:28:36.525170+0000","last_fullsized":"2024-09-17T08:28:36.525170+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:30.443251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:28:30.443251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:36.240388+0000","scrub_duration":0.0042606730000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.520946+0000","last_change":"2024-09-17T08:27:08.306361+0000","last_active":"2024-09-17T08:28:36.520946+0000","last_peered":"2024-09-17T08:28:36.520946+0000","last_clean":"2024-09-17T08:28:36.520946+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:28:36.520946+0000","last_undegraded":"2024-09-17T08:28:36.520946+0000","last_fullsized":"2024-09-17T08:28:36.520946+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:08.306241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:08.306241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035413850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":437,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529431+0000","last_change":"2024-09-17T08:28:29.478103+0000","last_active":"2024-09-17T08:28:36.529431+0000","last_peered":"2024-09-17T08:28:36.529431+0000","last_clean":"2024-09-17T08:28:36.529431+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:28:36.529431+0000","last_undegraded":"2024-09-17T08:28:36.529431+0000","last_fullsized":"2024-09-17T08:28:36.529431+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.477933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.477933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:41.041706+0000","scrub_duration":0.0027795519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":439,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528124+0000","last_change":"2024-09-17T08:28:11.338708+0000","last_active":"2024-09-17T08:28:36.528124+0000","last_peered":"2024-09-17T08:28:36.528124+0000","last_clean":"2024-09-17T08:28:36.528124+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:28:36.528124+0000","last_undegraded":"2024-09-17T08:28:36.528124+0000","last_fullsized":"2024-09-17T08:28:36.528124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:28:11.338595+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:28:11.338595+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:17.534512+0000","scrub_duration":0.0026129119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":311,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531327+0000","last_change":"2024-09-17T08:27:30.096764+0000","last_active":"2024-09-17T08:28:36.531327+0000","last_peered":"2024-09-17T08:28:36.531327+0000","last_clean":"2024-09-17T08:28:36.531327+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:28:36.531327+0000","last_undegraded":"2024-09-17T08:28:36.531327+0000","last_fullsized":"2024-09-17T08:28:36.531327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:30.096633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:27:30.096633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:56.989681+0000","scrub_duration":0.0048718219999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":424,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529846+0000","last_change":"2024-09-17T08:27:26.220147+0000","last_active":"2024-09-17T08:28:36.529846+0000","last_peered":"2024-09-17T08:28:36.529846+0000","last_clean":"2024-09-17T08:28:36.529846+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:28:36.529846+0000","last_undegraded":"2024-09-17T08:28:36.529846+0000","last_fullsized":"2024-09-17T08:28:36.529846+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:26.220022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:26.220022+0000","last_clean_scrub_stamp":"2024-09-17T08:27:26.220022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060144819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"371'3335","reported_seq":3356,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:41.802757+0000","last_change":"2024-09-17T08:28:00.528932+0000","last_active":"2024-09-17T08:28:41.802757+0000","last_peered":"2024-09-17T08:28:41.802757+0000","last_clean":"2024-09-17T08:28:41.802757+0000","last_became_active":"2024-09-17T08:28:00.528334+0000","last_became_peered":"2024-09-17T08:28:00.528334+0000","last_unstale":"2024-09-17T08:28:41.802757+0000","last_undegraded":"2024-09-17T08:28:41.802757+0000","last_fullsized":"2024-09-17T08:28:41.802757+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3335,"log_dups_size":0,"ondisk_log_size":3335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:05.975160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218562560,"num_objects":3335,"num_object_clones":0,"num_object_copies":10005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3335,"num_write_kb":213440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'130","reported_seq":338,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.644616+0000","last_change":"2024-09-17T08:27:25.890590+0000","last_active":"2024-09-17T08:28:35.644616+0000","last_peered":"2024-09-17T08:28:35.644616+0000","last_clean":"2024-09-17T08:28:35.644616+0000","last_became_active":"2024-09-17T08:26:56.652585+0000","last_became_peered":"2024-09-17T08:26:56.652585+0000","last_unstale":"2024-09-17T08:28:35.644616+0000","last_undegraded":"2024-09-17T08:28:35.644616+0000","last_fullsized":"2024-09-17T08:28:35.644616+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:27:25.890470+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.890470+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:37.009189+0000","scrub_duration":0.0071256449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":535,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528407+0000","last_change":"2024-09-17T08:27:11.266259+0000","last_active":"2024-09-17T08:28:36.528407+0000","last_peered":"2024-09-17T08:28:36.528407+0000","last_clean":"2024-09-17T08:28:36.528407+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:28:36.528407+0000","last_undegraded":"2024-09-17T08:28:36.528407+0000","last_fullsized":"2024-09-17T08:28:36.528407+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:27:11.266120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.266120+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002802545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":423,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525826+0000","last_change":"2024-09-17T08:27:25.708183+0000","last_active":"2024-09-17T08:28:36.525826+0000","last_peered":"2024-09-17T08:28:36.525826+0000","last_clean":"2024-09-17T08:28:36.525826+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:28:36.525826+0000","last_undegraded":"2024-09-17T08:28:36.525826+0000","last_fullsized":"2024-09-17T08:28:36.525826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:25.708052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.708052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:54.296939+0000","scrub_duration":0.0051712390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":302,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521926+0000","last_change":"2024-09-17T08:28:26.403888+0000","last_active":"2024-09-17T08:28:36.521926+0000","last_peered":"2024-09-17T08:28:36.521926+0000","last_clean":"2024-09-17T08:28:36.521926+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:28:36.521926+0000","last_undegraded":"2024-09-17T08:28:36.521926+0000","last_fullsized":"2024-09-17T08:28:36.521926+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:26.403765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.403765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:54.017352+0000","scrub_duration":0.0072874840000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":401,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646389+0000","last_change":"2024-09-17T08:28:20.191865+0000","last_active":"2024-09-17T08:28:35.646389+0000","last_peered":"2024-09-17T08:28:35.646389+0000","last_clean":"2024-09-17T08:28:35.646389+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:28:35.646389+0000","last_undegraded":"2024-09-17T08:28:35.646389+0000","last_fullsized":"2024-09-17T08:28:35.646389+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:20.191734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.191734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:37.075731+0000","scrub_duration":0.0041900449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"371'3415","reported_seq":3436,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.493652+0000","last_change":"2024-09-17T08:28:00.528028+0000","last_active":"2024-09-17T08:28:42.493652+0000","last_peered":"2024-09-17T08:28:42.493652+0000","last_clean":"2024-09-17T08:28:42.493652+0000","last_became_active":"2024-09-17T08:28:00.527680+0000","last_became_peered":"2024-09-17T08:28:00.527680+0000","last_unstale":"2024-09-17T08:28:42.493652+0000","last_undegraded":"2024-09-17T08:28:42.493652+0000","last_fullsized":"2024-09-17T08:28:42.493652+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3415,"log_dups_size":0,"ondisk_log_size":3415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:00.051273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223805440,"num_objects":3415,"num_object_clones":0,"num_object_copies":10245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":218560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'130","reported_seq":712,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525257+0000","last_change":"2024-09-17T08:28:13.537716+0000","last_active":"2024-09-17T08:28:36.525257+0000","last_peered":"2024-09-17T08:28:36.525257+0000","last_clean":"2024-09-17T08:28:36.525257+0000","last_became_active":"2024-09-17T08:26:56.652142+0000","last_became_peered":"2024-09-17T08:26:56.652142+0000","last_unstale":"2024-09-17T08:28:36.525257+0000","last_undegraded":"2024-09-17T08:28:36.525257+0000","last_fullsized":"2024-09-17T08:28:36.525257+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:13.537623+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.537623+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:25.567254+0000","scrub_duration":0.0020024349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"367'25","reported_seq":446,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528071+0000","last_change":"2024-09-17T08:28:25.297611+0000","last_active":"2024-09-17T08:28:36.528071+0000","last_peered":"2024-09-17T08:28:36.528071+0000","last_clean":"2024-09-17T08:28:36.528071+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:28:36.528071+0000","last_undegraded":"2024-09-17T08:28:36.528071+0000","last_fullsized":"2024-09-17T08:28:36.528071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'25","last_scrub_stamp":"2024-09-17T08:28:25.297479+0000","last_deep_scrub":"367'25","last_deep_scrub_stamp":"2024-09-17T08:28:25.297479+0000","last_clean_scrub_stamp":"2024-09-17T08:28:25.297479+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:48.016454+0000","scrub_duration":0.0095790600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":292,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521677+0000","last_change":"2024-09-17T08:28:15.377573+0000","last_active":"2024-09-17T08:28:36.521677+0000","last_peered":"2024-09-17T08:28:36.521677+0000","last_clean":"2024-09-17T08:28:36.521677+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:28:36.521677+0000","last_undegraded":"2024-09-17T08:28:36.521677+0000","last_fullsized":"2024-09-17T08:28:36.521677+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:15.377472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:28:15.377472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:24.792663+0000","scrub_duration":0.001907822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":605,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:39.178273+0000","last_change":"2024-09-17T08:28:39.178481+0000","last_active":"2024-09-17T08:28:39.178273+0000","last_peered":"2024-09-17T08:28:39.178273+0000","last_clean":"2024-09-17T08:28:39.178273+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:28:39.178273+0000","last_undegraded":"2024-09-17T08:28:39.178273+0000","last_fullsized":"2024-09-17T08:28:39.178273+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:28:39.178203+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.178203+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:50.267479+0000","scrub_duration":0.010732917,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":430,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528166+0000","last_change":"2024-09-17T08:28:35.289687+0000","last_active":"2024-09-17T08:28:36.528166+0000","last_peered":"2024-09-17T08:28:36.528166+0000","last_clean":"2024-09-17T08:28:36.528166+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:28:36.528166+0000","last_undegraded":"2024-09-17T08:28:36.528166+0000","last_fullsized":"2024-09-17T08:28:36.528166+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.289539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.289539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:52.053304+0000","scrub_duration":0.0086312880000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"372'3544","reported_seq":3566,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.886256+0000","last_change":"2024-09-17T08:28:00.529640+0000","last_active":"2024-09-17T08:28:43.886256+0000","last_peered":"2024-09-17T08:28:43.886256+0000","last_clean":"2024-09-17T08:28:43.886256+0000","last_became_active":"2024-09-17T08:28:00.529365+0000","last_became_peered":"2024-09-17T08:28:00.529365+0000","last_unstale":"2024-09-17T08:28:43.886256+0000","last_undegraded":"2024-09-17T08:28:43.886256+0000","last_fullsized":"2024-09-17T08:28:43.886256+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3544,"log_dups_size":0,"ondisk_log_size":3544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:24.885996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232259584,"num_objects":3544,"num_object_clones":0,"num_object_copies":10632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3544,"num_write_kb":226816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"75'130","reported_seq":339,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.645858+0000","last_change":"2024-09-17T08:27:29.987905+0000","last_active":"2024-09-17T08:28:35.645858+0000","last_peered":"2024-09-17T08:28:35.645858+0000","last_clean":"2024-09-17T08:28:35.645858+0000","last_became_active":"2024-09-17T08:26:56.658579+0000","last_became_peered":"2024-09-17T08:26:56.658579+0000","last_unstale":"2024-09-17T08:28:35.645858+0000","last_undegraded":"2024-09-17T08:28:35.645858+0000","last_fullsized":"2024-09-17T08:28:35.645858+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:27:29.987762+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:27:29.987762+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:57.649088+0000","scrub_duration":0.0089000539999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":292,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.645808+0000","last_change":"2024-09-17T08:27:32.021301+0000","last_active":"2024-09-17T08:28:35.645808+0000","last_peered":"2024-09-17T08:28:35.645808+0000","last_clean":"2024-09-17T08:28:35.645808+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:28:35.645808+0000","last_undegraded":"2024-09-17T08:28:35.645808+0000","last_fullsized":"2024-09-17T08:28:35.645808+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:27:32.021109+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:27:32.021109+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:53.475489+0000","scrub_duration":0.021738328000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":292,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521027+0000","last_change":"2024-09-17T08:27:14.214573+0000","last_active":"2024-09-17T08:28:36.521027+0000","last_peered":"2024-09-17T08:28:36.521027+0000","last_clean":"2024-09-17T08:28:36.521027+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:28:36.521027+0000","last_undegraded":"2024-09-17T08:28:36.521027+0000","last_fullsized":"2024-09-17T08:28:36.521027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:14.214464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:14.214464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002397092,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":632,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:38.055493+0000","last_change":"2024-09-17T08:27:12.269298+0000","last_active":"2024-09-17T08:28:38.055493+0000","last_peered":"2024-09-17T08:28:38.055493+0000","last_clean":"2024-09-17T08:28:38.055493+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:28:38.055493+0000","last_undegraded":"2024-09-17T08:28:38.055493+0000","last_fullsized":"2024-09-17T08:28:38.055493+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:27:12.269195+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.269195+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017091408999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521284+0000","last_change":"2024-09-17T08:27:25.238121+0000","last_active":"2024-09-17T08:28:36.521284+0000","last_peered":"2024-09-17T08:28:36.521284+0000","last_clean":"2024-09-17T08:28:36.521284+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:28:36.521284+0000","last_undegraded":"2024-09-17T08:28:36.521284+0000","last_fullsized":"2024-09-17T08:28:36.521284+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:25.238008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.238008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.074631296999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"371'3396","reported_seq":3416,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.484029+0000","last_change":"2024-09-17T08:28:00.529767+0000","last_active":"2024-09-17T08:28:42.484029+0000","last_peered":"2024-09-17T08:28:42.484029+0000","last_clean":"2024-09-17T08:28:42.484029+0000","last_became_active":"2024-09-17T08:28:00.529491+0000","last_became_peered":"2024-09-17T08:28:00.529491+0000","last_unstale":"2024-09-17T08:28:42.484029+0000","last_undegraded":"2024-09-17T08:28:42.484029+0000","last_fullsized":"2024-09-17T08:28:42.484029+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3396,"log_dups_size":0,"ondisk_log_size":3396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:09.043432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222560256,"num_objects":3396,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3396,"num_write_kb":217344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":479,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528406+0000","last_change":"2024-09-17T08:27:24.227739+0000","last_active":"2024-09-17T08:28:36.528406+0000","last_peered":"2024-09-17T08:28:36.528406+0000","last_clean":"2024-09-17T08:28:36.528406+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:28:36.528406+0000","last_undegraded":"2024-09-17T08:28:36.528406+0000","last_fullsized":"2024-09-17T08:28:36.528406+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:27:24.227610+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:27:24.227610+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079276539999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":430,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525072+0000","last_change":"2024-09-17T08:28:18.569946+0000","last_active":"2024-09-17T08:28:36.525072+0000","last_peered":"2024-09-17T08:28:36.525072+0000","last_clean":"2024-09-17T08:28:36.525072+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:28:36.525072+0000","last_undegraded":"2024-09-17T08:28:36.525072+0000","last_fullsized":"2024-09-17T08:28:36.525072+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.569818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.569818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:27.913640+0000","scrub_duration":0.0054611859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":302,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528384+0000","last_change":"2024-09-17T08:27:21.239377+0000","last_active":"2024-09-17T08:28:36.528384+0000","last_peered":"2024-09-17T08:28:36.528384+0000","last_clean":"2024-09-17T08:28:36.528384+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:28:36.528384+0000","last_undegraded":"2024-09-17T08:28:36.528384+0000","last_fullsized":"2024-09-17T08:28:36.528384+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:21.239266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:27:21.239266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":396,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528481+0000","last_change":"2024-09-17T08:28:21.692398+0000","last_active":"2024-09-17T08:28:36.528481+0000","last_peered":"2024-09-17T08:28:36.528481+0000","last_clean":"2024-09-17T08:28:36.528481+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:28:36.528481+0000","last_undegraded":"2024-09-17T08:28:36.528481+0000","last_fullsized":"2024-09-17T08:28:36.528481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:21.692268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:46.540560+0000","scrub_duration":0.0071970940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"372'3480","reported_seq":3502,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.758142+0000","last_change":"2024-09-17T08:28:00.528378+0000","last_active":"2024-09-17T08:28:42.758142+0000","last_peered":"2024-09-17T08:28:42.758142+0000","last_clean":"2024-09-17T08:28:42.758142+0000","last_became_active":"2024-09-17T08:28:00.527934+0000","last_became_peered":"2024-09-17T08:28:00.527934+0000","last_unstale":"2024-09-17T08:28:42.758142+0000","last_undegraded":"2024-09-17T08:28:42.758142+0000","last_fullsized":"2024-09-17T08:28:42.758142+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3480,"log_dups_size":0,"ondisk_log_size":3480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:06.051050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228065280,"num_objects":3480,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":222720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'130","reported_seq":340,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528350+0000","last_change":"2024-09-17T08:28:22.668840+0000","last_active":"2024-09-17T08:28:36.528350+0000","last_peered":"2024-09-17T08:28:36.528350+0000","last_clean":"2024-09-17T08:28:36.528350+0000","last_became_active":"2024-09-17T08:26:56.649122+0000","last_became_peered":"2024-09-17T08:26:56.649122+0000","last_unstale":"2024-09-17T08:28:36.528350+0000","last_undegraded":"2024-09-17T08:28:36.528350+0000","last_fullsized":"2024-09-17T08:28:36.528350+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.668701+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:34.018166+0000","scrub_duration":0.0073890500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"349'35","reported_seq":450,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.548664+0000","last_change":"2024-09-17T08:27:39.245204+0000","last_active":"2024-09-17T08:28:42.548664+0000","last_peered":"2024-09-17T08:28:42.548664+0000","last_clean":"2024-09-17T08:28:42.548664+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:28:42.548664+0000","last_undegraded":"2024-09-17T08:28:42.548664+0000","last_fullsized":"2024-09-17T08:28:42.548664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'35","last_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_deep_scrub":"349'35","last_deep_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_clean_scrub_stamp":"2024-09-17T08:27:39.245084+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:58.454165+0000","scrub_duration":0.0057119750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":287,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.654125+0000","last_change":"2024-09-17T08:28:16.192010+0000","last_active":"2024-09-17T08:28:35.654125+0000","last_peered":"2024-09-17T08:28:35.654125+0000","last_clean":"2024-09-17T08:28:35.654125+0000","last_became_active":"2024-09-17T08:27:47.563576+0000","last_became_peered":"2024-09-17T08:27:47.563576+0000","last_unstale":"2024-09-17T08:28:35.654125+0000","last_undegraded":"2024-09-17T08:28:35.654125+0000","last_fullsized":"2024-09-17T08:28:35.654125+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.191909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.191909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:19.086248+0000","scrub_duration":0.002546974,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":665,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.553314+0000","last_change":"2024-09-17T08:27:27.616731+0000","last_active":"2024-09-17T08:28:42.553314+0000","last_peered":"2024-09-17T08:28:42.553314+0000","last_clean":"2024-09-17T08:28:42.553314+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:28:42.553314+0000","last_undegraded":"2024-09-17T08:28:42.553314+0000","last_fullsized":"2024-09-17T08:28:42.553314+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:27:27.616647+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:16:48.677913+0000","last_clean_scrub_stamp":"2024-09-17T08:27:27.616647+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00377065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":422,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525439+0000","last_change":"2024-09-17T08:27:21.725512+0000","last_active":"2024-09-17T08:28:36.525439+0000","last_peered":"2024-09-17T08:28:36.525439+0000","last_clean":"2024-09-17T08:28:36.525439+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:28:36.525439+0000","last_undegraded":"2024-09-17T08:28:36.525439+0000","last_fullsized":"2024-09-17T08:28:36.525439+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:21.725424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:27:21.725424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010298175999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"372'3434","reported_seq":3456,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.872620+0000","last_change":"2024-09-17T08:28:00.530369+0000","last_active":"2024-09-17T08:28:42.872620+0000","last_peered":"2024-09-17T08:28:42.872620+0000","last_clean":"2024-09-17T08:28:42.872620+0000","last_became_active":"2024-09-17T08:28:00.530036+0000","last_became_peered":"2024-09-17T08:28:00.530036+0000","last_unstale":"2024-09-17T08:28:42.872620+0000","last_undegraded":"2024-09-17T08:28:42.872620+0000","last_fullsized":"2024-09-17T08:28:42.872620+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3434,"log_dups_size":0,"ondisk_log_size":3434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:21.966619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225050624,"num_objects":3434,"num_object_clones":0,"num_object_copies":10302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3434,"num_write_kb":219776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'130","reported_seq":339,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527911+0000","last_change":"2024-09-17T08:28:26.291855+0000","last_active":"2024-09-17T08:28:36.527911+0000","last_peered":"2024-09-17T08:28:36.527911+0000","last_clean":"2024-09-17T08:28:36.527911+0000","last_became_active":"2024-09-17T08:26:56.649679+0000","last_became_peered":"2024-09-17T08:26:56.649679+0000","last_unstale":"2024-09-17T08:28:36.527911+0000","last_undegraded":"2024-09-17T08:28:36.527911+0000","last_fullsized":"2024-09-17T08:28:36.527911+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:26.291665+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.291665+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:49.869649+0000","scrub_duration":0.00173146,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":462,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529737+0000","last_change":"2024-09-17T08:28:26.667711+0000","last_active":"2024-09-17T08:28:36.529737+0000","last_peered":"2024-09-17T08:28:36.529737+0000","last_clean":"2024-09-17T08:28:36.529737+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:28:36.529737+0000","last_undegraded":"2024-09-17T08:28:36.529737+0000","last_fullsized":"2024-09-17T08:28:36.529737+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:28:26.666614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.666614+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:43.831111+0000","scrub_duration":0.013055815,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.522649+0000","last_change":"2024-09-17T08:27:20.184771+0000","last_active":"2024-09-17T08:28:36.522649+0000","last_peered":"2024-09-17T08:28:36.522649+0000","last_clean":"2024-09-17T08:28:36.522649+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:28:36.522649+0000","last_undegraded":"2024-09-17T08:28:36.522649+0000","last_fullsized":"2024-09-17T08:28:36.522649+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:20.184644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:37.106607+0000","last_clean_scrub_stamp":"2024-09-17T08:27:20.184644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053395379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":410,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.548333+0000","last_change":"2024-09-17T08:28:37.288508+0000","last_active":"2024-09-17T08:28:42.548333+0000","last_peered":"2024-09-17T08:28:42.548333+0000","last_clean":"2024-09-17T08:28:42.548333+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:28:42.548333+0000","last_undegraded":"2024-09-17T08:28:42.548333+0000","last_fullsized":"2024-09-17T08:28:42.548333+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:37.288379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:28:37.288379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:04.707593+0000","scrub_duration":0.0062446639999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":423,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.548636+0000","last_change":"2024-09-17T08:28:23.270750+0000","last_active":"2024-09-17T08:28:42.548636+0000","last_peered":"2024-09-17T08:28:42.548636+0000","last_clean":"2024-09-17T08:28:42.548636+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:28:42.548636+0000","last_undegraded":"2024-09-17T08:28:42.548636+0000","last_fullsized":"2024-09-17T08:28:42.548636+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:23.270616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:23.270616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:50.555236+0000","scrub_duration":0.0048781409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"372'3572","reported_seq":3594,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.894597+0000","last_change":"2024-09-17T08:28:00.523359+0000","last_active":"2024-09-17T08:28:43.894597+0000","last_peered":"2024-09-17T08:28:43.894597+0000","last_clean":"2024-09-17T08:28:43.894597+0000","last_became_active":"2024-09-17T08:28:00.523053+0000","last_became_peered":"2024-09-17T08:28:00.523053+0000","last_unstale":"2024-09-17T08:28:43.894597+0000","last_undegraded":"2024-09-17T08:28:43.894597+0000","last_fullsized":"2024-09-17T08:28:43.894597+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3572,"log_dups_size":0,"ondisk_log_size":3572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:15.194169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234094592,"num_objects":3572,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3572,"num_write_kb":228608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"75'130","reported_seq":321,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653527+0000","last_change":"2024-09-17T08:28:29.091771+0000","last_active":"2024-09-17T08:28:35.653527+0000","last_peered":"2024-09-17T08:28:35.653527+0000","last_clean":"2024-09-17T08:28:35.653527+0000","last_became_active":"2024-09-17T08:27:47.568726+0000","last_became_peered":"2024-09-17T08:27:47.568726+0000","last_unstale":"2024-09-17T08:28:35.653527+0000","last_undegraded":"2024-09-17T08:28:35.653527+0000","last_fullsized":"2024-09-17T08:28:35.653527+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:29.091638+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:29.091638+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.091638+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:31.272063+0000","scrub_duration":0.0083305140000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":474,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.554083+0000","last_change":"2024-09-17T08:28:42.048805+0000","last_active":"2024-09-17T08:28:42.554083+0000","last_peered":"2024-09-17T08:28:42.554083+0000","last_clean":"2024-09-17T08:28:42.554083+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:28:42.554083+0000","last_undegraded":"2024-09-17T08:28:42.554083+0000","last_fullsized":"2024-09-17T08:28:42.554083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:28:42.048628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:42.048628+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.248846+0000","scrub_duration":0.0076376279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":430,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.547420+0000","last_change":"2024-09-17T08:28:20.547961+0000","last_active":"2024-09-17T08:28:42.547420+0000","last_peered":"2024-09-17T08:28:42.547420+0000","last_clean":"2024-09-17T08:28:42.547420+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:28:42.547420+0000","last_undegraded":"2024-09-17T08:28:42.547420+0000","last_fullsized":"2024-09-17T08:28:42.547420+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:28:20.547814+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.547814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:43.359007+0000","scrub_duration":0.0021902169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":84,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.652426+0000","last_change":"2024-09-17T08:27:47.564603+0000","last_active":"2024-09-17T08:28:35.652426+0000","last_peered":"2024-09-17T08:28:35.652426+0000","last_clean":"2024-09-17T08:28:35.652426+0000","last_became_active":"2024-09-17T08:27:47.562839+0000","last_became_peered":"2024-09-17T08:27:47.562839+0000","last_unstale":"2024-09-17T08:28:35.652426+0000","last_undegraded":"2024-09-17T08:28:35.652426+0000","last_fullsized":"2024-09-17T08:28:35.652426+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:23.085249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:27:23.085249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:52.470140+0000","scrub_duration":0.014983679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":307,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521171+0000","last_change":"2024-09-17T08:26:36.218759+0000","last_active":"2024-09-17T08:28:36.521171+0000","last_peered":"2024-09-17T08:28:36.521171+0000","last_clean":"2024-09-17T08:28:36.521171+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:28:36.521171+0000","last_undegraded":"2024-09-17T08:28:36.521171+0000","last_fullsized":"2024-09-17T08:28:36.521171+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:36.218614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:36.218614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017513920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.d","version":"371'3346","reported_seq":3367,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:41.814555+0000","last_change":"2024-09-17T08:28:00.530132+0000","last_active":"2024-09-17T08:28:41.814555+0000","last_peered":"2024-09-17T08:28:41.814555+0000","last_clean":"2024-09-17T08:28:41.814555+0000","last_became_active":"2024-09-17T08:28:00.529861+0000","last_became_peered":"2024-09-17T08:28:00.529861+0000","last_unstale":"2024-09-17T08:28:41.814555+0000","last_undegraded":"2024-09-17T08:28:41.814555+0000","last_fullsized":"2024-09-17T08:28:41.814555+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3346,"log_dups_size":0,"ondisk_log_size":3346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:01.015722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219283456,"num_objects":3346,"num_object_clones":0,"num_object_copies":10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3346,"num_write_kb":214144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"75'130","reported_seq":321,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653725+0000","last_change":"2024-09-17T08:28:32.005811+0000","last_active":"2024-09-17T08:28:35.653725+0000","last_peered":"2024-09-17T08:28:35.653725+0000","last_clean":"2024-09-17T08:28:35.653725+0000","last_became_active":"2024-09-17T08:27:47.567040+0000","last_became_peered":"2024-09-17T08:27:47.567040+0000","last_unstale":"2024-09-17T08:28:35.653725+0000","last_undegraded":"2024-09-17T08:28:35.653725+0000","last_fullsized":"2024-09-17T08:28:35.653725+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:32.005689+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.005689+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:35.300667+0000","scrub_duration":0.0051363199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":324,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.520875+0000","last_change":"2024-09-17T08:27:12.259887+0000","last_active":"2024-09-17T08:28:36.520875+0000","last_peered":"2024-09-17T08:28:36.520875+0000","last_clean":"2024-09-17T08:28:36.520875+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:28:36.520875+0000","last_undegraded":"2024-09-17T08:28:36.520875+0000","last_fullsized":"2024-09-17T08:28:36.520875+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:27:12.259507+0000","last_deep_scrub":"156'15","last_deep_scrub_stamp":"2024-09-17T08:15:44.010126+0000","last_clean_scrub_stamp":"2024-09-17T08:27:12.259507+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011044918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":292,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.522147+0000","last_change":"2024-09-17T08:27:19.209183+0000","last_active":"2024-09-17T08:28:36.522147+0000","last_peered":"2024-09-17T08:28:36.522147+0000","last_clean":"2024-09-17T08:28:36.522147+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:28:36.522147+0000","last_undegraded":"2024-09-17T08:28:36.522147+0000","last_fullsized":"2024-09-17T08:28:36.522147+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:19.209066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:27:19.209066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003693712,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":643,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:38.056562+0000","last_change":"2024-09-17T08:26:22.193822+0000","last_active":"2024-09-17T08:28:38.056562+0000","last_peered":"2024-09-17T08:28:38.056562+0000","last_clean":"2024-09-17T08:28:38.056562+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:28:38.056562+0000","last_undegraded":"2024-09-17T08:28:38.056562+0000","last_fullsized":"2024-09-17T08:28:38.056562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:26:22.193701+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:26:22.193701+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030149489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":424,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529705+0000","last_change":"2024-09-17T08:28:28.040603+0000","last_active":"2024-09-17T08:28:36.529705+0000","last_peered":"2024-09-17T08:28:36.529705+0000","last_clean":"2024-09-17T08:28:36.529705+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:28:36.529705+0000","last_undegraded":"2024-09-17T08:28:36.529705+0000","last_fullsized":"2024-09-17T08:28:36.529705+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:28.040487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:28.040487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:44.473388+0000","scrub_duration":0.0047520069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"372'3627","reported_seq":3649,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.911796+0000","last_change":"2024-09-17T08:28:00.529238+0000","last_active":"2024-09-17T08:28:43.911796+0000","last_peered":"2024-09-17T08:28:43.911796+0000","last_clean":"2024-09-17T08:28:43.911796+0000","last_became_active":"2024-09-17T08:28:00.528278+0000","last_became_peered":"2024-09-17T08:28:00.528278+0000","last_unstale":"2024-09-17T08:28:43.911796+0000","last_undegraded":"2024-09-17T08:28:43.911796+0000","last_fullsized":"2024-09-17T08:28:43.911796+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3627,"log_dups_size":0,"ondisk_log_size":3627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:10.002347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237699072,"num_objects":3627,"num_object_clones":0,"num_object_copies":10881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3627,"num_write_kb":232128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"75'130","reported_seq":348,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529619+0000","last_change":"2024-09-17T08:28:31.043325+0000","last_active":"2024-09-17T08:28:36.529619+0000","last_peered":"2024-09-17T08:28:36.529619+0000","last_clean":"2024-09-17T08:28:36.529619+0000","last_became_active":"2024-09-17T08:26:56.663984+0000","last_became_peered":"2024-09-17T08:26:56.663984+0000","last_unstale":"2024-09-17T08:28:36.529619+0000","last_undegraded":"2024-09-17T08:28:36.529619+0000","last_fullsized":"2024-09-17T08:28:36.529619+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:31.042950+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:31.042950+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.825421+0000","scrub_duration":0.0016036220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":512,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.553651+0000","last_change":"2024-09-17T08:26:27.354029+0000","last_active":"2024-09-17T08:28:42.553651+0000","last_peered":"2024-09-17T08:28:42.553651+0000","last_clean":"2024-09-17T08:28:42.553651+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:28:42.553651+0000","last_undegraded":"2024-09-17T08:28:42.553651+0000","last_fullsized":"2024-09-17T08:28:42.553651+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:26:27.353856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:26:27.353856+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088356479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":303,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521038+0000","last_change":"2024-09-17T08:28:35.378069+0000","last_active":"2024-09-17T08:28:36.521038+0000","last_peered":"2024-09-17T08:28:36.521038+0000","last_clean":"2024-09-17T08:28:36.521038+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:28:36.521038+0000","last_undegraded":"2024-09-17T08:28:36.521038+0000","last_fullsized":"2024-09-17T08:28:36.521038+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.377950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.377950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.428689+0000","scrub_duration":0.004717578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":427,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.530704+0000","last_change":"2024-09-17T08:26:30.174985+0000","last_active":"2024-09-17T08:28:36.530704+0000","last_peered":"2024-09-17T08:28:36.530704+0000","last_clean":"2024-09-17T08:28:36.530704+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:28:36.530704+0000","last_undegraded":"2024-09-17T08:28:36.530704+0000","last_fullsized":"2024-09-17T08:28:36.530704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:30.174489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:26:30.174489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034970859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":407,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529185+0000","last_change":"2024-09-17T08:27:39.662703+0000","last_active":"2024-09-17T08:28:36.529185+0000","last_peered":"2024-09-17T08:28:36.529185+0000","last_clean":"2024-09-17T08:28:36.529185+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:28:36.529185+0000","last_undegraded":"2024-09-17T08:28:36.529185+0000","last_fullsized":"2024-09-17T08:28:36.529185+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:39.662623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:27:39.662623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:43.427820+0000","scrub_duration":0.0018704419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"372'3580","reported_seq":3602,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.913890+0000","last_change":"2024-09-17T08:28:00.633965+0000","last_active":"2024-09-17T08:28:43.913890+0000","last_peered":"2024-09-17T08:28:43.913890+0000","last_clean":"2024-09-17T08:28:43.913890+0000","last_became_active":"2024-09-17T08:28:00.633649+0000","last_became_peered":"2024-09-17T08:28:00.633649+0000","last_unstale":"2024-09-17T08:28:43.913890+0000","last_undegraded":"2024-09-17T08:28:43.913890+0000","last_fullsized":"2024-09-17T08:28:43.913890+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:22.761935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234618880,"num_objects":3580,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":229120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'130","reported_seq":712,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525023+0000","last_change":"2024-09-17T08:27:32.617618+0000","last_active":"2024-09-17T08:28:36.525023+0000","last_peered":"2024-09-17T08:28:36.525023+0000","last_clean":"2024-09-17T08:28:36.525023+0000","last_became_active":"2024-09-17T08:26:56.652768+0000","last_became_peered":"2024-09-17T08:26:56.652768+0000","last_unstale":"2024-09-17T08:28:36.525023+0000","last_undegraded":"2024-09-17T08:28:36.525023+0000","last_fullsized":"2024-09-17T08:28:36.525023+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:27:32.617541+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:27:32.617541+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:47.331805+0000","scrub_duration":0.001252778,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":310,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.523680+0000","last_change":"2024-09-17T08:27:17.210028+0000","last_active":"2024-09-17T08:28:36.523680+0000","last_peered":"2024-09-17T08:28:36.523680+0000","last_clean":"2024-09-17T08:28:36.523680+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:28:36.523680+0000","last_undegraded":"2024-09-17T08:28:36.523680+0000","last_fullsized":"2024-09-17T08:28:36.523680+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:27:17.209895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:17.209895+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088673099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":1102,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.545840+0000","last_change":"2024-09-17T08:27:10.306884+0000","last_active":"2024-09-17T08:28:42.545840+0000","last_peered":"2024-09-17T08:28:42.545840+0000","last_clean":"2024-09-17T08:28:42.545840+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:28:42.545840+0000","last_undegraded":"2024-09-17T08:28:42.545840+0000","last_fullsized":"2024-09-17T08:28:42.545840+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:27:10.306650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:10.306650+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002725903,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":790,"num_read_kb":785,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":397,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.532525+0000","last_change":"2024-09-17T08:28:11.119128+0000","last_active":"2024-09-17T08:28:36.532525+0000","last_peered":"2024-09-17T08:28:36.532525+0000","last_clean":"2024-09-17T08:28:36.532525+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:28:36.532525+0000","last_undegraded":"2024-09-17T08:28:36.532525+0000","last_fullsized":"2024-09-17T08:28:36.532525+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:11.119037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:28:11.119037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:11.718586+0000","scrub_duration":0.0013758959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":404,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646246+0000","last_change":"2024-09-17T08:28:18.226962+0000","last_active":"2024-09-17T08:28:35.646246+0000","last_peered":"2024-09-17T08:28:35.646246+0000","last_clean":"2024-09-17T08:28:35.646246+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:28:35.646246+0000","last_undegraded":"2024-09-17T08:28:35.646246+0000","last_fullsized":"2024-09-17T08:28:35.646246+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.226847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.226847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:24.304524+0000","scrub_duration":0.0026678819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":428,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525149+0000","last_change":"2024-09-17T08:28:33.423821+0000","last_active":"2024-09-17T08:28:36.525149+0000","last_peered":"2024-09-17T08:28:36.525149+0000","last_clean":"2024-09-17T08:28:36.525149+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:28:36.525149+0000","last_undegraded":"2024-09-17T08:28:36.525149+0000","last_fullsized":"2024-09-17T08:28:36.525149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:33.423708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:28:33.423708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.644830+0000","scrub_duration":0.0040275149999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"372'3551","reported_seq":3573,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.910379+0000","last_change":"2024-09-17T08:28:00.529084+0000","last_active":"2024-09-17T08:28:43.910379+0000","last_peered":"2024-09-17T08:28:43.910379+0000","last_clean":"2024-09-17T08:28:43.910379+0000","last_became_active":"2024-09-17T08:28:00.528828+0000","last_became_peered":"2024-09-17T08:28:00.528828+0000","last_unstale":"2024-09-17T08:28:43.910379+0000","last_undegraded":"2024-09-17T08:28:43.910379+0000","last_fullsized":"2024-09-17T08:28:43.910379+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:10.554877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232718336,"num_objects":3551,"num_object_clones":0,"num_object_copies":10653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3551,"num_write_kb":227264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"75'130","reported_seq":340,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521702+0000","last_change":"2024-09-17T08:28:13.346758+0000","last_active":"2024-09-17T08:28:36.521702+0000","last_peered":"2024-09-17T08:28:36.521702+0000","last_clean":"2024-09-17T08:28:36.521702+0000","last_became_active":"2024-09-17T08:26:56.645417+0000","last_became_peered":"2024-09-17T08:26:56.645417+0000","last_unstale":"2024-09-17T08:28:36.521702+0000","last_undegraded":"2024-09-17T08:28:36.521702+0000","last_fullsized":"2024-09-17T08:28:36.521702+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:13.346675+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.346675+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:39.661847+0000","scrub_duration":0.0018158759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":529,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.532062+0000","last_change":"2024-09-17T08:28:27.034085+0000","last_active":"2024-09-17T08:28:36.532062+0000","last_peered":"2024-09-17T08:28:36.532062+0000","last_clean":"2024-09-17T08:28:36.532062+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:28:36.532062+0000","last_undegraded":"2024-09-17T08:28:36.532062+0000","last_fullsized":"2024-09-17T08:28:36.532062+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:28:27.033942+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.033942+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:56.987816+0000","scrub_duration":0.0076002580000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":432,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.552867+0000","last_change":"2024-09-17T08:28:24.124112+0000","last_active":"2024-09-17T08:28:42.552867+0000","last_peered":"2024-09-17T08:28:42.552867+0000","last_clean":"2024-09-17T08:28:42.552867+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:28:42.552867+0000","last_undegraded":"2024-09-17T08:28:42.552867+0000","last_fullsized":"2024-09-17T08:28:42.552867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:28:24.123939+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.123939+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:38.699994+0000","scrub_duration":0.0056523010000000002,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":453,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525145+0000","last_change":"2024-09-17T08:28:35.399624+0000","last_active":"2024-09-17T08:28:36.525145+0000","last_peered":"2024-09-17T08:28:36.525145+0000","last_clean":"2024-09-17T08:28:36.525145+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:28:36.525145+0000","last_undegraded":"2024-09-17T08:28:36.525145+0000","last_fullsized":"2024-09-17T08:28:36.525145+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.399515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:54.568431+0000","scrub_duration":0.0043382050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":303,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.524230+0000","last_change":"2024-09-17T08:28:29.458578+0000","last_active":"2024-09-17T08:28:36.524230+0000","last_peered":"2024-09-17T08:28:36.524230+0000","last_clean":"2024-09-17T08:28:36.524230+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:28:36.524230+0000","last_undegraded":"2024-09-17T08:28:36.524230+0000","last_fullsized":"2024-09-17T08:28:36.524230+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.458289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.458289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:34.145566+0000","scrub_duration":0.0063383909999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":292,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.522003+0000","last_change":"2024-09-17T08:27:11.262169+0000","last_active":"2024-09-17T08:28:36.522003+0000","last_peered":"2024-09-17T08:28:36.522003+0000","last_clean":"2024-09-17T08:28:36.522003+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:28:36.522003+0000","last_undegraded":"2024-09-17T08:28:36.522003+0000","last_fullsized":"2024-09-17T08:28:36.522003+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:11.262054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.262054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036509699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"372'3555","reported_seq":3577,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.901539+0000","last_change":"2024-09-17T08:28:00.633997+0000","last_active":"2024-09-17T08:28:43.901539+0000","last_peered":"2024-09-17T08:28:43.901539+0000","last_clean":"2024-09-17T08:28:43.901539+0000","last_became_active":"2024-09-17T08:28:00.633745+0000","last_became_peered":"2024-09-17T08:28:00.633745+0000","last_unstale":"2024-09-17T08:28:43.901539+0000","last_undegraded":"2024-09-17T08:28:43.901539+0000","last_fullsized":"2024-09-17T08:28:43.901539+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3555,"log_dups_size":0,"ondisk_log_size":3555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:00.427944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232980480,"num_objects":3555,"num_object_clones":0,"num_object_copies":10665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3555,"num_write_kb":227520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":252,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.654497+0000","last_change":"2024-09-17T08:28:35.037720+0000","last_active":"2024-09-17T08:28:35.654497+0000","last_peered":"2024-09-17T08:28:35.654497+0000","last_clean":"2024-09-17T08:28:35.654497+0000","last_became_active":"2024-09-17T08:27:47.562580+0000","last_became_peered":"2024-09-17T08:27:47.562580+0000","last_unstale":"2024-09-17T08:28:35.654497+0000","last_undegraded":"2024-09-17T08:28:35.654497+0000","last_fullsized":"2024-09-17T08:28:35.654497+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:28:35.037586+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.037586+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:44.844574+0000","scrub_duration":0.003341709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":421,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.552914+0000","last_change":"2024-09-17T08:28:35.737604+0000","last_active":"2024-09-17T08:28:42.552914+0000","last_peered":"2024-09-17T08:28:42.552914+0000","last_clean":"2024-09-17T08:28:42.552914+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:28:42.552914+0000","last_undegraded":"2024-09-17T08:28:42.552914+0000","last_fullsized":"2024-09-17T08:28:42.552914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:28:35.737203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.737203+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:43.015265+0000","scrub_duration":0.0041867500000000004,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":295,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.532865+0000","last_change":"2024-09-17T08:27:32.944663+0000","last_active":"2024-09-17T08:28:36.532865+0000","last_peered":"2024-09-17T08:28:36.532865+0000","last_clean":"2024-09-17T08:28:36.532865+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:28:36.532865+0000","last_undegraded":"2024-09-17T08:28:36.532865+0000","last_fullsized":"2024-09-17T08:28:36.532865+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:32.944346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:27:32.944346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:50.790732+0000","scrub_duration":0.0025490460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":696,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525239+0000","last_change":"2024-09-17T08:27:31.657036+0000","last_active":"2024-09-17T08:28:36.525239+0000","last_peered":"2024-09-17T08:28:36.525239+0000","last_clean":"2024-09-17T08:28:36.525239+0000","last_became_active":"2024-09-17T08:26:51.613195+0000","last_became_peered":"2024-09-17T08:26:51.613195+0000","last_unstale":"2024-09-17T08:28:36.525239+0000","last_undegraded":"2024-09-17T08:28:36.525239+0000","last_fullsized":"2024-09-17T08:28:36.525239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:27:31.656716+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:27:31.656716+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056491689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":423,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529269+0000","last_change":"2024-09-17T08:28:14.537148+0000","last_active":"2024-09-17T08:28:36.529269+0000","last_peered":"2024-09-17T08:28:36.529269+0000","last_clean":"2024-09-17T08:28:36.529269+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:28:36.529269+0000","last_undegraded":"2024-09-17T08:28:36.529269+0000","last_fullsized":"2024-09-17T08:28:36.529269+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:14.537050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.537050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:39.635759+0000","scrub_duration":0.001583777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":312,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528356+0000","last_change":"2024-09-17T08:28:19.752472+0000","last_active":"2024-09-17T08:28:36.528356+0000","last_peered":"2024-09-17T08:28:36.528356+0000","last_clean":"2024-09-17T08:28:36.528356+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:28:36.528356+0000","last_undegraded":"2024-09-17T08:28:36.528356+0000","last_fullsized":"2024-09-17T08:28:36.528356+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:19.752090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.752090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:30.027631+0000","scrub_duration":0.024911651999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"372'3400","reported_seq":3422,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.841177+0000","last_change":"2024-09-17T08:28:00.634484+0000","last_active":"2024-09-17T08:28:42.841177+0000","last_peered":"2024-09-17T08:28:42.841177+0000","last_clean":"2024-09-17T08:28:42.841177+0000","last_became_active":"2024-09-17T08:28:00.634149+0000","last_became_peered":"2024-09-17T08:28:00.634149+0000","last_unstale":"2024-09-17T08:28:42.841177+0000","last_undegraded":"2024-09-17T08:28:42.841177+0000","last_fullsized":"2024-09-17T08:28:42.841177+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3400,"log_dups_size":0,"ondisk_log_size":3400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:26.965340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222822400,"num_objects":3400,"num_object_clones":0,"num_object_copies":10200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3400,"num_write_kb":217600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":243,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653182+0000","last_change":"2024-09-17T08:27:47.560762+0000","last_active":"2024-09-17T08:28:35.653182+0000","last_peered":"2024-09-17T08:28:35.653182+0000","last_clean":"2024-09-17T08:28:35.653182+0000","last_became_active":"2024-09-17T08:27:47.559010+0000","last_became_peered":"2024-09-17T08:27:47.559010+0000","last_unstale":"2024-09-17T08:28:35.653182+0000","last_undegraded":"2024-09-17T08:28:35.653182+0000","last_fullsized":"2024-09-17T08:28:35.653182+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:27:23.142735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:23.142735+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010428712999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":385,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.647103+0000","last_change":"2024-09-17T08:28:10.080401+0000","last_active":"2024-09-17T08:28:35.647103+0000","last_peered":"2024-09-17T08:28:35.647103+0000","last_clean":"2024-09-17T08:28:35.647103+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:28:35.647103+0000","last_undegraded":"2024-09-17T08:28:35.647103+0000","last_fullsized":"2024-09-17T08:28:35.647103+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:10.080312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:10.080312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:24.905750+0000","scrub_duration":0.0019214270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":402,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646227+0000","last_change":"2024-09-17T08:28:28.198962+0000","last_active":"2024-09-17T08:28:35.646227+0000","last_peered":"2024-09-17T08:28:35.646227+0000","last_clean":"2024-09-17T08:28:35.646227+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:28:35.646227+0000","last_undegraded":"2024-09-17T08:28:35.646227+0000","last_fullsized":"2024-09-17T08:28:35.646227+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_clean_scrub_stamp":"2024-09-17T08:28:28.198833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:41.960568+0000","scrub_duration":0.0035093979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":421,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525518+0000","last_change":"2024-09-17T08:28:27.487067+0000","last_active":"2024-09-17T08:28:36.525518+0000","last_peered":"2024-09-17T08:28:36.525518+0000","last_clean":"2024-09-17T08:28:36.525518+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:28:36.525518+0000","last_undegraded":"2024-09-17T08:28:36.525518+0000","last_fullsized":"2024-09-17T08:28:36.525518+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.486978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.486978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:54.540285+0000","scrub_duration":0.0022327200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.4","version":"372'3573","reported_seq":3595,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.874846+0000","last_change":"2024-09-17T08:28:00.530309+0000","last_active":"2024-09-17T08:28:42.874846+0000","last_peered":"2024-09-17T08:28:42.874846+0000","last_clean":"2024-09-17T08:28:42.874846+0000","last_became_active":"2024-09-17T08:28:00.530013+0000","last_became_peered":"2024-09-17T08:28:00.530013+0000","last_unstale":"2024-09-17T08:28:42.874846+0000","last_undegraded":"2024-09-17T08:28:42.874846+0000","last_fullsized":"2024-09-17T08:28:42.874846+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:04.456088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234160128,"num_objects":3573,"num_object_clones":0,"num_object_copies":10719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3573,"num_write_kb":228672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":474,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531018+0000","last_change":"2024-09-17T08:28:26.065898+0000","last_active":"2024-09-17T08:28:36.531018+0000","last_peered":"2024-09-17T08:28:36.531018+0000","last_clean":"2024-09-17T08:28:36.531018+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:28:36.531018+0000","last_undegraded":"2024-09-17T08:28:36.531018+0000","last_fullsized":"2024-09-17T08:28:36.531018+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:28:26.065739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.065739+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:33.025135+0000","scrub_duration":0.0032677489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":302,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521134+0000","last_change":"2024-09-17T08:28:19.462647+0000","last_active":"2024-09-17T08:28:36.521134+0000","last_peered":"2024-09-17T08:28:36.521134+0000","last_clean":"2024-09-17T08:28:36.521134+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:28:36.521134+0000","last_undegraded":"2024-09-17T08:28:36.521134+0000","last_fullsized":"2024-09-17T08:28:36.521134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:19.462533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.462533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:39.217719+0000","scrub_duration":0.0034847369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":405,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646304+0000","last_change":"2024-09-17T08:28:13.138380+0000","last_active":"2024-09-17T08:28:35.646304+0000","last_peered":"2024-09-17T08:28:35.646304+0000","last_clean":"2024-09-17T08:28:35.646304+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:28:35.646304+0000","last_undegraded":"2024-09-17T08:28:35.646304+0000","last_fullsized":"2024-09-17T08:28:35.646304+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.138294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.138294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:15.166819+0000","scrub_duration":0.001677385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":422,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.530477+0000","last_change":"2024-09-17T08:28:17.190360+0000","last_active":"2024-09-17T08:28:36.530477+0000","last_peered":"2024-09-17T08:28:36.530477+0000","last_clean":"2024-09-17T08:28:36.530477+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:28:36.530477+0000","last_undegraded":"2024-09-17T08:28:36.530477+0000","last_fullsized":"2024-09-17T08:28:36.530477+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:17.190228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:28:17.190228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:22.461039+0000","scrub_duration":0.003265783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"372'3561","reported_seq":3583,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.904974+0000","last_change":"2024-09-17T08:28:00.634479+0000","last_active":"2024-09-17T08:28:43.904974+0000","last_peered":"2024-09-17T08:28:43.904974+0000","last_clean":"2024-09-17T08:28:43.904974+0000","last_became_active":"2024-09-17T08:28:00.634232+0000","last_became_peered":"2024-09-17T08:28:00.634232+0000","last_unstale":"2024-09-17T08:28:43.904974+0000","last_undegraded":"2024-09-17T08:28:43.904974+0000","last_fullsized":"2024-09-17T08:28:43.904974+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3561,"log_dups_size":0,"ondisk_log_size":3561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:21.375722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233373696,"num_objects":3561,"num_object_clones":0,"num_object_copies":10683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3561,"num_write_kb":227904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":466,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.524924+0000","last_change":"2024-09-17T08:27:18.748152+0000","last_active":"2024-09-17T08:28:36.524924+0000","last_peered":"2024-09-17T08:28:36.524924+0000","last_clean":"2024-09-17T08:28:36.524924+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:28:36.524924+0000","last_undegraded":"2024-09-17T08:28:36.524924+0000","last_fullsized":"2024-09-17T08:28:36.524924+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:27:18.748042+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:27:18.748042+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032172910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":406,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646425+0000","last_change":"2024-09-17T08:27:22.847551+0000","last_active":"2024-09-17T08:28:35.646425+0000","last_peered":"2024-09-17T08:28:35.646425+0000","last_clean":"2024-09-17T08:28:35.646425+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:28:35.646425+0000","last_undegraded":"2024-09-17T08:28:35.646425+0000","last_fullsized":"2024-09-17T08:28:35.646425+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:22.847352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:22.847352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:39.802502+0000","scrub_duration":0.0062528000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":653,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:39.372979+0000","last_change":"2024-09-17T08:28:39.372979+0000","last_active":"2024-09-17T08:28:39.372979+0000","last_peered":"2024-09-17T08:28:39.372979+0000","last_clean":"2024-09-17T08:28:39.372979+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:28:39.372979+0000","last_undegraded":"2024-09-17T08:28:39.372979+0000","last_fullsized":"2024-09-17T08:28:39.372979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.372822+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.253276+0000","scrub_duration":0.003653173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":164,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.652389+0000","last_change":"2024-09-17T08:28:24.092358+0000","last_active":"2024-09-17T08:28:35.652389+0000","last_peered":"2024-09-17T08:28:35.652389+0000","last_clean":"2024-09-17T08:28:35.652389+0000","last_became_active":"2024-09-17T08:27:47.569625+0000","last_became_peered":"2024-09-17T08:27:47.569625+0000","last_unstale":"2024-09-17T08:28:35.652389+0000","last_undegraded":"2024-09-17T08:28:35.652389+0000","last_fullsized":"2024-09-17T08:28:35.652389+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:24.092273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.092273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:45.807424+0000","scrub_duration":0.0034287760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"372'3437","reported_seq":3458,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.597269+0000","last_change":"2024-09-17T08:28:00.529206+0000","last_active":"2024-09-17T08:28:42.597269+0000","last_peered":"2024-09-17T08:28:42.597269+0000","last_clean":"2024-09-17T08:28:42.597269+0000","last_became_active":"2024-09-17T08:28:00.528548+0000","last_became_peered":"2024-09-17T08:28:00.528548+0000","last_unstale":"2024-09-17T08:28:42.597269+0000","last_undegraded":"2024-09-17T08:28:42.597269+0000","last_fullsized":"2024-09-17T08:28:42.597269+0000","mapping_epoch":359,"log_start":"0'0","ondisk_log_start":"0'0","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":3437,"log_dups_size":0,"ondisk_log_size":3437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:27.749640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225247232,"num_objects":3437,"num_object_clones":0,"num_object_copies":10311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3437,"num_write_kb":219968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":485,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527779+0000","last_change":"2024-09-17T08:28:19.238927+0000","last_active":"2024-09-17T08:28:36.527779+0000","last_peered":"2024-09-17T08:28:36.527779+0000","last_clean":"2024-09-17T08:28:36.527779+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:28:36.527779+0000","last_undegraded":"2024-09-17T08:28:36.527779+0000","last_fullsized":"2024-09-17T08:28:36.527779+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:28:19.238796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.238796+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:36.596307+0000","scrub_duration":0.0084402430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":420,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528561+0000","last_change":"2024-09-17T08:27:22.244977+0000","last_active":"2024-09-17T08:28:36.528561+0000","last_peered":"2024-09-17T08:28:36.528561+0000","last_clean":"2024-09-17T08:28:36.528561+0000","last_became_active":"2024-09-17T08:09:54.495818+0000","last_became_peered":"2024-09-17T08:09:54.495818+0000","last_unstale":"2024-09-17T08:28:36.528561+0000","last_undegraded":"2024-09-17T08:28:36.528561+0000","last_fullsized":"2024-09-17T08:28:36.528561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:27:22.244827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:22.244827+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:50.038809+0000","scrub_duration":0.0052538849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":410,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.644741+0000","last_change":"2024-09-17T08:27:24.914413+0000","last_active":"2024-09-17T08:28:35.644741+0000","last_peered":"2024-09-17T08:28:35.644741+0000","last_clean":"2024-09-17T08:28:35.644741+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:28:35.644741+0000","last_undegraded":"2024-09-17T08:28:35.644741+0000","last_fullsized":"2024-09-17T08:28:35.644741+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:24.914223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:27:24.914223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:48.370966+0000","scrub_duration":0.0033394449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":225,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646150+0000","last_change":"2024-09-17T08:28:14.165430+0000","last_active":"2024-09-17T08:28:35.646150+0000","last_peered":"2024-09-17T08:28:35.646150+0000","last_clean":"2024-09-17T08:28:35.646150+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:28:35.646150+0000","last_undegraded":"2024-09-17T08:28:35.646150+0000","last_fullsized":"2024-09-17T08:28:35.646150+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:14.165350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.165350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:14.235128+0000","scrub_duration":0.0015954630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":439,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531994+0000","last_change":"2024-09-17T08:28:33.991107+0000","last_active":"2024-09-17T08:28:36.531994+0000","last_peered":"2024-09-17T08:28:36.531994+0000","last_clean":"2024-09-17T08:28:36.531994+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:28:36.531994+0000","last_undegraded":"2024-09-17T08:28:36.531994+0000","last_fullsized":"2024-09-17T08:28:36.531994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:33.990923+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:28:33.990923+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:40.451057+0000","scrub_duration":0.0037671190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":435,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646480+0000","last_change":"2024-09-17T08:28:27.202142+0000","last_active":"2024-09-17T08:28:35.646480+0000","last_peered":"2024-09-17T08:28:35.646480+0000","last_clean":"2024-09-17T08:28:35.646480+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:28:35.646480+0000","last_undegraded":"2024-09-17T08:28:35.646480+0000","last_fullsized":"2024-09-17T08:28:35.646480+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.202005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.202005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:35.501754+0000","scrub_duration":0.008746116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":415,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528570+0000","last_change":"2024-09-17T08:28:08.835470+0000","last_active":"2024-09-17T08:28:36.528570+0000","last_peered":"2024-09-17T08:28:36.528570+0000","last_clean":"2024-09-17T08:28:36.528570+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:28:36.528570+0000","last_undegraded":"2024-09-17T08:28:36.528570+0000","last_fullsized":"2024-09-17T08:28:36.528570+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:08.835384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:08.835384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:28.959577+0000","scrub_duration":0.0034811569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":397,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646268+0000","last_change":"2024-09-17T08:27:37.989773+0000","last_active":"2024-09-17T08:28:35.646268+0000","last_peered":"2024-09-17T08:28:35.646268+0000","last_clean":"2024-09-17T08:28:35.646268+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:28:35.646268+0000","last_undegraded":"2024-09-17T08:28:35.646268+0000","last_fullsized":"2024-09-17T08:28:35.646268+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:37.989692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:27:37.989692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:59.371890+0000","scrub_duration":0.001602554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":223,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.652834+0000","last_change":"2024-09-17T08:28:10.117649+0000","last_active":"2024-09-17T08:28:35.652834+0000","last_peered":"2024-09-17T08:28:35.652834+0000","last_clean":"2024-09-17T08:28:35.652834+0000","last_became_active":"2024-09-17T08:27:47.561372+0000","last_became_peered":"2024-09-17T08:27:47.561372+0000","last_unstale":"2024-09-17T08:28:35.652834+0000","last_undegraded":"2024-09-17T08:28:35.652834+0000","last_fullsized":"2024-09-17T08:28:35.652834+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:10.117557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:10.117557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:30.473578+0000","scrub_duration":0.0015163640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":462,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528762+0000","last_change":"2024-09-17T08:28:24.657204+0000","last_active":"2024-09-17T08:28:36.528762+0000","last_peered":"2024-09-17T08:28:36.528762+0000","last_clean":"2024-09-17T08:28:36.528762+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:28:36.528762+0000","last_undegraded":"2024-09-17T08:28:36.528762+0000","last_fullsized":"2024-09-17T08:28:36.528762+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:28:24.657080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.657080+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:47.805671+0000","scrub_duration":0.0093465079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":318,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529108+0000","last_change":"2024-09-17T08:27:28.616429+0000","last_active":"2024-09-17T08:28:36.529108+0000","last_peered":"2024-09-17T08:28:36.529108+0000","last_clean":"2024-09-17T08:28:36.529108+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:28:36.529108+0000","last_undegraded":"2024-09-17T08:28:36.529108+0000","last_fullsized":"2024-09-17T08:28:36.529108+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:28.616307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:28.616307+0000","last_clean_scrub_stamp":"2024-09-17T08:27:28.616307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:47.992642+0000","scrub_duration":0.0050860280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":427,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525022+0000","last_change":"2024-09-17T08:28:32.421385+0000","last_active":"2024-09-17T08:28:36.525022+0000","last_peered":"2024-09-17T08:28:36.525022+0000","last_clean":"2024-09-17T08:28:36.525022+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:28:36.525022+0000","last_undegraded":"2024-09-17T08:28:36.525022+0000","last_fullsized":"2024-09-17T08:28:36.525022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:32.421272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.421272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:56.260584+0000","scrub_duration":0.0016128049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":472,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.555530+0000","last_change":"2024-09-17T08:27:15.719861+0000","last_active":"2024-09-17T08:28:42.555530+0000","last_peered":"2024-09-17T08:28:42.555530+0000","last_clean":"2024-09-17T08:28:42.555530+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:28:42.555530+0000","last_undegraded":"2024-09-17T08:28:42.555530+0000","last_fullsized":"2024-09-17T08:28:42.555530+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:27:15.719197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:15.719197+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010440194,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":358,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528873+0000","last_change":"2024-09-17T08:28:25.643306+0000","last_active":"2024-09-17T08:28:36.528873+0000","last_peered":"2024-09-17T08:28:36.528873+0000","last_clean":"2024-09-17T08:28:36.528873+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:28:36.528873+0000","last_undegraded":"2024-09-17T08:28:36.528873+0000","last_fullsized":"2024-09-17T08:28:36.528873+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:25.643230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:28:25.643230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:25.948366+0000","scrub_duration":0.001106164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":456,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.553469+0000","last_change":"2024-09-17T08:27:13.210198+0000","last_active":"2024-09-17T08:28:42.553469+0000","last_peered":"2024-09-17T08:28:42.553469+0000","last_clean":"2024-09-17T08:28:42.553469+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:28:42.553469+0000","last_undegraded":"2024-09-17T08:28:42.553469+0000","last_fullsized":"2024-09-17T08:28:42.553469+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:27:13.210073+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:18:28.926645+0000","last_clean_scrub_stamp":"2024-09-17T08:27:13.210073+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025890539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":406,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531490+0000","last_change":"2024-09-17T08:27:25.259320+0000","last_active":"2024-09-17T08:28:36.531490+0000","last_peered":"2024-09-17T08:28:36.531490+0000","last_clean":"2024-09-17T08:28:36.531490+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:28:36.531490+0000","last_undegraded":"2024-09-17T08:28:36.531490+0000","last_fullsized":"2024-09-17T08:28:36.531490+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:25.259208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.259208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021771546999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":363,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.525865+0000","last_change":"2024-09-17T08:28:15.581250+0000","last_active":"2024-09-17T08:28:36.525865+0000","last_peered":"2024-09-17T08:28:36.525865+0000","last_clean":"2024-09-17T08:28:36.525865+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:28:36.525865+0000","last_undegraded":"2024-09-17T08:28:36.525865+0000","last_fullsized":"2024-09-17T08:28:36.525865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:15.581112+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:28:15.581112+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:37.078728+0000","scrub_duration":0.0044324229999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":413,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.554201+0000","last_change":"2024-09-17T08:27:37.220337+0000","last_active":"2024-09-17T08:28:42.554201+0000","last_peered":"2024-09-17T08:28:42.554201+0000","last_clean":"2024-09-17T08:28:42.554201+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:28:42.554201+0000","last_undegraded":"2024-09-17T08:28:42.554201+0000","last_fullsized":"2024-09-17T08:28:42.554201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:37.220206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:27:37.220206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:56.471062+0000","scrub_duration":0.0076403579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":650,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:43.055561+0000","last_change":"2024-09-17T08:28:14.817503+0000","last_active":"2024-09-17T08:28:43.055561+0000","last_peered":"2024-09-17T08:28:43.055561+0000","last_clean":"2024-09-17T08:28:43.055561+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:28:43.055561+0000","last_undegraded":"2024-09-17T08:28:43.055561+0000","last_fullsized":"2024-09-17T08:28:43.055561+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:28:14.817416+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.817416+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:37.671603+0000","scrub_duration":0.001610317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":411,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:38.263488+0000","last_change":"2024-09-17T08:28:38.263488+0000","last_active":"2024-09-17T08:28:38.263488+0000","last_peered":"2024-09-17T08:28:38.263488+0000","last_clean":"2024-09-17T08:28:38.263488+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:28:38.263488+0000","last_undegraded":"2024-09-17T08:28:38.263488+0000","last_fullsized":"2024-09-17T08:28:38.263488+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_clean_scrub_stamp":"2024-09-17T08:28:38.263303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:43.741822+0000","scrub_duration":0.0049197570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":323,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.520961+0000","last_change":"2024-09-17T08:27:07.276176+0000","last_active":"2024-09-17T08:28:36.520961+0000","last_peered":"2024-09-17T08:28:36.520961+0000","last_clean":"2024-09-17T08:28:36.520961+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:28:36.520961+0000","last_undegraded":"2024-09-17T08:28:36.520961+0000","last_fullsized":"2024-09-17T08:28:36.520961+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:27:07.275632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:07.275632+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073795509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":194,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653902+0000","last_change":"2024-09-17T08:27:47.565182+0000","last_active":"2024-09-17T08:28:35.653902+0000","last_peered":"2024-09-17T08:28:35.653902+0000","last_clean":"2024-09-17T08:28:35.653902+0000","last_became_active":"2024-09-17T08:27:47.564870+0000","last_became_peered":"2024-09-17T08:27:47.564870+0000","last_unstale":"2024-09-17T08:28:35.653902+0000","last_undegraded":"2024-09-17T08:28:35.653902+0000","last_fullsized":"2024-09-17T08:28:35.653902+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:21.683096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:21.683096+0000","last_clean_scrub_stamp":"2024-09-17T08:27:21.683096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:49.734532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":416,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.552990+0000","last_change":"2024-09-17T08:27:26.647218+0000","last_active":"2024-09-17T08:28:42.552990+0000","last_peered":"2024-09-17T08:28:42.552990+0000","last_clean":"2024-09-17T08:28:42.552990+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:28:42.552990+0000","last_undegraded":"2024-09-17T08:28:42.552990+0000","last_fullsized":"2024-09-17T08:28:42.552990+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:26.646984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:27:26.646984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:56.504571+0000","scrub_duration":0.001558357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":425,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.551997+0000","last_change":"2024-09-17T08:27:16.007484+0000","last_active":"2024-09-17T08:28:42.551997+0000","last_peered":"2024-09-17T08:28:42.551997+0000","last_clean":"2024-09-17T08:28:42.551997+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:28:42.551997+0000","last_undegraded":"2024-09-17T08:28:42.551997+0000","last_fullsized":"2024-09-17T08:28:42.551997+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:16.007026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:16.007026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076085880000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":485,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.548523+0000","last_change":"2024-09-17T08:28:22.312962+0000","last_active":"2024-09-17T08:28:42.548523+0000","last_peered":"2024-09-17T08:28:42.548523+0000","last_clean":"2024-09-17T08:28:42.548523+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:28:42.548523+0000","last_undegraded":"2024-09-17T08:28:42.548523+0000","last_fullsized":"2024-09-17T08:28:42.548523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:22.312822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.312822+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:26.968691+0000","scrub_duration":0.0031592030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521498+0000","last_change":"2024-09-17T08:28:18.412470+0000","last_active":"2024-09-17T08:28:36.521498+0000","last_peered":"2024-09-17T08:28:36.521498+0000","last_clean":"2024-09-17T08:28:36.521498+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:28:36.521498+0000","last_undegraded":"2024-09-17T08:28:36.521498+0000","last_fullsized":"2024-09-17T08:28:36.521498+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.412336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.412336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:37.050269+0000","scrub_duration":0.0034612430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521584+0000","last_change":"2024-09-17T08:28:30.480847+0000","last_active":"2024-09-17T08:28:36.521584+0000","last_peered":"2024-09-17T08:28:36.521584+0000","last_clean":"2024-09-17T08:28:36.521584+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:28:36.521584+0000","last_undegraded":"2024-09-17T08:28:36.521584+0000","last_fullsized":"2024-09-17T08:28:36.521584+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:30.480721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:30.480721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:43.464412+0000","scrub_duration":0.0099748570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":407,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.553632+0000","last_change":"2024-09-17T08:28:34.729325+0000","last_active":"2024-09-17T08:28:42.553632+0000","last_peered":"2024-09-17T08:28:42.553632+0000","last_clean":"2024-09-17T08:28:42.553632+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:28:42.553632+0000","last_undegraded":"2024-09-17T08:28:42.553632+0000","last_fullsized":"2024-09-17T08:28:42.553632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:34.725717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:34.725717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:35.020216+0000","scrub_duration":0.0070493120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":124,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527585+0000","last_change":"2024-09-17T08:27:20.254549+0000","last_active":"2024-09-17T08:28:36.527585+0000","last_peered":"2024-09-17T08:28:36.527585+0000","last_clean":"2024-09-17T08:28:36.527585+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:28:36.527585+0000","last_undegraded":"2024-09-17T08:28:36.527585+0000","last_fullsized":"2024-09-17T08:28:36.527585+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:27:20.254413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:20.254413+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032240540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":153,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653592+0000","last_change":"2024-09-17T08:27:47.566906+0000","last_active":"2024-09-17T08:28:35.653592+0000","last_peered":"2024-09-17T08:28:35.653592+0000","last_clean":"2024-09-17T08:28:35.653592+0000","last_became_active":"2024-09-17T08:27:47.566307+0000","last_became_peered":"2024-09-17T08:27:47.566307+0000","last_unstale":"2024-09-17T08:28:35.653592+0000","last_undegraded":"2024-09-17T08:28:35.653592+0000","last_fullsized":"2024-09-17T08:28:35.653592+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:05.718134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:05.718134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":205,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.652607+0000","last_change":"2024-09-17T08:28:13.150686+0000","last_active":"2024-09-17T08:28:35.652607+0000","last_peered":"2024-09-17T08:28:35.652607+0000","last_clean":"2024-09-17T08:28:35.652607+0000","last_became_active":"2024-09-17T08:27:47.561205+0000","last_became_peered":"2024-09-17T08:27:47.561205+0000","last_unstale":"2024-09-17T08:28:35.652607+0000","last_undegraded":"2024-09-17T08:28:35.652607+0000","last_fullsized":"2024-09-17T08:28:35.652607+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.150568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.150568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:40.848318+0000","scrub_duration":0.0023719209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":432,"reported_epoch":372,"state":"active+clean","last_fresh":"2024-09-17T08:28:42.546242+0000","last_change":"2024-09-17T08:28:36.410539+0000","last_active":"2024-09-17T08:28:42.546242+0000","last_peered":"2024-09-17T08:28:42.546242+0000","last_clean":"2024-09-17T08:28:42.546242+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:28:42.546242+0000","last_undegraded":"2024-09-17T08:28:42.546242+0000","last_fullsized":"2024-09-17T08:28:42.546242+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:36.410466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:28:36.410466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:37.169681+0000","scrub_duration":0.0036944650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":431,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:38.911426+0000","last_change":"2024-09-17T08:27:23.623195+0000","last_active":"2024-09-17T08:28:38.911426+0000","last_peered":"2024-09-17T08:28:38.911426+0000","last_clean":"2024-09-17T08:28:38.911426+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:28:38.911426+0000","last_undegraded":"2024-09-17T08:28:38.911426+0000","last_fullsized":"2024-09-17T08:28:38.911426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:27:23.623056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:23.623056+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037634259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":226,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:39.133600+0000","last_change":"2024-09-17T08:28:39.133714+0000","last_active":"2024-09-17T08:28:39.133600+0000","last_peered":"2024-09-17T08:28:39.133600+0000","last_clean":"2024-09-17T08:28:39.133600+0000","last_became_active":"2024-09-17T08:27:47.564429+0000","last_became_peered":"2024-09-17T08:27:47.564429+0000","last_unstale":"2024-09-17T08:28:39.133600+0000","last_undegraded":"2024-09-17T08:28:39.133600+0000","last_fullsized":"2024-09-17T08:28:39.133600+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:28:39.133574+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.133574+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.000527+0000","scrub_duration":0.0064456499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":419,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:37.873786+0000","last_change":"2024-09-17T08:28:22.551519+0000","last_active":"2024-09-17T08:28:37.873786+0000","last_peered":"2024-09-17T08:28:37.873786+0000","last_clean":"2024-09-17T08:28:37.873786+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:28:37.873786+0000","last_undegraded":"2024-09-17T08:28:37.873786+0000","last_fullsized":"2024-09-17T08:28:37.873786+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:22.551401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.551401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:48.359744+0000","scrub_duration":0.0030279500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":395,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528727+0000","last_change":"2024-09-17T08:27:29.641693+0000","last_active":"2024-09-17T08:28:36.528727+0000","last_peered":"2024-09-17T08:28:36.528727+0000","last_clean":"2024-09-17T08:28:36.528727+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:28:36.528727+0000","last_undegraded":"2024-09-17T08:28:36.528727+0000","last_fullsized":"2024-09-17T08:28:36.528727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_clean_scrub_stamp":"2024-09-17T08:27:29.641570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:59.051804+0000","scrub_duration":0.0060037839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":422,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529282+0000","last_change":"2024-09-17T08:27:10.716378+0000","last_active":"2024-09-17T08:28:36.529282+0000","last_peered":"2024-09-17T08:28:36.529282+0000","last_clean":"2024-09-17T08:28:36.529282+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:28:36.529282+0000","last_undegraded":"2024-09-17T08:28:36.529282+0000","last_fullsized":"2024-09-17T08:28:36.529282+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:27:10.716109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:10.716109+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010987006000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":312,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.527736+0000","last_change":"2024-09-17T08:28:27.267072+0000","last_active":"2024-09-17T08:28:36.527736+0000","last_peered":"2024-09-17T08:28:36.527736+0000","last_clean":"2024-09-17T08:28:36.527736+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:28:36.527736+0000","last_undegraded":"2024-09-17T08:28:36.527736+0000","last_fullsized":"2024-09-17T08:28:36.527736+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.266982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.266982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:54.026350+0000","scrub_duration":0.0011529369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":442,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531922+0000","last_change":"2024-09-17T08:27:11.144999+0000","last_active":"2024-09-17T08:28:36.531922+0000","last_peered":"2024-09-17T08:28:36.531922+0000","last_clean":"2024-09-17T08:28:36.531922+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:28:36.531922+0000","last_undegraded":"2024-09-17T08:28:36.531922+0000","last_fullsized":"2024-09-17T08:28:36.531922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:27:11.144705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.144705+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013738378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"353'10","reported_seq":162,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.522981+0000","last_change":"2024-09-17T08:28:20.143550+0000","last_active":"2024-09-17T08:28:36.522981+0000","last_peered":"2024-09-17T08:28:36.522981+0000","last_clean":"2024-09-17T08:28:36.522981+0000","last_became_active":"2024-09-17T08:27:47.561825+0000","last_became_peered":"2024-09-17T08:27:47.561825+0000","last_unstale":"2024-09-17T08:28:36.522981+0000","last_undegraded":"2024-09-17T08:28:36.522981+0000","last_fullsized":"2024-09-17T08:28:36.522981+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'10","last_scrub_stamp":"2024-09-17T08:28:20.143416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.143416+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:41.894700+0000","scrub_duration":0.0035354610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521212+0000","last_change":"2024-09-17T08:26:21.302147+0000","last_active":"2024-09-17T08:28:36.521212+0000","last_peered":"2024-09-17T08:28:36.521212+0000","last_clean":"2024-09-17T08:28:36.521212+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:28:36.521212+0000","last_undegraded":"2024-09-17T08:28:36.521212+0000","last_fullsized":"2024-09-17T08:28:36.521212+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:21.302035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:26:21.302035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014448862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":412,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.530536+0000","last_change":"2024-09-17T08:28:32.738220+0000","last_active":"2024-09-17T08:28:36.530536+0000","last_peered":"2024-09-17T08:28:36.530536+0000","last_clean":"2024-09-17T08:28:36.530536+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:28:36.530536+0000","last_undegraded":"2024-09-17T08:28:36.530536+0000","last_fullsized":"2024-09-17T08:28:36.530536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:32.738028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.738028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:34.362601+0000","scrub_duration":0.014925677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":219,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653972+0000","last_change":"2024-09-17T08:28:14.189235+0000","last_active":"2024-09-17T08:28:35.653972+0000","last_peered":"2024-09-17T08:28:35.653972+0000","last_clean":"2024-09-17T08:28:35.653972+0000","last_became_active":"2024-09-17T08:27:47.563331+0000","last_became_peered":"2024-09-17T08:27:47.563331+0000","last_unstale":"2024-09-17T08:28:35.653972+0000","last_undegraded":"2024-09-17T08:28:35.653972+0000","last_fullsized":"2024-09-17T08:28:35.653972+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:14.189064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.189064+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:15.900020+0000","scrub_duration":0.0022061400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":427,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.531706+0000","last_change":"2024-09-17T08:26:16.157165+0000","last_active":"2024-09-17T08:28:36.531706+0000","last_peered":"2024-09-17T08:28:36.531706+0000","last_clean":"2024-09-17T08:28:36.531706+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:28:36.531706+0000","last_undegraded":"2024-09-17T08:28:36.531706+0000","last_fullsized":"2024-09-17T08:28:36.531706+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:26:16.157047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:26:16.157047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.052471214000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":397,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528234+0000","last_change":"2024-09-17T08:28:13.307062+0000","last_active":"2024-09-17T08:28:36.528234+0000","last_peered":"2024-09-17T08:28:36.528234+0000","last_clean":"2024-09-17T08:28:36.528234+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:28:36.528234+0000","last_undegraded":"2024-09-17T08:28:36.528234+0000","last_fullsized":"2024-09-17T08:28:36.528234+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.306936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.306936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:32.659700+0000","scrub_duration":0.001457446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":99,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.653292+0000","last_change":"2024-09-17T08:28:18.200418+0000","last_active":"2024-09-17T08:28:35.653292+0000","last_peered":"2024-09-17T08:28:35.653292+0000","last_clean":"2024-09-17T08:28:35.653292+0000","last_became_active":"2024-09-17T08:27:47.563398+0000","last_became_peered":"2024-09-17T08:27:47.563398+0000","last_unstale":"2024-09-17T08:28:35.653292+0000","last_undegraded":"2024-09-17T08:28:35.653292+0000","last_fullsized":"2024-09-17T08:28:35.653292+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.200302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.200302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:25.848650+0000","scrub_duration":0.003324605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":528,"reported_epoch":370,"state":"active+clean","last_fresh":"2024-09-17T08:28:35.646758+0000","last_change":"2024-09-17T08:28:19.204373+0000","last_active":"2024-09-17T08:28:35.646758+0000","last_peered":"2024-09-17T08:28:35.646758+0000","last_clean":"2024-09-17T08:28:35.646758+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:28:35.646758+0000","last_undegraded":"2024-09-17T08:28:35.646758+0000","last_fullsized":"2024-09-17T08:28:35.646758+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:28:19.204197+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.204197+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:46.846407+0000","scrub_duration":0.0095927449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":291,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.521350+0000","last_change":"2024-09-17T08:27:39.265326+0000","last_active":"2024-09-17T08:28:36.521350+0000","last_peered":"2024-09-17T08:28:36.521350+0000","last_clean":"2024-09-17T08:28:36.521350+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:28:36.521350+0000","last_undegraded":"2024-09-17T08:28:36.521350+0000","last_fullsized":"2024-09-17T08:28:36.521350+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:39.265233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:27:39.265233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:54.519631+0000","scrub_duration":0.0013430779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":396,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528495+0000","last_change":"2024-09-17T08:28:23.645255+0000","last_active":"2024-09-17T08:28:36.528495+0000","last_peered":"2024-09-17T08:28:36.528495+0000","last_clean":"2024-09-17T08:28:36.528495+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:28:36.528495+0000","last_undegraded":"2024-09-17T08:28:36.528495+0000","last_fullsized":"2024-09-17T08:28:36.528495+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:23.645106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:23.645106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:35.139662+0000","scrub_duration":0.0036194209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":412,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528984+0000","last_change":"2024-09-17T08:28:20.695325+0000","last_active":"2024-09-17T08:28:36.528984+0000","last_peered":"2024-09-17T08:28:36.528984+0000","last_clean":"2024-09-17T08:28:36.528984+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:28:36.528984+0000","last_undegraded":"2024-09-17T08:28:36.528984+0000","last_fullsized":"2024-09-17T08:28:36.528984+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:20.695195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.695195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:34.987372+0000","scrub_duration":0.0052696519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":484,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.529062+0000","last_change":"2024-09-17T08:28:31.708556+0000","last_active":"2024-09-17T08:28:36.529062+0000","last_peered":"2024-09-17T08:28:36.529062+0000","last_clean":"2024-09-17T08:28:36.529062+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:28:36.529062+0000","last_undegraded":"2024-09-17T08:28:36.529062+0000","last_fullsized":"2024-09-17T08:28:36.529062+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:28:31.708312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:31.708312+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.017871+0000","scrub_duration":0.0060376420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":424,"reported_epoch":371,"state":"active+clean","last_fresh":"2024-09-17T08:28:36.528022+0000","last_change":"2024-09-17T08:28:29.221115+0000","last_active":"2024-09-17T08:28:36.528022+0000","last_peered":"2024-09-17T08:28:36.528022+0000","last_clean":"2024-09-17T08:28:36.528022+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:28:36.528022+0000","last_undegraded":"2024-09-17T08:28:36.528022+0000","last_fullsized":"2024-09-17T08:28:36.528022+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.221035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:29.221035+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.221035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:40.779150+0000","scrub_duration":0.0014868539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3657302016,"num_objects":55806,"num_object_clones":0,"num_object_copies":167418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55806,"num_write_kb":3571584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10918887424,"data_stored":10918887424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55806,"ondisk_log_size":55806,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1120,"num_read_kb":877,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":790,"num_read_kb":785,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1430,"ondisk_log_size":1430,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070695,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1952936,"kb_used_data":1313952,"kb_used_omap":1,"kb_used_meta":638974,"kb_avail":91792216,"statfs":{"total":95995035648,"available":93995229184,"internally_reserved":0,"allocated":1345486848,"data_stored":1345005594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":654309835},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.196,"15min":1.5169999999999999},"min":{"1min":0.22700000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":2.4399999999999999,"5min":6.2460000000000004,"15min":62.018000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.1499999999999999,"15min":1.5600000000000001},"min":{"1min":0.40400000000000003,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":2.2400000000000002,"5min":9.4169999999999998,"15min":61.896999999999998},"last":0.252}]},{"osd":1,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.77500000000000002,"15min":1.4890000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":1.3089999999999999,"5min":5.8369999999999997,"15min":62.067},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.85099999999999998,"15min":1.4950000000000001},"min":{"1min":0.29199999999999998,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":2.863,"5min":5.5179999999999998,"15min":62.194000000000003},"last":2.1819999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:28:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.88200000000000001,"15min":0.92100000000000004},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2929999999999999,"5min":9.3109999999999999,"15min":9.3109999999999999},"last":2.4670000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.002,"15min":0.79400000000000004},"min":{"1min":0.36899999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.0649999999999999,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.8420000000000001,"15min":1.694},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":1.391,"5min":74.376000000000005,"15min":74.376000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.748,"5min":1.6879999999999999,"15min":1.75},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.27300000000000002},"max":{"1min":2.7120000000000002,"5min":75.183000000000007,"15min":75.183000000000007},"last":7.3380000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56599999999999995,"5min":4.3719999999999999,"15min":5.335},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.70099999999999996,"15min":0.64700000000000002},"min":{"1min":0.32400000000000001,"5min":0.25600000000000001,"15min":0.23899999999999999},"max":{"1min":0.85999999999999999,"5min":3.3580000000000001,"15min":5.9669999999999996},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.93500000000000005,"15min":0.89000000000000001},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":1.3420000000000001,"5min":34.436,"15min":34.436},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":1.014,"15min":0.88100000000000001},"min":{"1min":0.317,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":1.091,"5min":34.965000000000003,"15min":34.965000000000003},"last":0.315}]},{"osd":6,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.71899999999999997,"15min":0.80000000000000004},"min":{"1min":0.308,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":0.65700000000000003,"5min":4.194,"15min":7.2359999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.72499999999999998,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.69799999999999995,"5min":3.3010000000000002,"15min":5.9400000000000004},"last":0.59499999999999997}]}]},{"osd":6,"up_from":50,"seq":214748365036,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2278232,"kb_used_data":1733584,"kb_used_omap":1,"kb_used_meta":544638,"kb_avail":91466920,"statfs":{"total":95995035648,"available":93662126080,"internally_reserved":0,"allocated":1775190016,"data_stored":1774724406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":557709767},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:27:52 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":2.0339999999999998,"15min":2.5299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":6.4039999999999999,"5min":55.625,"15min":81.007000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.9470000000000001,"15min":2.5590000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":4.8159999999999998,"5min":55.597000000000001,"15min":80.310000000000002},"last":0.86799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:27:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.0589999999999999,"15min":1.7090000000000001},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.22500000000000001},"max":{"1min":6.3029999999999999,"5min":9.8789999999999996,"15min":80.813000000000002},"last":2.0059999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":1.7350000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.24299999999999999},"max":{"1min":3.8340000000000001,"5min":10.01,"15min":80.281000000000006},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":1.5269999999999999,"15min":1.113},"min":{"1min":0.32800000000000001,"5min":0.254,"15min":0.254},"max":{"1min":1.9419999999999999,"5min":70.451999999999998,"15min":70.451999999999998},"last":1.9419999999999999},{"interface":"front","average":{"1min":0.754,"5min":1.5980000000000001,"15min":1.0840000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.968,"5min":70.488,"15min":70.488},"last":1.968}]},{"osd":3,"last update":"Tue Sep 17 08:27:52 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":0.99399999999999999,"15min":1.9910000000000001},"min":{"1min":0.42199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.7450000000000001,"5min":9.9309999999999992,"15min":80.427000000000007},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.206,"5min":1.073,"15min":1.9410000000000001},"min":{"1min":0.30499999999999999,"5min":0.26900000000000002,"15min":0.26000000000000001},"max":{"1min":6.1580000000000004,"5min":11.859,"15min":80.379999999999995},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.93899999999999995,"15min":0.749},"min":{"1min":0.246,"5min":0.246,"15min":0.215},"max":{"1min":1.208,"5min":25.289999999999999,"15min":25.289999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0389999999999999,"15min":0.73199999999999998},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":10.984999999999999,"5min":25.184999999999999,"15min":25.184999999999999},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.73999999999999999,"15min":0.72599999999999998},"min":{"1min":0.28100000000000003,"5min":0.25600000000000001,"15min":0.245},"max":{"1min":0.93500000000000005,"5min":4.8570000000000002,"15min":6.1859999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26200000000000001},"max":{"1min":5.2240000000000002,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64600000000000002,"15min":0.71299999999999997},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":3.306,"5min":3.306,"15min":4.774},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.65500000000000003},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":1.3360000000000001,"5min":3.516,"15min":5.0599999999999996},"last":0.57399999999999995}]}]},{"osd":5,"up_from":41,"seq":176093659377,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1467424,"kb_used_data":858776,"kb_used_omap":1,"kb_used_meta":608638,"kb_avail":92277728,"statfs":{"total":95995035648,"available":94492393472,"internally_reserved":0,"allocated":879386624,"data_stored":878913315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":623245766},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":3.2490000000000001,"15min":2.528},"min":{"1min":0.23699999999999999,"5min":0.22,"15min":0.20300000000000001},"max":{"1min":3.524,"5min":64.218999999999994,"15min":65.941000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":3.2629999999999999,"15min":2.4670000000000001},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.218},"max":{"1min":2.5209999999999999,"5min":65.405000000000001,"15min":66.135000000000005},"last":0.29099999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:28:31 2024","interfaces":[{"interface":"back","average":{"1min":3.448,"5min":1.9650000000000001,"15min":1.77},"min":{"1min":0.27600000000000002,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":50.426000000000002,"5min":50.426000000000002,"15min":66.981999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":4.2439999999999998,"5min":2.1139999999999999,"15min":1.891},"min":{"1min":0.35799999999999998,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":50.744,"5min":50.744,"15min":66.316000000000003},"last":0.25800000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:28:39 2024","interfaces":[{"interface":"back","average":{"1min":3.419,"5min":2.5150000000000001,"15min":1.7969999999999999},"min":{"1min":0.373,"5min":0.221,"15min":0.221},"max":{"1min":50.518000000000001,"5min":64.177999999999997,"15min":131.727},"last":0.83899999999999997},{"interface":"front","average":{"1min":3.246,"5min":2.7229999999999999,"15min":2.0489999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.27200000000000002},"max":{"1min":50.204000000000001,"5min":64.879000000000005,"15min":131.50399999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:28:31 2024","interfaces":[{"interface":"back","average":{"1min":4.343,"5min":2.2589999999999999,"15min":2.1970000000000001},"min":{"1min":0.27600000000000002,"5min":0.26200000000000001,"15min":0.253},"max":{"1min":50.450000000000003,"5min":50.450000000000003,"15min":75.286000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":4.351,"5min":2.181,"15min":2.1509999999999998},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.27100000000000002},"max":{"1min":50.469000000000001,"5min":50.469000000000001,"15min":77.206999999999994},"last":1.1799999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.88900000000000001,"15min":0.85899999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":1.6619999999999999,"5min":6.3600000000000003,"15min":42.421999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.79700000000000004,"15min":0.82699999999999996},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.24399999999999999},"max":{"1min":0.66400000000000003,"5min":6.0899999999999999,"15min":42.191000000000003},"last":0.36699999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:27:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.82299999999999995,"15min":0.70399999999999996},"min":{"1min":0.30199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.7519999999999998,"5min":7.8399999999999999,"15min":7.8399999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.88900000000000001,"15min":0.73699999999999999},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.26700000000000002},"max":{"1min":2.6970000000000001,"5min":5.883,"15min":5.883},"last":0.318}]},{"osd":7,"last update":"Tue Sep 17 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.88100000000000001,"15min":0.81399999999999995},"min":{"1min":0.375,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":6.9470000000000001,"15min":7.5270000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.82099999999999995,"15min":0.76600000000000001},"min":{"1min":0.41999999999999998,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":1.4299999999999999,"5min":5.4980000000000002,"15min":5.4980000000000002},"last":0.41899999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953718,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1387752,"kb_used_data":873440,"kb_used_omap":1,"kb_used_meta":514302,"kb_avail":92357400,"statfs":{"total":95995035648,"available":94573977600,"internally_reserved":0,"allocated":894402560,"data_stored":893921144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":526645702},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.1779999999999999,"15min":1.1200000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.19600000000000001},"max":{"1min":3.3439999999999999,"5min":20.271000000000001,"15min":23.786000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.2330000000000001,"15min":1.1799999999999999},"min":{"1min":0.26700000000000002,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":3.0710000000000002,"5min":19.798999999999999,"15min":23.245000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Sep 17 08:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":1.21},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.23200000000000001},"max":{"1min":2.7629999999999999,"5min":5.5839999999999996,"15min":23.135000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.90300000000000002,"15min":1.2170000000000001},"min":{"1min":0.32100000000000001,"5min":0.25800000000000001,"15min":0.24099999999999999},"max":{"1min":2.6629999999999998,"5min":8.1959999999999997,"15min":23.434999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:28:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.97899999999999998,"15min":0.73299999999999998},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":3.403,"5min":3.9910000000000001,"15min":3.9910000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.76300000000000001},"min":{"1min":0.36799999999999999,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":2.117}]},{"osd":3,"last update":"Tue Sep 17 08:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.127,"15min":1.988},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":2.2789999999999999,"5min":28.138000000000002,"15min":154.137},"last":0.443},{"interface":"front","average":{"1min":0.998,"5min":1.3360000000000001,"15min":1.9370000000000001},"min":{"1min":0.33100000000000002,"5min":0.313,"15min":0.26700000000000002},"max":{"1min":2.427,"5min":28.117000000000001,"15min":153.54400000000001},"last":0.433}]},{"osd":5,"last update":"Tue Sep 17 08:28:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.255,"5min":0.255,"15min":0.23000000000000001},"max":{"1min":3.3799999999999999,"5min":4.093,"15min":6.2329999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.56699999999999995,"15min":0.69299999999999995},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":3.9409999999999998,"15min":4.5300000000000002},"last":0.47599999999999998}]},{"osd":6,"last update":"Tue Sep 17 08:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.71399999999999997,"15min":0.64500000000000002},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":4.2880000000000003,"5min":4.2880000000000003,"15min":4.827},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.747,"15min":0.67900000000000005},"min":{"1min":0.38600000000000001,"5min":0.28399999999999997,"15min":0.28000000000000003},"max":{"1min":2.5449999999999999,"5min":3.6659999999999999,"15min":6.0010000000000003},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:28:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67800000000000005,"15min":0.748},"min":{"1min":0.375,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":4.1719999999999997,"5min":4.1719999999999997,"15min":4.875},"last":0.379},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.27300000000000002},"max":{"1min":1.901,"5min":2.5579999999999998,"15min":4.6950000000000003},"last":0.39900000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117242,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2292232,"kb_used_data":1589244,"kb_used_omap":1,"kb_used_meta":702974,"kb_avail":91452920,"statfs":{"total":95995035648,"available":93647790080,"internally_reserved":0,"allocated":1627385856,"data_stored":1626884026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":719845832},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:27:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":3.5920000000000001,"15min":2.4319999999999999},"min":{"1min":0.24099999999999999,"5min":0.17699999999999999,"15min":0.156},"max":{"1min":5.6399999999999997,"5min":130.40299999999999,"15min":130.40299999999999},"last":2.077},{"interface":"front","average":{"1min":0.75800000000000001,"5min":3.1549999999999998,"15min":2.1589999999999998},"min":{"1min":0.20899999999999999,"5min":0.17599999999999999,"15min":0.16200000000000001},"max":{"1min":2.6070000000000002,"5min":130.70099999999999,"15min":130.70099999999999},"last":1.038}]},{"osd":1,"last update":"Tue Sep 17 08:27:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.76,"15min":1.7370000000000001},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.186},"max":{"1min":1.3500000000000001,"5min":71.459999999999994,"15min":71.459999999999994},"last":2.2869999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":2.0310000000000001,"15min":1.8169999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.188},"max":{"1min":4.0430000000000001,"5min":71.438999999999993,"15min":71.438999999999993},"last":6.2210000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.454,"15min":1.4159999999999999},"min":{"1min":0.28499999999999998,"5min":0.253,"15min":0.215},"max":{"1min":4.4989999999999997,"5min":17.097999999999999,"15min":60.423999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.5349999999999999,"15min":1.7430000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.1070000000000002,"5min":14.925000000000001,"15min":60.368000000000002},"last":6.1070000000000002}]},{"osd":4,"last update":"Tue Sep 17 08:28:05 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.597,"15min":1.5640000000000001},"min":{"1min":0.40100000000000002,"5min":0.32300000000000001,"15min":0.27200000000000002},"max":{"1min":2.6240000000000001,"5min":30.722999999999999,"15min":35.841000000000001},"last":1.746},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.3979999999999999,"15min":1.425},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.27200000000000002},"max":{"1min":1.71,"5min":30.884,"15min":35.857999999999997},"last":1.073}]},{"osd":5,"last update":"Tue Sep 17 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.1439999999999999,"15min":1.325},"min":{"1min":0.39500000000000002,"5min":0.32900000000000001,"15min":0.29799999999999999},"max":{"1min":2.6579999999999999,"5min":4.3179999999999996,"15min":8.2240000000000002},"last":1.853},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1919999999999999,"15min":1.446},"min":{"1min":0.42599999999999999,"5min":0.32600000000000001,"15min":0.29499999999999998},"max":{"1min":3.278,"5min":5.5759999999999996,"15min":9.7289999999999992},"last":2.1419999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":1.03,"15min":1.2310000000000001},"min":{"1min":0.34999999999999998,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":1.3620000000000001,"5min":8.4550000000000001,"15min":10.945},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.5529999999999999},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.311},"max":{"1min":3.9470000000000001,"5min":6.3620000000000001,"15min":9.6600000000000001},"last":2.1669999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":2.4380000000000002,"15min":1.821},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":1.0669999999999999,"5min":163.006,"15min":163.006},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":2.7229999999999999,"15min":1.923},"min":{"1min":0.44400000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":2.7000000000000002,"5min":162.83699999999999,"15min":162.83699999999999},"last":6.2809999999999997}]}]},{"osd":2,"up_from":227,"seq":974957576293,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1569444,"kb_used_data":1270940,"kb_used_omap":4,"kb_used_meta":298491,"kb_avail":92175708,"statfs":{"total":95995035648,"available":94387924992,"internally_reserved":0,"allocated":1301442560,"data_stored":1300965553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":305655459},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.6200000000000001,"15min":1.256},"min":{"1min":0.23499999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":9.516,"5min":9.6389999999999993,"15min":9.6389999999999993},"last":1.589},{"interface":"front","average":{"1min":1.232,"5min":1.577,"15min":1.423},"min":{"1min":0.223,"5min":0.221,"15min":0.184},"max":{"1min":10.237,"5min":10.237,"15min":10.237},"last":0.94499999999999995}]},{"osd":1,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.032,"15min":1.399},"min":{"1min":0.246,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":2.3290000000000002,"5min":5.2789999999999999,"15min":41.405000000000001},"last":2.3820000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":1.276,"15min":1.4930000000000001},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":1.9099999999999999,"5min":5.617,"15min":41.25},"last":1.8480000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":1.4890000000000001,"15min":1.5449999999999999},"min":{"1min":0.27200000000000002,"5min":0.218,"15min":0.21299999999999999},"max":{"1min":1.95,"5min":21.477,"15min":44.185000000000002},"last":1.073},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.569,"15min":1.5680000000000001},"min":{"1min":0.26800000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":2.5110000000000001,"5min":21.422000000000001,"15min":43.994},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":2.4409999999999998,"15min":1.3819999999999999},"min":{"1min":0.39800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5669999999999999,"5min":62.298999999999999,"15min":62.298999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":2.2429999999999999,"15min":1.3049999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.6259999999999999,"5min":62.469000000000001,"15min":62.469000000000001},"last":0.36399999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.095,"15min":1.0009999999999999},"min":{"1min":0.40999999999999998,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":1.625,"5min":5.6420000000000003,"15min":5.6420000000000003},"last":0.432},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94299999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.512,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.33700000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0920000000000001,"15min":0.876},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.748,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":1.198,"15min":1.0009999999999999},"min":{"1min":0.42199999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.8029999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.41299999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.103,"15min":1.026},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.5449999999999999,"5min":8.9849999999999994,"15min":8.9849999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":1.1160000000000001,"15min":0.98799999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.7190000000000001,"5min":7.468,"15min":7.468},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575108,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2523084,"kb_used_data":1802048,"kb_used_omap":1,"kb_used_meta":721022,"kb_avail":91222068,"statfs":{"total":95995035648,"available":93411397632,"internally_reserved":0,"allocated":1845297152,"data_stored":1844791272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":738326982},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:28:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.833,"15min":1.821},"min":{"1min":0.17299999999999999,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":8.1460000000000008,"5min":27.670999999999999,"15min":31.585999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.6319999999999999,"15min":1.9350000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.17199999999999999},"max":{"1min":4.4939999999999998,"5min":26.120999999999999,"15min":35.807000000000002},"last":1.7330000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:28:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":1.375,"15min":1.131},"min":{"1min":0.27500000000000002,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":2.6970000000000001,"5min":26.297999999999998,"15min":26.297999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.27100000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.5739999999999998,"5min":26.602,"15min":26.602},"last":1.002}]},{"osd":3,"last update":"Tue Sep 17 08:27:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.161,"15min":1.488},"min":{"1min":0.223,"5min":0.223,"15min":0.184},"max":{"1min":4.5439999999999996,"5min":5.3739999999999997,"15min":8.0559999999999992},"last":1.052},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.1140000000000001,"15min":1.667},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20399999999999999},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":22.984999999999999},"last":1.6559999999999999}]},{"osd":4,"last update":"Tue Sep 17 08:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.2030000000000001,"15min":1.3939999999999999},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.25800000000000001},"max":{"1min":6.0720000000000001,"5min":14.109,"15min":16.884},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":1.0900000000000001,"15min":1.23},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":1.2529999999999999,"5min":14.151999999999999,"15min":14.151999999999999},"last":1.0640000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.92800000000000005,"15min":1.204},"min":{"1min":0.33900000000000002,"5min":0.29699999999999999,"15min":0.254},"max":{"1min":2.113,"5min":4.2460000000000004,"15min":7.9139999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.92800000000000005,"15min":1.409},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":3.9710000000000001,"5min":6.1520000000000001,"15min":16.853999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:27:48 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1319999999999999,"15min":1.3260000000000001},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":3.6709999999999998,"5min":9.7159999999999993,"15min":10.253},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0780000000000001,"15min":1.321},"min":{"1min":0.31,"5min":0.31,"15min":0.27200000000000002},"max":{"1min":4.2930000000000001,"5min":4.5369999999999999,"15min":7.9459999999999997},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.97999999999999998,"15min":1.605},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.1260000000000003,"5min":6.1260000000000003,"15min":47.442},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0029999999999999,"15min":1.4850000000000001},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5089999999999999,"5min":3.7759999999999998,"15min":47.475000000000001},"last":1.0880000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738630,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1781156,"kb_used_data":1082268,"kb_used_omap":1,"kb_used_meta":698878,"kb_avail":91963996,"statfs":{"total":95995035648,"available":94171131904,"internally_reserved":0,"allocated":1108242432,"data_stored":1107764958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":715651526},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:28:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.2649999999999999,"15min":1.9850000000000001},"min":{"1min":0.22,"5min":0.193,"15min":0.189},"max":{"1min":4.9279999999999999,"5min":20.497,"15min":61.649000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.5069999999999999,"15min":2.2130000000000001},"min":{"1min":0.24399999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.4630000000000001,"5min":20.088000000000001,"15min":61.281999999999996},"last":0.377}]},{"osd":2,"last update":"Tue Sep 17 08:28:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":2.8370000000000002,"15min":2.1299999999999999},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.17799999999999999},"max":{"1min":2.839,"5min":114.771,"15min":114.771},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":2.8519999999999999,"15min":2.0590000000000002},"min":{"1min":0.25700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":3.9849999999999999,"5min":114.732,"15min":114.732},"last":0.40699999999999997}]},{"osd":3,"last update":"Tue Sep 17 08:27:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.351},"min":{"1min":0.28299999999999997,"5min":0.20899999999999999,"15min":0.20399999999999999},"max":{"1min":7.6779999999999999,"5min":8.2129999999999992,"15min":61.926000000000002},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3080000000000001,"15min":1.9430000000000001},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":5.9710000000000001,"5min":10.868,"15min":61.616},"last":0.83299999999999996}]},{"osd":4,"last update":"Tue Sep 17 08:28:11 2024","interfaces":[{"interface":"back","average":{"1min":3.5339999999999998,"5min":1.782,"15min":1.637},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.26300000000000001},"max":{"1min":54.176000000000002,"5min":54.176000000000002,"15min":54.176000000000002},"last":2.6829999999999998},{"interface":"front","average":{"1min":3.2970000000000002,"5min":1.8700000000000001,"15min":1.647},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":54.097000000000001,"5min":54.097000000000001,"15min":54.097000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Tue Sep 17 08:28:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.1950000000000001,"15min":1.3879999999999999},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.889,"5min":10.805999999999999,"15min":39.058},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":1.8500000000000001,"15min":1.7050000000000001},"min":{"1min":0.35199999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.81299999999999994,"5min":54.116,"15min":54.116},"last":0.43099999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.1659999999999999,"15min":1.3799999999999999},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.27700000000000002},"max":{"1min":5.0890000000000004,"5min":10.827999999999999,"15min":38.917999999999999},"last":0.443},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.496,"15min":1.631},"min":{"1min":0.42499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9950000000000001,"5min":5.2400000000000002,"15min":41.189},"last":2.746}]},{"osd":7,"last update":"Tue Sep 17 08:28:21 2024","interfaces":[{"interface":"back","average":{"1min":3.4529999999999998,"5min":2.0619999999999998,"15min":1.859},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.29699999999999999},"max":{"1min":54.161000000000001,"5min":54.161000000000001,"15min":54.161000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":3.2210000000000001,"5min":1.6879999999999999,"15min":1.706},"min":{"1min":0.42099999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":53.604999999999997,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1124401152,"data_stored":1124401152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1871904768,"data_stored":1871904768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1324417024,"data_stored":1324417024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1634336768,"data_stored":1634336768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":905379840,"data_stored":905379840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":900530176,"data_stored":900530176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1795227648,"data_stored":1795227648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1362690048,"data_stored":1362690048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:28:48.595 INFO:tasks.thrashosds.thrasher:Setting 2.f to [7, 2, 1, 6, 4, 0] 2024-09-17T08:28:48.595 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.f', '7', '2', '1', '6', '4', '0'] 2024-09-17T08:28:48.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 2.f 7 2 1 6 4 0 2024-09-17T08:28:48.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:48.622 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:48.648 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:48.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:48.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:48.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.812 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:48.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:48.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[40649]: from='client.25781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:48.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[29565]: from='client.25781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:28:48.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:48.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:48.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:48 smithi173 ceph-mon[33187]: from='client.25781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:28:48.874 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:48.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:28:48.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:48.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:48.971 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:48.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:49.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:49.113 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:49.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:49.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:49.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:49.356 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:49.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:49.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:49.391 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:28:49.391 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:28:49.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 34 16 64934 64918 119.323 79.4375 0.018839 0.0083759 2024-09-17T08:28:49.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:49.510 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:49.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:49.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.655 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:49.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:49.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:49.783 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:49.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:49.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:49 smithi079 ceph-mon[40649]: pgmap v849: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:28:49.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:49 smithi079 ceph-mon[29565]: pgmap v849: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:28:49.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:49.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:49.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:49.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:49 smithi173 ceph-mon[33187]: pgmap v849: 158 pgs: 158 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-17T08:28:50.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:50.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.066 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:50.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:50.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:50.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.199 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:50.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:50.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:50.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:50.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:50.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:50.448 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 35 16 66189 66173 118.154 78.4375 0.0110276 0.00845897 2024-09-17T08:28:50.464 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:50.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:50.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:50.599 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:50.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:50.701 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:50.729 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:50.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:50.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:50.866 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:50.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:50.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:50.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:51.009 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:51.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:51.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:51.145 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:51.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:51.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:51.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.280 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:51.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:51.381 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:51.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:51.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:51.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 36 16 67428 67412 117.023 77.4375 0.0114421 0.00854036 2024-09-17T08:28:51.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:51.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:51.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:51.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:51.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:51.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:51.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:51.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:28:51.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:51.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:51.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:51 smithi079 ceph-mon[29565]: pgmap v850: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:28:51.855 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:51 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:51.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:51 smithi079 ceph-mon[40649]: pgmap v850: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:28:51.855 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:51 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:51.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:51.958 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:51.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:51.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:51 smithi173 ceph-mon[33187]: pgmap v850: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-17T08:28:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:51 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:28:52.060 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:52.097 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:52.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.098 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:52.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:52.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:52.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:52.329 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:52.362 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:52.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:52.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 37 16 68632 68616 115.894 75.25 0.0140989 0.00862378 2024-09-17T08:28:52.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:52.503 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:52.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:52.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:52.641 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:52.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:52.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:52.725 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:52.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:52.726 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:52 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:52.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:52.788 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:52.789 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:52.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:52.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:52.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:52.938 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:52.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:52 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:28:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:52 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:28:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:52 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:28:53.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:53.073 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:53.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.074 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:53.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:53.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:53.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:53.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:53.338 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:28:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:28:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:28:53.342 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:53.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:53.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:53.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 38 16 69994 69978 115.083 85.125 0.00724412 0.00868399 2024-09-17T08:28:53.476 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:53.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:53.578 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:53.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:53.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.609 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:53.614 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:28:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:28:53.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:53.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:53.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:53.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:53.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:53 smithi079 ceph-mon[29565]: pgmap v851: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:53 smithi079 ceph-mon[40649]: pgmap v851: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:53.891 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:53.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:53.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:53 smithi173 ceph-mon[33187]: pgmap v851: 158 pgs: 158 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:28:53.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:54.028 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:54.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:54.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:54.152 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:54.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:54.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:54.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:54.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:54.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:54.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:54.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:54.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 39 16 71328 71312 114.27 83.375 0.00552101 0.00874633 2024-09-17T08:28:54.520 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:54.548 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:54.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:54.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:54.690 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:54 smithi079 ceph-mon[29565]: pgmap v852: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:54.690 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:54 smithi079 ceph-mon[40649]: pgmap v852: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:54.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:54.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:54.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:54.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:54.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:54.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:54.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:54.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:54 smithi173 ceph-mon[33187]: pgmap v852: 158 pgs: 158 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-17T08:28:55.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:55.093 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:55.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:55.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:55.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:55.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:55.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:55.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:55.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:55.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:28:55.449969+0000 min lat: 0.00119686 max lat: 0.343744 avg lat: 0.00880883 2024-09-17T08:28:55.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:28:55.449 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 40 16 72640 72624 113.463 82 0.0112358 0.00880883 2024-09-17T08:28:55.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:55.497 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:55.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:55.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:55.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:55.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:55.648 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:28:55.717 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:28:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:28:55.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:55.750 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:28:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:28:55.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:55.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:55.784 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:28:55.784 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:28:55.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3360887073' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:55.785 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:55.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3360887073' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:55.785 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:55.819 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:28:55.819 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:28:55.852 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:28:55.853 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:28:55.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:55.888 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:28:55.888 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:28:55.905 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:55.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:55.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:55.920 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:28:55.920 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:28:55.957 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:28:55.958 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:28:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3360887073' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:28:56.006 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:56.036 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.037 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:56.137 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:56.174 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:56.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:56.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.312 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.312 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:56.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:56.449 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:56.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:56.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 41 15 73933 73918 112.667 80.875 0.00884476 0.00887065 2024-09-17T08:28:56.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:56.583 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:56.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:56.685 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:56.725 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: pgmap v853: 158 pgs: 158 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1017 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/643049919' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:56.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[29565]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: pgmap v853: 158 pgs: 158 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1017 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/643049919' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:56.760 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:56 smithi079 ceph-mon[40649]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T08:28:56.783 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:28:56.827 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:56.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:56.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:28:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: pgmap v853: 158 pgs: 158 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1017 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-17T08:28:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:28:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T08:28:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/643049919' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:28:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:56 smithi173 ceph-mon[33187]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T08:28:56.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:56.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:56.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:57.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:57.139 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:57.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:57.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:57.307 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:57.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:57.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:28:57.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 42 16 75246 75230 111.936 82 0.0155052 0.00892897 2024-09-17T08:28:57.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:57.451 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:28:57.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:57.674 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:57.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:57.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:57.804 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:57.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:57.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:57.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:57.950 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:57.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:58.052 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:58.084 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:58.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:58.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:58.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.231 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:58.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:58.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:58.375 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:58.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:58.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 43 16 76530 76514 111.199 80.25 0.0178391 0.00898811 2024-09-17T08:28:58.478 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:28:58.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:58.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:28:58.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:28:58.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.641 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:58.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:28:58.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:58.775 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:58.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:58.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:28:58.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:28:58.900 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:58.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:58.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:28:58.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:58 smithi173 ceph-mon[33187]: pgmap v856: 158 pgs: 158 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:28:59.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:59.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.047 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:58 smithi079 ceph-mon[29565]: pgmap v856: 158 pgs: 158 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:28:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:58 smithi079 ceph-mon[40649]: pgmap v856: 158 pgs: 158 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-17T08:28:59.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:28:59.185 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:28:59.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:59.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:59.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 44 16 77775 77759 110.44 77.8125 0.00841157 0.00905006 2024-09-17T08:28:59.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.472 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:59.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:28:59.602 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.603 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:28:59.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:28:59.743 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:28:59.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:28:59.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:28:59.875 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:28:59.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:28:59.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:28:59.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:28:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:28:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:28:59.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:00.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.009 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:00.110 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:00.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:00.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:28:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:00.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:00.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:28:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:00.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:29:00.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:00.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:00.276 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:00.378 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:00.408 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:00.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 45 15 78918 78903 109.575 71.5 0.00738591 0.00912129 2024-09-17T08:29:00.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:00.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.545 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:00.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:00.646 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:00.676 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:00.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:00.813 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:00.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:00 smithi079 ceph-mon[40649]: pgmap v857: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:29:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:00 smithi079 ceph-mon[40649]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T08:29:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:00 smithi079 ceph-mon[29565]: pgmap v857: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:29:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:00 smithi079 ceph-mon[29565]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T08:29:00.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:00.944 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:00.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:00.945 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:00 smithi173 ceph-mon[33187]: pgmap v857: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-17T08:29:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:00 smithi173 ceph-mon[33187]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T08:29:01.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:01.084 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:01.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:01.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:01.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:01.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:01.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:01.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:01.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:01.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 46 16 80072 80056 108.759 72.0625 0.0159583 0.00918956 2024-09-17T08:29:01.475 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:01.496 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:01.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:01.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:01.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:01.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:01.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:01.755 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:01.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:01.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]: dispatch 2024-09-17T08:29:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]': finished 2024-09-17T08:29:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[40649]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T08:29:01.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]: dispatch 2024-09-17T08:29:01.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]': finished 2024-09-17T08:29:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:01 smithi079 ceph-mon[29565]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T08:29:01.881 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:01.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:01.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]: dispatch 2024-09-17T08:29:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/29230578' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.f", "id": [7, 2, 1, 6, 4, 0]}]': finished 2024-09-17T08:29:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:01 smithi173 ceph-mon[33187]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:02.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:02.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:02.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:02.156 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:02.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:02.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:02.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:02.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:02.301 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.f pg_upmap_items mapping to [7->2,1->6,4->0] 2024-09-17T08:29:02.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:02.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:02.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:02.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 47 16 81101 81085 107.813 64.3125 0.0122667 0.0092705 2024-09-17T08:29:02.515 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:02.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.544 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:02.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:02.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:02.681 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:02.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:02.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:02.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:02.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:02.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:02.955 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:02.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:02.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:03.057 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:03.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:03.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:03.187 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:03.223 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:03.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:03.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:03.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:03.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:03.450 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 48 16 82319 82303 107.153 76.125 0.0153508 0.00932738 2024-09-17T08:29:03.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:03.492 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:03.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:03.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:03.618 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:03.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:03.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:03.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:03 smithi173 ceph-mon[33187]: pgmap v860: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 854 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:29:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:03 smithi173 ceph-mon[33187]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T08:29:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:03.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.755 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:03.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:03.856 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[40649]: pgmap v860: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 854 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:29:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[40649]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T08:29:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[29565]: pgmap v860: 158 pgs: 158 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 854 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-17T08:29:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[29565]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T08:29:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:03.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:03.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:03.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:03.986 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:04.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:04.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.015 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:04.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:04.154 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:04.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:04.292 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:04.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:04.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:04.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:04.451 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 49 16 83377 83361 106.316 66.125 0.00717792 0.00940114 2024-09-17T08:29:04.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:04.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.556 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.557 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:04.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:04.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:04.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:04.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:04.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:04.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:04.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:04 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:04.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:04.972 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:04.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:04.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:04 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:04 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:05.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:05.111 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:05.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:05.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:05.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:05.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:05.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:05.387 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:05.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:05.451 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 50 16 84664 84648 105.798 80.4375 0.00935663 0.00944745 2024-09-17T08:29:05.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:05.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.520 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:05.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:05.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:05 smithi079 ceph-mon[29565]: pgmap v862: 158 pgs: 158 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:05.619 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:05 smithi079 ceph-mon[29565]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T08:29:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:05 smithi079 ceph-mon[40649]: pgmap v862: 158 pgs: 158 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:05.620 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:05 smithi079 ceph-mon[40649]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T08:29:05.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:05.653 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:05.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:05.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:05.780 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:05.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:05.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:05.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:05.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:05.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:05 smithi173 ceph-mon[33187]: pgmap v862: 158 pgs: 158 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:05 smithi173 ceph-mon[33187]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T08:29:06.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:06.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:06.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:06.165 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:06.201 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:06.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:06.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:06.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:29:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:29:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:85763 11.a e379/359)", 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.359553+0000", 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.003003571, 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030259779999999999, 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359553+0000", 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359553+0000", 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359552+0000", 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359557+0000", 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3270000000000002e-06 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359566+0000", 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3379999999999994e-06 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359571+0000", 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5220000000000001e-06 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:85750 11.1 e379/359 11:83962d2f:::benchmark_data_smithi079_199625_object85749:head v 379'5318, mlcod=379'5316)", 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.334302+0000", 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.028254194, 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028330062999999999, 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:29:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334302+0000", 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334302+0000", 2024-09-17T08:29:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334302+0000", 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334338+0000", 2024-09-17T08:29:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6356999999999998e-05 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334347+0000", 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5949999999999999e-06 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.334352+0000", 2024-09-17T08:29:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0270000000000003e-06 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.340449+0000", 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060974260000000004 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.340472+0000", 2024-09-17T08:29:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3295999999999999e-05 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:85760 11.1 e379/359 11:8796eb6d:::benchmark_data_smithi079_199625_object85759:head v 379'5319, mlcod=379'5316)", 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.345238+0000", 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.017318699, 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017448374999999999, 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345238+0000", 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345238+0000", 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345237+0000", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345276+0000", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9446000000000001e-05 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345290+0000", 2024-09-17T08:29:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3988000000000001e-05 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345297+0000", 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6769999999999998e-06 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345352+0000", 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5535999999999998e-05 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345375+0000", 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2685999999999999e-05 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:85765 11.1 e379/359 11:8e884771:::benchmark_data_smithi079_199625_object85764:head v 379'5320, mlcod=379'5316)", 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.345478+0000", 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.017078759999999998, 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017260725000000001, 2024-09-17T08:29:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345478+0000", 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345478+0000", 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345477+0000", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345506+0000", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8674999999999999e-05 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345514+0000", 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6690000000000006e-06 2024-09-17T08:29:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345527+0000", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3478999999999999e-05 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.349436+0000", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039088980000000001 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.349455+0000", 2024-09-17T08:29:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9269000000000001e-05 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:85763 11.a 11:5c2cee7c:::benchmark_data_smithi079_199625_object85762:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.345577+0000", 2024-09-17T08:29:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.016979036999999999, 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017228721999999998, 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 85763 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345577+0000", 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345577+0000", 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345576+0000", 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345626+0000", 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0052999999999997e-05 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345634+0000", 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1640000000000001e-06 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.345639+0000", 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0069999999999997e-06 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.355846+0000", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010206461999999999 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.356169+0000", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032340299999999999 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.356597+0000", 2024-09-17T08:29:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042822900000000002 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:85766 11.9 11:95e90753:::benchmark_data_smithi079_199625_object85765:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.348816+0000", 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.013740204000000001, 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014058202000000001, 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:29:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 85766 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348816+0000", 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348816+0000", 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348814+0000", 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348844+0000", 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9856e-05 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348867+0000", 2024-09-17T08:29:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3044000000000002e-05 2024-09-17T08:29:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348879+0000", 2024-09-17T08:29:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1588e-05 2024-09-17T08:29:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348982+0000", 2024-09-17T08:29:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010356700000000001 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.349141+0000", 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015862800000000001 2024-09-17T08:29:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.349302+0000", 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016149099999999999 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:85762 11.2 e379/359 11:409eb21f:::benchmark_data_smithi079_199625_object85761:head v 379'5398, mlcod=379'5395)", 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.348902+0000", 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.013654537, 2024-09-17T08:29:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014034655, 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348902+0000", 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348902+0000", 2024-09-17T08:29:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348901+0000", 2024-09-17T08:29:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348923+0000", 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1746000000000002e-05 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348933+0000", 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789999999999993e-06 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.348940+0000", 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1600000000000001e-06 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.351442+0000", 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025021750000000001 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.351491+0000", 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000001e-05 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.360383+0000", 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088916640000000005 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:85766 11.9 e379/359)", 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.352638+0000", 2024-09-17T08:29:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0099184779999999997, 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010351449, 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352638+0000", 2024-09-17T08:29:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.404 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:06.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:06.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352638+0000", 2024-09-17T08:29:06.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352637+0000", 2024-09-17T08:29:06.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:29:06.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352647+0000", 2024-09-17T08:29:06.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.044e-05 2024-09-17T08:29:06.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352663+0000", 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-05 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.352671+0000", 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0499999999999992e-06 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:85773 11.a 11:5185b0be:::benchmark_data_smithi079_199625_object85772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-17T08:29:06.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.358082+0000", 2024-09-17T08:29:06.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0044741379999999999, 2024-09-17T08:29:06.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049557789999999996, 2024-09-17T08:29:06.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 85773 2024-09-17T08:29:06.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358082+0000", 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358082+0000", 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358080+0000", 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999976 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358104+0000", 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4213e-05 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358124+0000", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9555000000000002e-05 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358133+0000", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3000000000000007e-06 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.360475+0000", 2024-09-17T08:29:06.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023416000000000001 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.360830+0000", 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035556200000000002 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.361321+0000", 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000490655 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:85774 11.9 11.537365a9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e379)", 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.358141+0000", 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0044154110000000002, 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049594670000000004, 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:06.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 85774 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358141+0000", 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358141+0000", 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358141+0000", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358160+0000", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9023999999999999e-05 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358167+0000", 2024-09-17T08:29:06.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0650000000000001e-06 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.358172+0000", 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5430000000000001e-06 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:85763 11.a e379/359)", 2024-09-17T08:29:06.420 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:06.359489+0000", 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0030671980000000001, 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036616750000000001, 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359489+0000", 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:29:06.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359489+0000", 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359488+0000", 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359498+0000", 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2860000000000005e-06 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359512+0000", 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4363e-05 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:06.359519+0000", 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0500000000000003e-06 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:06.424 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:06.424 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 11 2024-09-17T08:29:06.424 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:29:06.452 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 51 16 85871 85855 105.2 75.4375 0.00832241 0.00950084 2024-09-17T08:29:06.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:06.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.545 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:06.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:06.633 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:06.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:06.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.693 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:06.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:06.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:06.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.826 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:06.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:06.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:06.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:06.959 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:06.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:07.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:07.097 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:07.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:07.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:07.225 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:07.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:07.278 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:07.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:29:07.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:07.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.405 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:07.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:07.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 52 16 87085 87069 104.634 75.875 0.0126455 0.00955184 2024-09-17T08:29:07.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:07.531 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:07.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:07.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:07.678 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:07.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:07.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:07.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:07 smithi079 ceph-mon[40649]: pgmap v864: 158 pgs: 158 active+clean; 5.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 863 B/s rd, 61 MiB/s wr, 969 op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:07.819 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:07 smithi079 ceph-mon[29565]: pgmap v864: 158 pgs: 158 active+clean; 5.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 863 B/s rd, 61 MiB/s wr, 969 op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:07.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.822 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:07.822 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:07.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:07.950 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:07.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:07.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:07 smithi173 ceph-mon[33187]: pgmap v864: 158 pgs: 158 active+clean; 5.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 863 B/s rd, 61 MiB/s wr, 969 op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:08.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:08.096 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:08.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:08.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:08.234 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:08.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:08.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:08.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:08.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:08.453 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 53 16 88315 88299 104.11 76.875 0.0108535 0.00960057 2024-09-17T08:29:08.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:08.502 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:08.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:08.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:08.639 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:08.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:08.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:08.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:08.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:08.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:08.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:08.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:09.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:09.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.028 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:09.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:09.158 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:09.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:09.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:09.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:09.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:09.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:09.453 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 54 15 89552 89537 103.615 77.375 0.0173563 0.00964576 2024-09-17T08:29:09.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:09.575 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:09.595 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:09.677 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:09.708 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:09.810 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:09.846 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[40649]: pgmap v865: 158 pgs: 158 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 837 B/s rd, 74 MiB/s wr, 1.19k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:09.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[40649]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T08:29:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[29565]: pgmap v865: 158 pgs: 158 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 837 B/s rd, 74 MiB/s wr, 1.19k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:09 smithi079 ceph-mon[29565]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T08:29:09.948 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:09 smithi173 ceph-mon[33187]: pgmap v865: 158 pgs: 158 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 837 B/s rd, 74 MiB/s wr, 1.19k op/s; 15 B/s, 0 objects/s recovering 2024-09-17T08:29:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:09 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:09 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:09 smithi173 ceph-mon[33187]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T08:29:09.976 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:09.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:09.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:10.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:10.104 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:10.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:10.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:10.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.251 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:10.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:10.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:10.391 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:10.453 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 55 16 90782 90766 103.128 76.8125 0.0167384 0.00969229 2024-09-17T08:29:10.494 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:10.529 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:10.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:10.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:10.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.663 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:10.663 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:10.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:10.772 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:29:10.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:29:10.773 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T08:29:10.773 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:29:10.773 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:29:10.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-17T08:29:10.799 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:10.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:10.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:10.936 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:10.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:10.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:11.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:11.090 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:11.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:11.157 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:11.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:11.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.238 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:11.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:11.381 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:11.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 56 16 91924 91908 102.56 71.375 0.00726154 0.00974101 2024-09-17T08:29:11.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:11.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:11.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:11.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.658 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:11.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:11.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:11 smithi079 ceph-mon[29565]: pgmap v867: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 12 B/s, 0 objects/s recovering 2024-09-17T08:29:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:11 smithi079 ceph-mon[29565]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T08:29:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:11 smithi079 ceph-mon[40649]: pgmap v867: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 12 B/s, 0 objects/s recovering 2024-09-17T08:29:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:11 smithi079 ceph-mon[40649]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T08:29:11.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:11.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:11.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:11.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:11 smithi173 ceph-mon[33187]: pgmap v867: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 12 B/s, 0 objects/s recovering 2024-09-17T08:29:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:11 smithi173 ceph-mon[33187]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T08:29:12.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:12.094 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:29:12.096 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:29:12.099 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:12.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:12.131 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:29:12.132 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:29:12.166 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:29:12.167 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:29:12.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:12.205 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:29:12.205 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:29:12.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.237 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:12.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:12.240 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:29:12.241 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:29:12.272 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:29:12.272 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:29:12.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:29:12.290 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:29:12.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:12.364 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:29:12.364 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:29:12.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.376 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:12.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:12.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 57 16 92939 92923 101.874 63.4375 0.00302578 0.00980836 2024-09-17T08:29:12.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:12.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:12.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:12.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:12.612 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:29:12.612 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:29:12.649 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:12.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:12.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3420077007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:12.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:12.791 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:12.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:12.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3420077007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:12.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3420077007' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:12.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:12.929 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:12.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:12.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:13.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:13.062 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:13.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:13.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:13.263 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:13.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:13.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:13.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:13.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 58 16 94139 94123 101.41 75 0.0120979 0.00985631 2024-09-17T08:29:13.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:13.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:13.616 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:13.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:13.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:13.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:13 smithi173 ceph-mon[33187]: pgmap v869: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 721 B/s rd, 54 MiB/s wr, 860 op/s 2024-09-17T08:29:13.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:13.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:13.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:13 smithi079 ceph-mon[29565]: pgmap v869: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 721 B/s rd, 54 MiB/s wr, 860 op/s 2024-09-17T08:29:13.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:13 smithi079 ceph-mon[40649]: pgmap v869: 158 pgs: 158 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 721 B/s rd, 54 MiB/s wr, 860 op/s 2024-09-17T08:29:13.881 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:13.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:13.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:13.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:14.019 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:14.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:14.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:14.160 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:14.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:14.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:14.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.296 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:14.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:14.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:14.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.443 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:14.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:14.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 59 15 95339 95324 100.964 75.0625 0.0146533 0.00990017 2024-09-17T08:29:14.526 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:14.526 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "" 2024-09-17T08:29:14.526 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:14.545 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:14.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.586 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:14.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:14.687 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:14.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.720 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:14.720 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:14.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:14.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.862 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:14.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:14.863 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:14.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:14.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:29:14.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:14.996 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:14.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:14.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:15.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:15.152 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:15.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:15.287 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:15.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:15.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:29:15.389 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:15.397 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:15.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:15.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:15.454 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:29:15.455503+0000 min lat: 0.00119686 max lat: 0.343744 avg lat: 0.00994774 2024-09-17T08:29:15.455 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:29:15.455 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 60 16 96477 96461 100.464 71.0625 0.0119802 0.00994774 2024-09-17T08:29:15.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:15.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:15.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:15.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.697 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:15.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:15.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:15.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:15.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:15.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:15 smithi079 ceph-mon[40649]: pgmap v870: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:15 smithi079 ceph-mon[29565]: pgmap v870: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:15.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:15.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:15.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:15.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:15.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:15 smithi173 ceph-mon[33187]: pgmap v870: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:16.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:16.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.088 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:16.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:16.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:16.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.227 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:16.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:16.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:16.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.366 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:16.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:16.455 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 61 16 97649 97633 100.018 73.25 0.0054959 0.00999375 2024-09-17T08:29:16.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:16.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:16.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:16.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:16.626 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:29:16.644 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:16.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:16.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:16.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:16.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:16.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:16 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:29:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:16 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:29:16.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:16.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:16.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:16.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:16.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:16 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:29:17.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:17.077 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:17.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:17.179 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:17.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:17.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:17.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:17.333 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:17.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.335 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:17.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:17.455 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 62 16 98869 98853 99.6342 76.25 0.0232852 0.0100319 2024-09-17T08:29:17.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.479 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:17.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:17.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:17.621 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:17.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:17.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:17.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:17.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:17.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:17.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:17.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:17.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:17.929 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: pgmap v871: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:29:17.929 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: pgmap v871: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T08:29:17.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:29:17.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:17 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:29:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: pgmap v871: 158 pgs: 158 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-17T08:29:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1669676756' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:29:17.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T08:29:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:29:17.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:17 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:29:18.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:18.054 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:18.139 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:29:18.139 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:29:18.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:18.190 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:18.292 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:18.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:18.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:18.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:18.455 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 63 16 100050 100034 99.2232 73.8125 0.0179701 0.0100737 2024-09-17T08:29:18.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:18.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:18.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.609 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:18.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:18.750 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:18.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: 1.8 scrub starts 2024-09-17T08:29:18.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: 1.8 scrub ok 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: pgmap v873: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 917 op/s 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: 1.8 scrub starts 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: 1.8 scrub ok 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: pgmap v873: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 917 op/s 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:29:18.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:18 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:29:18.888 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:18.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:18.889 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: 1.8 scrub starts 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: 1.8 scrub ok 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: pgmap v873: 158 pgs: 158 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 917 op/s 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:29:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:18 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:29:18.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:19.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.025 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:19.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:19.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:19.165 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:19.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:19.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:19.306 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:19.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:19.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.447 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:19.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:19.456 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 64 16 101292 101276 98.8856 77.625 0.013064 0.0101079 2024-09-17T08:29:19.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:19.580 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:19.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:19.682 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:19.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.720 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:19.720 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:19.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:29:19.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:19.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:19.867 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:19.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:19.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:29:19.979 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:19 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:29:20.030 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:20.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:20.111 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:29:20.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:19 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:29:20.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:20.174 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:20.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:20.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:20.298 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:29:20.299 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:29:20.299 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T08:29:20.299 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:29:20.299 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:29:20.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-17T08:29:20.317 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:20.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:20.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:20.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:20.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:20.456 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 65 16 102475 102459 98.5017 73.9375 0.00832158 0.0101467 2024-09-17T08:29:20.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:20.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.593 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:20.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:20.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:20.727 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:20.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:20.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:20.858 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[40649]: pgmap v874: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:29:20.859 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:29:20.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:29:20.860 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:20 smithi079 ceph-mon[29565]: pgmap v874: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-17T08:29:20.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:20 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:29:20.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:20 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:29:20.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:20 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:29:20.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:20 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:29:20.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:20 smithi173 ceph-mon[33187]: pgmap v874: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-17T08:29:20.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:20.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:20.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:20.973 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:21.003 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:21.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:21.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:29:21.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:21.182 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:21.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:21.284 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:21.312 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:21.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:21.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:21.458 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 66 16 103500 103484 97.9785 64.0625 0.0154329 0.0102018 2024-09-17T08:29:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:21.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:21.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:21.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.597 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:21.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:21.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:21.722 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:21.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:21.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:29:21.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:29:21.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:29:21.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:29:21.789 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:29:21.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:21 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:29:21.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:21.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:21.860 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:21.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:29:21.880 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:29:21.880 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:29:21.880 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:29:21.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:29:21.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:29:21.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:29:21.885 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:29:21.920 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:29:21.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:29:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:29:21.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:29:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:29:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:29:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:21 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:29:22.004 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:22.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:22.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:22.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.128 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:22.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:22.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:22.257 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:22.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:22.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:22.392 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:22.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:22.457 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 67 16 104405 104389 97.3603 56.5625 0.00857527 0.0102667 2024-09-17T08:29:22.483 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:22.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:22.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:22.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:22.587 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:22.656 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:22.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.688 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:22.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:22.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:22 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:29:22.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:22 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:29:22.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:22 smithi173 ceph-mon[33187]: pgmap v875: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:22.785 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2213803194' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:22.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:22.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.829 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:22.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:22.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:22.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:22.965 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:22.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:29:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:29:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[40649]: pgmap v875: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:23.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2213803194' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:23.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:29:23.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:29:23.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[29565]: pgmap v875: 158 pgs: 158 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-17T08:29:23.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2213803194' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:23.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:23.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.107 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:23.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:23.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:23.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:23.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:23.300 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:23.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:23.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:23.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:23.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 68 16 105650 105634 97.0702 77.8125 0.0122234 0.0102973 2024-09-17T08:29:23.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:23.518 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:23.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:23.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:23.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:23.673 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:23.751 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:23.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:23.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:29:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:29:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:29:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:29:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:23.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:23.879 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:23.879 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:29:23.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106094 11.7 e382/359 11:ef3e18d7:::benchmark_data_smithi079_199625_object106093:head v 382'6603, mlcod=382'6600)", 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:23.878938+0000", 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00026512200000000001, 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028540100000000001, 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.878938+0000", 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.878938+0000", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.878937+0000", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.879001+0000", 2024-09-17T08:29:23.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3628999999999997e-05 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.879015+0000", 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4343e-05 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.879022+0000", 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8199999999999999e-06 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.879057+0000", 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4929999999999999e-05 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:23.879071+0000", 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4012999999999999e-05 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:29:23.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 1 2024-09-17T08:29:23.884 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:23.909 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:23.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:23 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:29:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:23 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:29:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:24.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:24.051 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:24.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:24.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:24.188 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:24.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:24.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:24.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.316 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:24.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:24.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:24.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.446 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:24.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:24.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 69 16 106723 106707 96.6352 67.0625 0.00596644 0.0103433 2024-09-17T08:29:24.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:24.567 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:24.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:24.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:24.690 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:24.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:24.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:24.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:24.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:24.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:24.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:24.973 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:24.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:24 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:29:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:24 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:29:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:24 smithi173 ceph-mon[33187]: pgmap v876: 158 pgs: 158 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:25.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:25.107 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:25.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:29:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:29:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[29565]: pgmap v876: 158 pgs: 158 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:29:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:29:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:24 smithi079 ceph-mon[40649]: pgmap v876: 158 pgs: 158 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:25.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:29:25.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:25.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.273 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:25.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:25.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:25.419 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:25.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:25.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 70 16 107515 107499 95.9619 49.5 0.00367749 0.0104152 2024-09-17T08:29:25.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:25.552 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:25.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:25.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:25.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:25.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:25.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:25.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:25.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:25.888 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:29:25.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:29:25.937 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:25 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:29:25.967 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:25.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:25.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:26.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:26.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:26.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:29:26.105 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:29:26.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:29:26.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:29:26.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:29:26.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:29:26.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:29:26.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:25 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:29:26.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:26.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.249 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:26.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:26.262 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:26.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:26.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:26.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:26.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 71 16 108698 108682 95.6517 73.9375 0.0136974 0.0104501 2024-09-17T08:29:26.482 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:26.517 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:26.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:26.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:26.650 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:26.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:26.751 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:26.800 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:26.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:29:26.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[29565]: pgmap v877: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:29:26.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:29:26.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:26 smithi079 ceph-mon[40649]: pgmap v877: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:26.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:26.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:26.938 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:26.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:26.940 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:29:26.940 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:29:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:29:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:29:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:29:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:29:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:26 smithi173 ceph-mon[33187]: pgmap v877: 158 pgs: 158 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-17T08:29:27.039 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:27.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:27.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:27.090 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:27.091 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "" 2024-09-17T08:29:27.091 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:27.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:27.224 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:27.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:27.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:27.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:27.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:27.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 72 16 109977 109961 95.4333 79.9375 0.0074773 0.0104736 2024-09-17T08:29:27.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:27.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:27.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:27.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:27.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.638 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:27.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:27.693 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:27 smithi173 ceph-mon[33187]: 1.a scrub starts 2024-09-17T08:29:27.693 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:27 smithi173 ceph-mon[33187]: 1.a scrub ok 2024-09-17T08:29:27.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:27.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.775 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:27.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:27.776 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:27.876 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:27.909 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:27.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:27.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:28.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:28.046 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:28.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:28.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:27 smithi079 ceph-mon[29565]: 1.a scrub starts 2024-09-17T08:29:28.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:27 smithi079 ceph-mon[29565]: 1.a scrub ok 2024-09-17T08:29:28.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:27 smithi079 ceph-mon[40649]: 1.a scrub starts 2024-09-17T08:29:28.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:27 smithi079 ceph-mon[40649]: 1.a scrub ok 2024-09-17T08:29:28.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:28.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.192 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:28.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:28.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:28.327 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:28.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:28.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:28.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.459 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:28.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:28.460 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 73 16 111171 111155 95.1481 74.625 0.0088726 0.0105053 2024-09-17T08:29:28.561 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:28.587 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:28.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:28.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:28.728 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:28.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:28.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:28.872 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:28.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:28.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:28.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 1.2 scrub starts 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 1.2 scrub ok 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: pgmap v878: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 928 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:29:28.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:28 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:29:29.015 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:29.117 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:29:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:29:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 1.2 scrub starts 2024-09-17T08:29:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 1.2 scrub ok 2024-09-17T08:29:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: pgmap v878: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 928 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 1.2 scrub starts 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 1.2 scrub ok 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: pgmap v878: 158 pgs: 158 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 928 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:29:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:28 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:29:29.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:29.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:29.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:29.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:29.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:29.429 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:29.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:29.459 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 74 16 112348 112332 94.8563 73.5625 0.00589803 0.0105377 2024-09-17T08:29:29.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:29.567 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:29.655 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:29:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:29:29.655 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:29:29.655 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:29.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:29.714 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:29.716 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:29:29.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:29.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.858 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:29.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3571120570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:29.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:29 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:29.996 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:29.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:29.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:30.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3571120570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:29 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:30.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3571120570' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:30.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:29 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:30.136 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:30.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:30.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:30.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:30.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:30.369 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:30.394 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:30.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:30.460 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 75 16 113531 113515 94.5773 73.9375 0.00855736 0.0105691 2024-09-17T08:29:30.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:30.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.540 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:30.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:30.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:30.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:30.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:30.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:29:30.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:30.815 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:30.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:30.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:30.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:30.947 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:30.947 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: pgmap v879: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:30 smithi173 ceph-mon[33187]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T08:29:31.045 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:31.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:31.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:31.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: pgmap v879: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[40649]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: pgmap v879: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:31.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:30 smithi079 ceph-mon[29565]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T08:29:31.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:31.214 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:31.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:31.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:31.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:31.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:31.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:31.460 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 76 16 114698 114682 94.2925 72.9375 0.0142558 0.010601 2024-09-17T08:29:31.491 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:31.492 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:29:31.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:29:31.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:31.530 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:29:31.530 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:29:31.570 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:29:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:29:31.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:31.602 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:29:31.603 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:29:31.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:31.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:31.633 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:29:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:29:31.664 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:29:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:29:31.696 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:29:31.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:29:31.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:31.732 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:29:31.732 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:29:31.766 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:31.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:31.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:31.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:31.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:31.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:31 smithi173 ceph-mon[33187]: 4.2 deep-scrub starts 2024-09-17T08:29:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:31 smithi173 ceph-mon[33187]: 4.2 deep-scrub ok 2024-09-17T08:29:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:31 smithi173 ceph-mon[33187]: 11.4 scrub starts 2024-09-17T08:29:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:31 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1793959512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:32.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:32.044 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:32.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[40649]: 4.2 deep-scrub starts 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[40649]: 4.2 deep-scrub ok 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[40649]: 11.4 scrub starts 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1793959512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[29565]: 4.2 deep-scrub starts 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[29565]: 4.2 deep-scrub ok 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[29565]: 11.4 scrub starts 2024-09-17T08:29:32.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:31 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1793959512' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:32.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:32.182 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:32.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:32.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:32.323 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:32.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:32.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:32.455 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:32.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:32.460 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 77 16 115808 115792 93.9689 69.375 0.00821358 0.0106367 2024-09-17T08:29:32.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:32.592 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:32.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:32.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:32.730 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:32.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:32.811 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:29:32.811 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:29:32.811 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-17T08:29:32.811 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:29:32.812 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:29:32.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 config set filestore_inject_stall 3 2024-09-17T08:29:32.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:32.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:32.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:32.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:32.944 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:32 smithi173 ceph-mon[33187]: 11.f scrub starts 2024-09-17T08:29:32.944 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:32 smithi173 ceph-mon[33187]: pgmap v881: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-17T08:29:32.981 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:29:32.981 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:29:32.996 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:33.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:32 smithi079 ceph-mon[29565]: 11.f scrub starts 2024-09-17T08:29:33.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:32 smithi079 ceph-mon[29565]: pgmap v881: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-17T08:29:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:32 smithi079 ceph-mon[40649]: 11.f scrub starts 2024-09-17T08:29:33.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:32 smithi079 ceph-mon[40649]: pgmap v881: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-17T08:29:33.030 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:33.035 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:33.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:33.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:33.153 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:33.163 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:33.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:33.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:29:33.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:33.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.328 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:33.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:33.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:33.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 78 16 116918 116902 93.6522 69.375 0.01604 0.0106727 2024-09-17T08:29:33.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:33.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:33.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:33.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:33.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:33.651 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:33.685 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:29:33.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:33.780 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:33.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:33.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:33.930 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:33.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:33.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:33.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: 11.4 scrub ok 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:34.016 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:34.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:34.048 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:34.058 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:34.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: 11.4 scrub ok 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: 11.4 scrub ok 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:34.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:34.203 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:34.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:34.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:34.342 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:34.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:34.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:34.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 79 16 118090 118074 93.3918 73.25 0.0104087 0.0107033 2024-09-17T08:29:34.477 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:34.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.478 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:34.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:29:34.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:34.627 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:34.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:34.729 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:34.755 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:34.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:34.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:29:34.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:29:34.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:29:34.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:29:34.835 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: pgmap v882: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 11.f scrub ok 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:29:34.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:29:34.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: pgmap v882: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-17T08:29:34.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 11.f scrub ok 2024-09-17T08:29:34.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:29:34.837 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:34 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:29:34.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:34.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:34.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:34.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:34.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:35.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.034 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:35.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:35.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:35.176 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:35.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.177 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: pgmap v882: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 11.f scrub ok 2024-09-17T08:29:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:29:35.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:34 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:29:35.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:35.317 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:35.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:35.419 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:35.449 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:35.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:35.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:2024-09-17T08:29:35.463884+0000 min lat: 0.00119686 max lat: 0.343744 avg lat: 0.010752 2024-09-17T08:29:35.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T08:29:35.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 80 16 119039 119023 92.9658 59.3125 0.0131692 0.010752 2024-09-17T08:29:35.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:35.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:35.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:35.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:35.715 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:35.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:35.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:35.846 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:35.948 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:35.974 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:35.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:35.975 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:36.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:36.109 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:36.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:36.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[40649]: 11.d scrub starts 2024-09-17T08:29:36.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[40649]: 5.15 deep-scrub starts 2024-09-17T08:29:36.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[40649]: 5.15 deep-scrub ok 2024-09-17T08:29:36.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[40649]: 1.9 scrub starts 2024-09-17T08:29:36.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[40649]: 1.9 scrub ok 2024-09-17T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[29565]: 11.d scrub starts 2024-09-17T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[29565]: 5.15 deep-scrub starts 2024-09-17T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[29565]: 5.15 deep-scrub ok 2024-09-17T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[29565]: 1.9 scrub starts 2024-09-17T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:35 smithi079 ceph-mon[29565]: 1.9 scrub ok 2024-09-17T08:29:36.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:35 smithi173 ceph-mon[33187]: 11.d scrub starts 2024-09-17T08:29:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:35 smithi173 ceph-mon[33187]: 5.15 deep-scrub starts 2024-09-17T08:29:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:35 smithi173 ceph-mon[33187]: 5.15 deep-scrub ok 2024-09-17T08:29:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:35 smithi173 ceph-mon[33187]: 1.9 scrub starts 2024-09-17T08:29:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:35 smithi173 ceph-mon[33187]: 1.9 scrub ok 2024-09-17T08:29:36.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:36.323 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:29:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T08:29:36.324 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:29:36.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:36.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:36.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:36.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 81 16 119993 119977 92.5541 59.625 0.0805605 0.0107964 2024-09-17T08:29:36.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:36.525 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:36.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:36.627 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:36.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:36.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:36.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:36.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:36.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:36.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:36.927 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:36.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:36.928 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:37.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:37.065 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:37.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[29565]: 11.2 scrub starts 2024-09-17T08:29:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:29:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:29:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[29565]: pgmap v883: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:29:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[40649]: 11.2 scrub starts 2024-09-17T08:29:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:29:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:29:37.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:36 smithi079 ceph-mon[40649]: pgmap v883: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:29:37.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:37.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:37.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:37.202 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:36 smithi173 ceph-mon[33187]: 11.2 scrub starts 2024-09-17T08:29:37.202 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:36 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:29:37.202 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:36 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:29:37.202 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:36 smithi173 ceph-mon[33187]: pgmap v883: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-17T08:29:37.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:37.332 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:37.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:37.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:37.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 82 16 120969 120953 92.1693 61 0.0161256 0.0108441 2024-09-17T08:29:37.472 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:37.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:37.496 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:29:37.496 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:29:37.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:37.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:37.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:37.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:37.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:37.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:37.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:37.876 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:37.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:37.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:37.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:38.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:38.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 11.d scrub ok 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 11.3 deep-scrub starts 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 5.0 deep-scrub starts 2024-09-17T08:29:38.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[40649]: 5.0 deep-scrub ok 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 11.d scrub ok 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 11.3 deep-scrub starts 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 5.0 deep-scrub starts 2024-09-17T08:29:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:37 smithi079 ceph-mon[29565]: 5.0 deep-scrub ok 2024-09-17T08:29:38.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:29:38.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:29:38.141 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 11.d scrub ok 2024-09-17T08:29:38.141 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 11.3 deep-scrub starts 2024-09-17T08:29:38.141 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 5.0 deep-scrub starts 2024-09-17T08:29:38.141 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:37 smithi173 ceph-mon[33187]: 5.0 deep-scrub ok 2024-09-17T08:29:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:38.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:29:38.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:38.288 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:38.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:38.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.429 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:38.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 83 16 122054 122038 91.8758 67.8125 0.0196954 0.0108792 2024-09-17T08:29:38.494 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:38.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:38.568 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:38.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:38.708 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:38.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:38.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:38.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:38.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:38.968 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:38.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:38.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: 2.6 deep-scrub starts 2024-09-17T08:29:38.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: 2.6 deep-scrub ok 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: pgmap v884: 158 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 155 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: 11.2 scrub ok 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:38.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:38 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:39.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:39.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:39.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:39.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:39.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:39.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:39.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:39.357 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:39.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: 2.6 deep-scrub starts 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: 2.6 deep-scrub ok 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: pgmap v884: 158 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 155 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: 11.2 scrub ok 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:29:39.367 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: 2.6 deep-scrub starts 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: 2.6 deep-scrub ok 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: pgmap v884: 158 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 155 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: 11.2 scrub ok 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:39.368 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:38 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:39.388 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:39.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:39.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 84 16 122831 122815 91.3602 48.5625 0.00347226 0.0109154 2024-09-17T08:29:39.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:39.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:39.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:39.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.608 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:39.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:39.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:39.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:39.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:39.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:39.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:39.877 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:39.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:39.979 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:39.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:29:39.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:29:39.989 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[29565]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T08:29:39.990 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:29:39.990 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:29:39.990 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:39 smithi079 ceph-mon[40649]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T08:29:40.010 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:40.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:40.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:39 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:29:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:39 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:29:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:39 smithi173 ceph-mon[33187]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T08:29:40.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:40.281 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:40.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:40.383 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:40.413 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:40.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 85 16 123731 123715 90.9471 56.25 0.0123683 0.0109893 2024-09-17T08:29:40.515 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:40.549 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:40.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:40.683 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:40.785 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:40.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:40.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:40.910 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:29:40.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:40.953 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:40.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:40.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:41.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:41.085 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:41.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:41.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: 11.1 scrub starts 2024-09-17T08:29:41.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: pgmap v886: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:41.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: 11.5 scrub starts 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: 11.1 scrub starts 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: pgmap v886: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T08:29:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: 11.5 scrub starts 2024-09-17T08:29:41.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:40 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:29:41.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: 11.1 scrub starts 2024-09-17T08:29:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: pgmap v886: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-17T08:29:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T08:29:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: 11.5 scrub starts 2024-09-17T08:29:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:40 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:29:41.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:41.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:41.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:41.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:41.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:41.456 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:41.463 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 86 15 124736 124721 90.6207 62.875 0.00802003 0.0110294 2024-09-17T08:29:41.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:41.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:41.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:41.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.622 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:41.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:41.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:41.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:41.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:41.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:41.885 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:41.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:41.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:41.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:42.020 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:42.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:42.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[29565]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:29:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:41 smithi079 ceph-mon[40649]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T08:29:42.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:42.149 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:41 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:29:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:41 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:29:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:41 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:41 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1941976526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:29:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:41 smithi173 ceph-mon[33187]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T08:29:42.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:42.280 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:42.303 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:29:42.303 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:29:42.335 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:29:42.335 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:29:42.368 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:29:42.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:29:42.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:42.397 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:29:42.397 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:29:42.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:42.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:29:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:29:42.459 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:29:42.460 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:29:42.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 87 16 125716 125700 90.2824 61.1875 0.023027 0.0110688 2024-09-17T08:29:42.499 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:29:42.499 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:29:42.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:42.533 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:29:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:29:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.556 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:42.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:42.657 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:42.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.693 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:42.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:42.831 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:42.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:42.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:42.980 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:42.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:42 smithi079 ceph-mon[29565]: pgmap v889: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 854 op/s 2024-09-17T08:29:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/61246655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:43.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:42 smithi079 ceph-mon[40649]: pgmap v889: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 854 op/s 2024-09-17T08:29:43.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/61246655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:43.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:43.121 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:43.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:43.176 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:42 smithi173 ceph-mon[33187]: pgmap v889: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 854 op/s 2024-09-17T08:29:43.176 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/61246655' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:43.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:43.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:43.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:43.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:43.384 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:43.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:43.429 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:43.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 88 16 126782 126766 90.0135 66.625 0.00903051 0.0111036 2024-09-17T08:29:43.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:43.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:43.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:43.559 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:29:43.559 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:29:43.559 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T08:29:43.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:29:43.582 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:43.582 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:29:43.582 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:29:43.583 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:29:43.583 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:43.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:43.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:43.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:43.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:43.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.806 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:43.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:43.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:43.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:43.941 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:43.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:44.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:44.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.080 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:44.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:44.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:44.213 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:44.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:44.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:44.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:44.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:29:44.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:44.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:44.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:44.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:44.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:44.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout: 89 16 127887 127871 89.7781 69.0625 0.00756205 0.0111328 2024-09-17T08:29:44.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:44.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:44.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:44.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.626 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:44.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:44.647 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:44.679 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:44.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:44.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:44.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:44.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:29:44.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:44.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:44.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:44.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:45.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:45.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:45.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:45.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[40649]: pgmap v890: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.6 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:29:45.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[40649]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[29565]: pgmap v890: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.6 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:45.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:44 smithi079 ceph-mon[29565]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T08:29:45.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:45.174 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:45.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:45.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:44 smithi173 ceph-mon[33187]: pgmap v890: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.6 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-17T08:29:45.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:44 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:45.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:44 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:45.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:44 smithi173 ceph-mon[33187]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T08:29:45.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:45.313 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:45.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:45.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:45.448 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:45.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total time run: 90.0103 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Total writes made: 128875 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Write size: 65536 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Object size: 65536 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Bandwidth (MB/sec): 89.4863 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Bandwidth: 32.9435 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max bandwidth (MB/sec): 227.438 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min bandwidth (MB/sec): 48.5625 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average IOPS: 1431 2024-09-17T08:29:45.464 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev IOPS: 527.096 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max IOPS: 3639 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min IOPS: 777 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Average Latency(s): 0.0111709 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Stddev Latency(s): 0.0111771 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Max latency(s): 0.343744 2024-09-17T08:29:45.465 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Min latency(s): 0.00119686 2024-09-17T08:29:45.470 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T08:29:45.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:45.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.584 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:45.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:45.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:45.723 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:45.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:45.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:45.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:45.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:45.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:45.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:45.998 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:45.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:46.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:46.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:46.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:46.149 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:46.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:46.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:46.379 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:46.406 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:46.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:29:46.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:46.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:46.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:46.682 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:46.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:46.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:46.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:46.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:46.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:46.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:47.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:47.082 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:47.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:47.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:46 smithi079 ceph-mon[40649]: pgmap v892: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 792 op/s 2024-09-17T08:29:47.167 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:46 smithi079 ceph-mon[29565]: pgmap v892: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 792 op/s 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 387, 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:29:43.900118+0000", 2024-09-17T08:29:47.171 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:20:22.465407+0000", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:15:28.332713+0000", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5801984, 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.172 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 19, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 11, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:29:47.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.174 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 11, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 11, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 11, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 11, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 11, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "334", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "332", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:29:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9099999666213989, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9099999666213989, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5399999618530273, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5399999618530273, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:29:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:29:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.3299999237060547, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.3299999237060547, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.190000057220459, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.190000057220459, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.5, 2024-09-17T08:29:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.5, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:29:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.3200000524520874, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.3200000524520874, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.5, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.5, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:29:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:29:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.5399999618530273, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.5399999618530273, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.75, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.75, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 11, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_4", 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:27:59.045325+0000", 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:29:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:29:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "364", 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:29:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:29:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:29:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.75, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.75, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.87362480163574219, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 18, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 204, 2024-09-17T08:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 227, 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 205, 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/359954782", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/359954782", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/359954782", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/359954782", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 377, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 205, 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.9889984130859375, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 359, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:29:43.758608+0000", 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:28:44.692969+0000", 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:29:03.550764+0000", 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 204 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:29:38.327894+0000", 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:28:58.738432+0000", 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:29:39.842238+0000", 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:28:35.603545+0000", 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:29:08.061818+0000", 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: 4, 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: 6 2024-09-17T08:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "5.2", 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: 6, 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.f", 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 4, 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 0 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "6.b", 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 6 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-17T08:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:29:47.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:47.285 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:46 smithi173 ceph-mon[33187]: pgmap v892: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 792 op/s 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238266 9.9 e279/221 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head v 279'14961, mlcod=279'14957)", 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 388.31103177300002, 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.074654730000000002, 2024-09-17T08:29:47.286 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965477+0000", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965486+0000", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4800000000000001e-06 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033392+0000", 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067906525999999995 2024-09-17T08:29:47.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033402+0000", 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0030999999999999e-05 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033429+0000", 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6608e-05 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033446+0000", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6991e-05 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040094+0000", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0066476039999999997 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040133+0000", 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9164e-05 2024-09-17T08:29:47.290 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:116954 10.9 e315/287 10:95a59d3e:::benchmark_data_smithi079_172150_object116953:head v 315'7258, mlcod=315'7255)", 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.06711999300001, 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.027391598, 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209389+0000", 2024-09-17T08:29:47.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209455+0000", 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6304999999999996e-05 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209467+0000", 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1497e-05 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209472+0000", 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.733e-06 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209511+0000", 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8862999999999997e-05 2024-09-17T08:29:47.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209524+0000", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.294e-05 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236634+0000", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.027110026999999998 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236781+0000", 2024-09-17T08:29:47.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000147019 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:116958 10.9 e315/287 10:925c62cc:::benchmark_data_smithi079_172150_object116957:head v 315'7259, mlcod=315'7255)", 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 253.062914288, 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023238808, 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213594+0000", 2024-09-17T08:29:47.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213696+0000", 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000101399 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213710+0000", 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4599e-05 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213717+0000", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9349999999999997e-06 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.216068+0000", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023509519999999999 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.216118+0000", 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9301000000000001e-05 2024-09-17T08:29:47.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236785+0000", 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020667723999999998 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236834+0000", 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8998000000000001e-05 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:159938 10.3 e325/287 10:cc1b825c:::benchmark_data_smithi079_172150_object27406:head v 325'10006, mlcod=325'10004)", 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 216.02839630400001, 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020320609, 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248119+0000", 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6669999999999999e-06 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248130+0000", 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1043000000000001e-05 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248136+0000", 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0839999999999997e-06 2024-09-17T08:29:47.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248255+0000", 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118835 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248275+0000", 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9669000000000001e-05 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.268393+0000", 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020117939000000001 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.268434+0000", 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1230000000000003e-05 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:182969 10.9 e328/287 10:9fd709d5:::benchmark_data_smithi079_172150_object50437:head v 328'11344, mlcod=328'11342)", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.067615133, 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020290156, 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208893+0000", 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208902+0000", 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4929999999999998e-06 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208921+0000", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9609000000000001e-05 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208929+0000", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5039999999999996e-06 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.209027+0000", 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8041999999999994e-05 2024-09-17T08:29:47.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.209047+0000", 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9690999999999998e-05 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.229119+0000", 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020072494999999999 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.229185+0000", 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5562999999999996e-05 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:182971 10.9 e328/287 10:9d474694:::benchmark_data_smithi079_172150_object50439:head v 328'11345, mlcod=328'11342)", 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 196.06485573500001, 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022397592000000001, 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211653+0000", 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211661+0000", 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.619e-06 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211678+0000", 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6467e-05 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211685+0000", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4309999999999997e-06 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.224431+0000", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012746061 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.225375+0000", 2024-09-17T08:29:47.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00094351000000000005 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.233968+0000", 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085933899999999994 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.234052+0000", 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3244000000000005e-05 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.315 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:187094 10.7 e329/287 10:ea312feb:::benchmark_data_smithi079_172150_object54562:head v 329'11879, mlcod=328'11877)", 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 192.49443808500001, 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022168162000000002, 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782071+0000", 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782079+0000", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1540000000000002e-06 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782095+0000", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6490999999999998e-05 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782103+0000", 2024-09-17T08:29:47.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2910000000000002e-06 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782256+0000", 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015338400000000001 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782276+0000", 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9857999999999999e-05 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.804124+0000", 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.021847833000000001 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.804240+0000", 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116125 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.320 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189324 10.9 e330/287 10:9d7ba2fc:::benchmark_data_smithi079_172150_object56792:head v 330'11742, mlcod=329'11739)", 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.462353834, 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037981067, 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814155+0000", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814162+0000", 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.233e-06 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814172+0000", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0161e-05 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814178+0000", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0419999999999999e-06 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814210+0000", 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1576999999999999e-05 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814222+0000", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2439999999999999e-05 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.852112+0000", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037889277999999998 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.852137+0000", 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5248e-05 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.324 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189320 10.a e330/287 10:510c5709:::benchmark_data_smithi079_172150_object56788:head v 330'11641, mlcod=329'11638)", 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.461621716, 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040285575999999997, 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814887+0000", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814893+0000", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2989999999999996e-06 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844966+0000", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.030073150999999999 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844982+0000", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5826000000000001e-05 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845125+0000", 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014297100000000001 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845146+0000", 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1331999999999998e-05 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.855122+0000", 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099756859999999992 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.855174+0000", 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2009999999999998e-05 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189327 10.7 e330/287 10:eda6383a:::benchmark_data_smithi079_172150_object56795:head v 330'12005, mlcod=329'12003)", 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.46142940199999, 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.044942533999999999, 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.329 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815084+0000", 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6940000000000001e-06 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844976+0000", 2024-09-17T08:29:47.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.029892245000000001 2024-09-17T08:29:47.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844988+0000", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.241e-05 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845185+0000", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019688299999999999 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845205+0000", 2024-09-17T08:29:47.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9544000000000001e-05 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.859975+0000", 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.014770471 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.860023+0000", 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7463000000000003e-05 2024-09-17T08:29:47.335 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189339 10.3 e330/287 10:c4086931:::benchmark_data_smithi079_172150_object56807:head v 330'11923, mlcod=329'11921)", 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.45135570799999, 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025840539999999999, 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825153+0000", 2024-09-17T08:29:47.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825163+0000", 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9520000000000006e-06 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845010+0000", 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019847265999999999 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845018+0000", 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2099999999999993e-06 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845124+0000", 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106198 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845146+0000", 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1807e-05 2024-09-17T08:29:47.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.850941+0000", 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057952309999999996 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.850995+0000", 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3202999999999999e-05 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189616 10.6 e330/287 10:6e001a48:::benchmark_data_smithi079_172150_object57084:head v 330'11764, mlcod=330'11762)", 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.19597642900001, 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.056560214999999997, 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080532+0000", 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080540+0000", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6850000000000001e-06 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080551+0000", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1029000000000001e-05 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080557+0000", 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6080000000000001e-06 2024-09-17T08:29:47.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080598+0000", 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1151e-05 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080616+0000", 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7897000000000002e-05 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.136110+0000", 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055493847999999998 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.137094+0000", 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098344300000000008 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.345 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189620 10.a e330/287 10:537f11e0:::benchmark_data_smithi079_172150_object57088:head v 330'11661, mlcod=330'11659)", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 190.19531385499999, 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.048894895000000001, 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081195+0000", 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081201+0000", 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0260000000000004e-06 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118855+0000", 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037654253999999998 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118871+0000", 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5211000000000001e-05 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118929+0000", 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7923000000000001e-05 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118951+0000", 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2662000000000001e-05 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.129820+0000", 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010868274000000001 2024-09-17T08:29:47.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.130091+0000", 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00027126199999999998 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:195080 10.6 e330/287 10:6651d373:::benchmark_data_smithi079_172150_object62548:head v 330'12097, mlcod=330'12094)", 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 185.67759859899999, 2024-09-17T08:29:47.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023128816999999999, 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598910+0000", 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598919+0000", 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4910000000000005e-06 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598929+0000", 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8109999999999997e-06 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598934+0000", 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6699999999999999e-06 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.605407+0000", 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064728019999999997 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.353 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.605427+0000", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.963e-05 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.621926+0000", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.016499533 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.622040+0000", 2024-09-17T08:29:47.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000113596 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:195597 10.9 e330/287 10:96153e48:::benchmark_data_smithi079_172150_object63065:head v 330'12114, mlcod=330'12112)", 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 185.250605269, 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.021097376000000001, 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025903+0000", 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:47.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025912+0000", 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8230000000000001e-06 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025931+0000", 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9462000000000001e-05 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025939+0000", 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7190000000000001e-06 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.026286+0000", 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034661099999999998 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.026357+0000", 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1161999999999994e-05 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.046853+0000", 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020496209000000001 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.047002+0000", 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000148651 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106703 11.a e382/359 11:5180121c:::benchmark_data_smithi079_199625_object106702:head v 382'6641, mlcod=382'6638)", 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.833011678999998, 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29516395099999998, 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443497+0000", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443545+0000", 2024-09-17T08:29:47.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8253e-05 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443557+0000", 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2259000000000001e-05 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443564+0000", 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0740000000000004e-06 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.362 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443636+0000", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1633999999999999e-05 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443654+0000", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7776000000000001e-05 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446434+0000", 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027802529999999999 2024-09-17T08:29:47.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738662+0000", 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29222788500000002 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106701 11.7 e382/359 11:e771ded9:::benchmark_data_smithi079_199625_object106700:head v 382'6643, mlcod=382'6641)", 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.832375192000001, 2024-09-17T08:29:47.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29689527900000001, 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444134+0000", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444176+0000", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2512000000000002e-05 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444187+0000", 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1239e-05 2024-09-17T08:29:47.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444193+0000", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0270000000000001e-06 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444240+0000", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7003999999999999e-05 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444258+0000", 2024-09-17T08:29:47.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7932999999999999e-05 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.740993+0000", 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29673467399999998 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741030+0000", 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6742999999999997e-05 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106709 11.5 e382/359 11:aac0c29c:::benchmark_data_smithi079_199625_object106708:head v 382'6653, mlcod=382'6651)", 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.828835667, 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.300296497, 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:29:47.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447673+0000", 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447697+0000", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4358e-05 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738547+0000", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29084915 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738570+0000", 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3396000000000001e-05 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738689+0000", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119388 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738711+0000", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1302e-05 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.747842+0000", 2024-09-17T08:29:47.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.009131676 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.747971+0000", 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012826299999999999 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106708 11.8 e382/359 11:12156efd:::benchmark_data_smithi079_199625_object106707:head v 382'6620, mlcod=382'6618)", 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.823515860000001, 2024-09-17T08:29:47.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28997071800000002, 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452993+0000", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.453044+0000", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1183000000000001e-05 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738567+0000", 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28552359399999999 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738574+0000", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6900000000000003e-06 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738619+0000", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4552000000000003e-05 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738649+0000", 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0037000000000001e-05 2024-09-17T08:29:47.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.742942+0000", 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004293056 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.742965+0000", 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2759e-05 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106751 11.7 e382/359 11:e1a8d295:::benchmark_data_smithi079_199625_object106750:head v 382'6644, mlcod=382'6641)", 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.781766847, 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.24684335199999999, 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494741+0000", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494797+0000", 2024-09-17T08:29:47.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6169999999999999e-05 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738553+0000", 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.243755797 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738566+0000", 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2909e-05 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.380 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738738+0000", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000172225 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738757+0000", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8749999999999998e-05 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741555+0000", 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027980050000000001 2024-09-17T08:29:47.381 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741586+0000", 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1399000000000002e-05 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:47.382 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:47.383 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:47.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:47.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:47.515 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:47.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:47.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:47.651 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:47.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:47.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:47.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:47.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:47.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:47.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:47.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:47.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:48.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:48.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:48.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:48.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:48.199 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3370441503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:29:48.200 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:48.200 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:47 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:48.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:29:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:48.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:48.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:48.343 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:48.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:48.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3370441503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:29:48.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:48.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:48.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3370441503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:29:48.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:48.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:47 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:29:48.445 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:48.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:48.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:48.550 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:48.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:48.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:48.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:48.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:48.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:48.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:48.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:48.916 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:48.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:48.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:49.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:49.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.054 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.054 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:49.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:49.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.199 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:49.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:48 smithi173 ceph-mon[33187]: pgmap v893: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 790 B/s rd, 45 MiB/s wr, 783 op/s 2024-09-17T08:29:49.236 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:48 smithi079 ceph-mon[29565]: pgmap v893: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 790 B/s rd, 45 MiB/s wr, 783 op/s 2024-09-17T08:29:49.236 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:48 smithi079 ceph-mon[40649]: pgmap v893: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 790 B/s rd, 45 MiB/s wr, 783 op/s 2024-09-17T08:29:49.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:49.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:49.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:49.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:49.453 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:49.555 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:49.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:49.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:49.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:49.705 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:49.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:49.840 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:49.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:49.978 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:49.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:49.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:50.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:50.118 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:50.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:50.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:50.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:50.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.413 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:50.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:50.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:50.545 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:50.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:50.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:50.674 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:50.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:50.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:50.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.809 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:50.810 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:50.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:50.943 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:50.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:50.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:51.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:51.078 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:51.180 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:51.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:51.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:50 smithi173 ceph-mon[33187]: pgmap v894: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 51 MiB/s wr, 1.16k op/s 2024-09-17T08:29:51.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:51.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:51.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:51.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:50 smithi079 ceph-mon[40649]: pgmap v894: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 51 MiB/s wr, 1.16k op/s 2024-09-17T08:29:51.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:50 smithi079 ceph-mon[29565]: pgmap v894: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 51 MiB/s wr, 1.16k op/s 2024-09-17T08:29:51.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:51.484 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:51.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:51.586 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:51.613 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:51.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:51.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:51.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:51.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:51.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:51.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:51.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:51.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:51.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:52.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:52.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(osd.6.0:30385 11.3 e387/359)", 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105474+0000", 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00052971699999999997, 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00055203299999999995, 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.108 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105474+0000", 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105474+0000", 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105473+0000", 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105476+0000", 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9909999999999998e-06 2024-09-17T08:29:52.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105483+0000", 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9349999999999997e-06 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105488+0000", 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4109999999999998e-06 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(osd.6.0:30386 11.3 e387/359)", 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105492+0000", 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00051188399999999999, 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000580089, 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105492+0000", 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105492+0000", 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105492+0000", 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105493+0000", 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.747e-06 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105498+0000", 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4220000000000002e-06 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105501+0000", 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.388e-06 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.114 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(osd.6.0:30387 11.3 e387/359)", 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105504+0000", 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00049961999999999999, 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00060804900000000005, 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105504+0000", 2024-09-17T08:29:52.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105504+0000", 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105504+0000", 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105505+0000", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5370000000000001e-06 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105510+0000", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4039999999999997e-06 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105513+0000", 2024-09-17T08:29:52.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.529e-06 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135840 11.3 11:c086c4b5:::benchmark_data_smithi079_199625_object6962:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.052433+0000", 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.053570679000000003, 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.053726569000000002, 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.118 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135840 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052433+0000", 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052433+0000", 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052431+0000", 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052441+0000", 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5750000000000007e-06 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052452+0000", 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1134999999999999e-05 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.052458+0000", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3539999999999996e-06 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.092291+0000", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.039832514999999999 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.092410+0000", 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119608 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.092541+0000", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013046999999999999 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.097927+0000", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0053866749999999996 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.103569+0000", 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056412459999999999 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.122 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.123 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135811 11.3 11:ca1e1165:::benchmark_data_smithi079_199625_object6933:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.029889+0000", 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.076114525000000002, 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.076350003, 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135811 2024-09-17T08:29:52.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029889+0000", 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029889+0000", 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029888+0000", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029895+0000", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9929999999999998e-06 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029909+0000", 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4555999999999999e-05 2024-09-17T08:29:52.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.029916+0000", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9319999999999999e-06 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.084247+0000", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.054330615999999998 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.084394+0000", 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000147465 2024-09-17T08:29:52.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.084533+0000", 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013905100000000001 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.092147+0000", 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0076137870000000003 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:29:52.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.092269+0000", 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000122065 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135866 11.3 11:c8fa16d4:::benchmark_data_smithi079_199625_object6988:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.083055+0000", 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.022948611000000001, 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023269451999999999, 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.129 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135866 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083055+0000", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083055+0000", 2024-09-17T08:29:52.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083054+0000", 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083060+0000", 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5719999999999999e-06 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083072+0000", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1878e-05 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.083079+0000", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.311e-06 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.100792+0000", 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.017713069000000001 2024-09-17T08:29:52.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.100916+0000", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012333600000000001 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.101035+0000", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119116 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.103668+0000", 2024-09-17T08:29:52.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0026332790000000001 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135845 11.3 11:ccc401af:::benchmark_data_smithi079_199625_object6967:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.056403+0000", 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.049600900000000003, 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.049992714000000001, 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.134 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135845 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056403+0000", 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056403+0000", 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056401+0000", 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056406+0000", 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1440000000000002e-06 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056415+0000", 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5250000000000005e-06 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.056419+0000", 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1339999999999997e-06 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.095177+0000", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.038757683000000001 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.095286+0000", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010908600000000001 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.095394+0000", 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108761 2024-09-17T08:29:52.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.100768+0000", 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0053732989999999998 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.103595+0000", 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028269549999999999 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135854 11.3 11:c9a24887:::benchmark_data_smithi079_199625_object6976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.063184+0000", 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.042819207999999997, 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.043288726, 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135854 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.140 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063184+0000", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063184+0000", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063183+0000", 2024-09-17T08:29:52.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063189+0000", 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3709999999999996e-06 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063206+0000", 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6481999999999999e-05 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.063213+0000", 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2450000000000001e-06 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.097953+0000", 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.034739550000000001 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.098072+0000", 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119418 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.098224+0000", 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015216200000000001 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.103660+0000", 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005435779 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.144 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:135886 11.9 e387/359 11:922df347:::benchmark_data_smithi079_199625_object7008:head v 387'8511, mlcod=387'8509)", 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.104687+0000", 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0013168240000000001, 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001855022, 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104687+0000", 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.145 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104687+0000", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104686+0000", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104694+0000", 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.303e-06 2024-09-17T08:29:52.146 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104708+0000", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3619e-05 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104714+0000", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2559999999999999e-06 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104771+0000", 2024-09-17T08:29:52.147 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7240000000000001e-05 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104790+0000", 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8864000000000001e-05 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135889 11.3 11:cfa19086:::benchmark_data_smithi079_199625_object7011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.104819+0000", 2024-09-17T08:29:52.148 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.001184137, 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017751729999999999, 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135889 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104819+0000", 2024-09-17T08:29:52.149 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104819+0000", 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104818+0000", 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.150 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104824+0000", 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1020000000000002e-06 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104844+0000", 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.022e-05 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104851+0000", 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0949999999999998e-06 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.151 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104904+0000", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2488000000000003e-05 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105023+0000", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119442 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105152+0000", 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012848800000000001 2024-09-17T08:29:52.152 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:135888 11.9 e387/359)", 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.104908+0000", 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.0010951489999999999, 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0017453869999999999, 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.153 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104908+0000", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104908+0000", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104908+0000", 2024-09-17T08:29:52.154 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104915+0000", 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0690000000000004e-06 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104925+0000", 2024-09-17T08:29:52.155 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0669000000000001e-05 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.104932+0000", 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6969999999999996e-06 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:135837 11.3 11:c75b6ea0:::benchmark_data_smithi079_199625_object6959:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.047340+0000", 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.058663446000000001, 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.059358312000000003, 2024-09-17T08:29:52.156 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 135837 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047340+0000", 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.157 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047340+0000", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047339+0000", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047344+0000", 2024-09-17T08:29:52.158 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6380000000000003e-06 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047354+0000", 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0095000000000001e-05 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.047361+0000", 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8789999999999997e-06 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.159 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.087994+0000", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040633048999999997 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.088117+0000", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012337099999999999 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.088240+0000", 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012226699999999999 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.160 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.097878+0000", 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0096386260000000008 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.097920+0000", 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1906999999999997e-05 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.161 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.25757.0:135854 11.3 e387/359)", 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105119+0000", 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00088493999999999997, 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001649654, 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105119+0000", 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.162 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105119+0000", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105118+0000", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105122+0000", 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2660000000000003e-06 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.163 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105130+0000", 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2220000000000002e-06 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105136+0000", 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5219999999999998e-06 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.164 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.25757.0:135866 11.3 e387/359)", 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105217+0000", 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00078684100000000004, 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001590803, 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105217+0000", 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.165 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105217+0000", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105216+0000", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105220+0000", 2024-09-17T08:29:52.166 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9809999999999997e-06 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105229+0000", 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4579999999999993e-06 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105234+0000", 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5339999999999999e-06 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(osd.6.0:30384 11.3 e387/359)", 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:52.105403+0000", 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 0.00060089200000000001, 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001444374, 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:29:52.168 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "queued for pg", 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105403+0000", 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105403+0000", 2024-09-17T08:29:52.169 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105402+0000", 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105405+0000", 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3569999999999998e-06 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:29:52.170 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105414+0000", 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7509999999999997e-06 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:52.105419+0000", 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9570000000000001e-06 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: ], 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 15 2024-09-17T08:29:52.171 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:52.202 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:52.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:52.303 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:52.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:52.351 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:52.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:52.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:52.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:52.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:52.630 INFO:tasks.thrashosds.thrasher:j is {'epoch': 387, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:29:43.900118+0000', 'last_up_change': '2024-09-17T08:20:22.465407+0000', 'last_in_change': '2024-09-17T08:15:28.332713+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '334', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '332', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 1.909999966621399, 'score_stable': 1.909999966621399, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-17T08:27:59.045325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '364', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8736248016357422, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 18, 'last_clean_end': 204, 'up_from': 227, 'up_thru': 359, 'down_at': 205, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 359954782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 359954782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 359954782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 359954782}]}, 'public_addr': '172.21.15.79:6819/359954782', 'cluster_addr': '172.21.15.79:6821/359954782', 'heartbeat_back_addr': '172.21.15.79:6825/359954782', 'heartbeat_front_addr': '172.21.15.79:6823/359954782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 377, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 205, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9889984130859375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 359, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:29:43.758608+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:28:44.692969+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:29:03.550764+0000', 'dead_epoch': 204}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:29:38.327894+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:28:58.738432+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:29:39.842238+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:28:35.603545+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:29:08.061818+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '5.2', 'osds': [5, 6, 2]}], 'pg_upmap_items': [{'pgid': '2.f', 'mappings': [{'from': 7, 'to': 2}, {'from': 4, 'to': 0}]}, {'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 2}, {'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:29:52.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:29:52.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.661 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:52.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:52.762 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:52.791 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:52.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:52.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:52 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:52.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:52 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:52.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:52.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:52.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:52.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:52 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:29:53.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:53.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.059 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:53.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:53.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:29:53.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:53.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.200 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:53.200 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:53.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:53.317 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:29:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:29:53.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:53.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:53.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:53.451 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:53.468 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:53.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:53.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:53.575 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:29:53.621 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[29565]: pgmap v895: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 44 MiB/s wr, 1.02k op/s 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[40649]: pgmap v895: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 44 MiB/s wr, 1.02k op/s 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:53.622 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:53 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:53.625 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:53.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:53.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:29:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:29:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:53 smithi173 ceph-mon[33187]: pgmap v895: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.7 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 44 MiB/s wr, 1.02k op/s 2024-09-17T08:29:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:53 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:29:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:53 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:29:53.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:53 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:29:53.728 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:53.763 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:53.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:53.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:53.906 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:53.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:53.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:54.007 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:54.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:54.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.045 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:54.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:54.175 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:54.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:54.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:54.310 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:54.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:54.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:54.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:54.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:54.582 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:29:54.583 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:54.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:54.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:54.713 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:54.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:54.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:54.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:54.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:54.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:54.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:29:54.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:29:54.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:54.998 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:54.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:54.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:55.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:55.128 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:55.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:55.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:55.262 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:55.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:55.364 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:55.393 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:55.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:55.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:55.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:55.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:55.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: pgmap v896: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.04k op/s 2024-09-17T08:29:55.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:55.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:55.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T08:29:55.574 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:55.575 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:55.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:55.684 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:55.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:55.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: pgmap v896: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.04k op/s 2024-09-17T08:29:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T08:29:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:55 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:55.786 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:55.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.830 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:55.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:55.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: pgmap v896: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.04k op/s 2024-09-17T08:29:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:29:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3169447550' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:29:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T08:29:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:55 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:55.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:55.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:55.963 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:55.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:56.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:56.110 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:56.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:56.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:56.258 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:56.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:56.360 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:56.393 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:56.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:56.425 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:29:56.425 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:29:56.425 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:29:56.425 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:29:56.425 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:29:56.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:29:56.530 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:56.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:29:56.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:56.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:56.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:56 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:29:56.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:56 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:29:56.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:56 smithi173 ceph-mon[33187]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T08:29:56.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:56.794 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:56.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[29565]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:29:56.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:56 smithi079 ceph-mon[40649]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T08:29:56.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:56.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:56.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:56.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:57.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:57.073 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:57.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:57.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:29:57.175 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:57.217 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:57.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:57.319 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:57.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:57.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:57.437 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:29:57.456 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:57.498 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:29:57.498 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:29:57.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.500 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:57.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:57.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:29:57.529 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:29:57.562 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:29:57.562 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:29:57.595 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:29:57.595 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:29:57.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:29:57.628 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:29:57.628 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:29:57.639 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:57.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:29:57.656 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:29:57.656 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:29:57.688 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:29:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:29:57.726 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:29:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:29:57.741 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:57.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:57 smithi079 ceph-mon[29565]: pgmap v899: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-17T08:29:57.759 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:57 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/259318146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:57.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:57 smithi079 ceph-mon[40649]: pgmap v899: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-17T08:29:57.759 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:57 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/259318146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:57.771 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:57.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:57.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:29:57.899 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:57 smithi173 ceph-mon[33187]: pgmap v899: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-17T08:29:57.899 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:57 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/259318146' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:29:57.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:57.904 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:57.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:29:58.005 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:58.042 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:58.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:58.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:58.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:58.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:58.279 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:58.309 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:58.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:58.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:58.447 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:58.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:58.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:58.570 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:58.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:58.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:29:58.699 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:58.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:29:58.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:58.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:58.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:58.937 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:29:58.975 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:58.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:58.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:29:59.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:59.103 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:59.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:59.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:29:59.239 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:59.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:29:59.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:29:59.341 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:59.368 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:59.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:59.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:59.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.504 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:59.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:59.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:59.631 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:59.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:59.733 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:29:59.761 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:29:59.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:29:59.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:29:59.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:29:59 smithi079 ceph-mon[40649]: pgmap v900: 158 pgs: 158 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 810 op/s 2024-09-17T08:29:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:29:59 smithi079 ceph-mon[29565]: pgmap v900: 158 pgs: 158 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 810 op/s 2024-09-17T08:29:59.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:29:59.899 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:29:59.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:29:59.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:29:59 smithi173 ceph-mon[33187]: pgmap v900: 158 pgs: 158 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 810 op/s 2024-09-17T08:30:00.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:00.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:00.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:00.092 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:00.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:00.170 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:30:00.173 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:30:00.197 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:00.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:00.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:00.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.342 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:00.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:00.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:00.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:00.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:00.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:00.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.622 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:00.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:00.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:00 smithi173 ceph-mon[33187]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:00 smithi173 ceph-mon[33187]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.752 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:00.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:00.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:30:00.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:00 smithi079 ceph-mon[40649]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:00 smithi079 ceph-mon[40649]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:00 smithi079 ceph-mon[29565]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.799 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:00 smithi079 ceph-mon[29565]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-17T08:30:00.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:00.881 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:00.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:00.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:00.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:01.005 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:30:01.005 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:30:01.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:01.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:01.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:01.155 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:01.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:01.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:01.298 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:01.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:01.400 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:01.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.436 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:01.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:01.478 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":901,"stamp":"2024-09-17T08:29:59.658981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7517378862,"num_objects":114894,"num_object_clones":0,"num_object_copies":344682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114894,"num_whiteouts":0,"num_read":1998,"num_read_kb":1750,"num_write":143770,"num_write_kb":8248044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":4717,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81930,"ondisk_log_size":81930,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":27425224,"kb_used_data":22238576,"kb_used_omap":15,"kb_used_meta":5186544,"kb_avail":722535992,"statfs":{"total":767960285184,"available":739876855808,"internally_reserved":0,"allocated":22772301824,"data_stored":22769648050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15595,"internal_metadata":5311021845},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,5,5,10,4,17,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-729677824,"num_objects":-11134,"num_object_clones":0,"num_object_copies":-33402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11134,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11134,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002363"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":423,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584666+0000","last_change":"2024-09-17T08:29:18.942232+0000","last_active":"2024-09-17T08:29:54.584666+0000","last_peered":"2024-09-17T08:29:54.584666+0000","last_clean":"2024-09-17T08:29:54.584666+0000","last_became_active":"2024-09-17T08:10:00.599060+0000","last_became_peered":"2024-09-17T08:10:00.599060+0000","last_unstale":"2024-09-17T08:29:54.584666+0000","last_undegraded":"2024-09-17T08:29:54.584666+0000","last_fullsized":"2024-09-17T08:29:54.584666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:18.941837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:29:18.941837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:26.954479+0000","scrub_duration":0.0071331160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":442,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.578792+0000","last_change":"2024-09-17T08:29:29.655258+0000","last_active":"2024-09-17T08:29:54.578792+0000","last_peered":"2024-09-17T08:29:54.578792+0000","last_clean":"2024-09-17T08:29:54.578792+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:29:54.578792+0000","last_undegraded":"2024-09-17T08:29:54.578792+0000","last_fullsized":"2024-09-17T08:29:54.578792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:29.655116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:42.337157+0000","last_clean_scrub_stamp":"2024-09-17T08:29:29.655116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:32.117899+0000","scrub_duration":0.0023301870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"220'27","reported_seq":490,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582861+0000","last_change":"2024-09-17T08:28:29.665518+0000","last_active":"2024-09-17T08:29:54.582861+0000","last_peered":"2024-09-17T08:29:54.582861+0000","last_clean":"2024-09-17T08:29:54.582861+0000","last_became_active":"2024-09-17T08:20:23.484859+0000","last_became_peered":"2024-09-17T08:20:23.484859+0000","last_unstale":"2024-09-17T08:29:54.582861+0000","last_undegraded":"2024-09-17T08:29:54.582861+0000","last_fullsized":"2024-09-17T08:29:54.582861+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'27","last_scrub_stamp":"2024-09-17T08:28:29.665385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.665385+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.448791+0000","scrub_duration":0.0073568490000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":444,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574367+0000","last_change":"2024-09-17T08:28:41.281153+0000","last_active":"2024-09-17T08:29:54.574367+0000","last_peered":"2024-09-17T08:29:54.574367+0000","last_clean":"2024-09-17T08:29:54.574367+0000","last_became_active":"2024-09-17T08:15:29.362980+0000","last_became_peered":"2024-09-17T08:15:29.362980+0000","last_unstale":"2024-09-17T08:29:54.574367+0000","last_undegraded":"2024-09-17T08:29:54.574367+0000","last_fullsized":"2024-09-17T08:29:54.574367+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:41.281073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:41.281073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001166683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":420,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585513+0000","last_change":"2024-09-17T08:28:09.842951+0000","last_active":"2024-09-17T08:29:54.585513+0000","last_peered":"2024-09-17T08:29:54.585513+0000","last_clean":"2024-09-17T08:29:54.585513+0000","last_became_active":"2024-09-17T08:09:54.500728+0000","last_became_peered":"2024-09-17T08:09:54.500728+0000","last_unstale":"2024-09-17T08:29:54.585513+0000","last_undegraded":"2024-09-17T08:29:54.585513+0000","last_fullsized":"2024-09-17T08:29:54.585513+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:09.842851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:28:09.842851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001617739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"220'19","reported_seq":546,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574387+0000","last_change":"2024-09-17T08:27:38.219718+0000","last_active":"2024-09-17T08:29:54.574387+0000","last_peered":"2024-09-17T08:29:54.574387+0000","last_clean":"2024-09-17T08:29:54.574387+0000","last_became_active":"2024-09-17T08:20:23.477392+0000","last_became_peered":"2024-09-17T08:20:23.477392+0000","last_unstale":"2024-09-17T08:29:54.574387+0000","last_undegraded":"2024-09-17T08:29:54.574387+0000","last_fullsized":"2024-09-17T08:29:54.574387+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:27:38.219617+0000","last_deep_scrub":"87'13","last_deep_scrub_stamp":"2024-09-17T08:11:33.629197+0000","last_clean_scrub_stamp":"2024-09-17T08:27:38.219617+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021625220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":308,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577107+0000","last_change":"2024-09-17T08:28:16.352900+0000","last_active":"2024-09-17T08:29:54.577107+0000","last_peered":"2024-09-17T08:29:54.577107+0000","last_clean":"2024-09-17T08:29:54.577107+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:29:54.577107+0000","last_undegraded":"2024-09-17T08:29:54.577107+0000","last_fullsized":"2024-09-17T08:29:54.577107+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.352564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.352564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029315639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":442,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.593015+0000","last_change":"2024-09-17T08:28:16.236102+0000","last_active":"2024-09-17T08:29:55.593015+0000","last_peered":"2024-09-17T08:29:55.593015+0000","last_clean":"2024-09-17T08:29:55.593015+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:29:55.593015+0000","last_undegraded":"2024-09-17T08:29:55.593015+0000","last_fullsized":"2024-09-17T08:29:55.593015+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.235973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.235973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003218683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":440,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.598597+0000","last_change":"2024-09-17T08:28:38.384894+0000","last_active":"2024-09-17T08:29:55.598597+0000","last_peered":"2024-09-17T08:29:55.598597+0000","last_clean":"2024-09-17T08:29:55.598597+0000","last_became_active":"2024-09-17T08:09:54.506900+0000","last_became_peered":"2024-09-17T08:09:54.506900+0000","last_unstale":"2024-09-17T08:29:55.598597+0000","last_undegraded":"2024-09-17T08:29:55.598597+0000","last_fullsized":"2024-09-17T08:29:55.598597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:38.384792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:28:38.384792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022556690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"220'9","reported_seq":473,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.593056+0000","last_change":"2024-09-17T08:29:40.577859+0000","last_active":"2024-09-17T08:29:55.593056+0000","last_peered":"2024-09-17T08:29:55.593056+0000","last_clean":"2024-09-17T08:29:55.593056+0000","last_became_active":"2024-09-17T08:15:29.383610+0000","last_became_peered":"2024-09-17T08:15:29.383610+0000","last_unstale":"2024-09-17T08:29:55.593056+0000","last_undegraded":"2024-09-17T08:29:55.593056+0000","last_fullsized":"2024-09-17T08:29:55.593056+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'9","last_scrub_stamp":"2024-09-17T08:29:40.577730+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:29:40.577730+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:06.483466+0000","scrub_duration":0.014996302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":465,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.595735+0000","last_change":"2024-09-17T08:28:30.443367+0000","last_active":"2024-09-17T08:29:55.595735+0000","last_peered":"2024-09-17T08:29:55.595735+0000","last_clean":"2024-09-17T08:29:55.595735+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:29:55.595735+0000","last_undegraded":"2024-09-17T08:29:55.595735+0000","last_fullsized":"2024-09-17T08:29:55.595735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:30.443251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:22.423862+0000","last_clean_scrub_stamp":"2024-09-17T08:28:30.443251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042606730000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":319,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576970+0000","last_change":"2024-09-17T08:29:32.379233+0000","last_active":"2024-09-17T08:29:54.576970+0000","last_peered":"2024-09-17T08:29:54.576970+0000","last_clean":"2024-09-17T08:29:54.576970+0000","last_became_active":"2024-09-17T08:15:29.379131+0000","last_became_peered":"2024-09-17T08:15:29.379131+0000","last_unstale":"2024-09-17T08:29:54.576970+0000","last_undegraded":"2024-09-17T08:29:54.576970+0000","last_fullsized":"2024-09-17T08:29:54.576970+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:32.379108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:29:32.379108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:01.667860+0000","scrub_duration":0.0070768630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":455,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.596457+0000","last_change":"2024-09-17T08:28:29.478103+0000","last_active":"2024-09-17T08:29:55.596457+0000","last_peered":"2024-09-17T08:29:55.596457+0000","last_clean":"2024-09-17T08:29:55.596457+0000","last_became_active":"2024-09-17T08:20:23.489085+0000","last_became_peered":"2024-09-17T08:20:23.489085+0000","last_unstale":"2024-09-17T08:29:55.596457+0000","last_undegraded":"2024-09-17T08:29:55.596457+0000","last_fullsized":"2024-09-17T08:29:55.596457+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.477933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:01.908732+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.477933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027795519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"267'19","reported_seq":457,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.593178+0000","last_change":"2024-09-17T08:28:11.338708+0000","last_active":"2024-09-17T08:29:55.593178+0000","last_peered":"2024-09-17T08:29:55.593178+0000","last_clean":"2024-09-17T08:29:55.593178+0000","last_became_active":"2024-09-17T08:09:56.503127+0000","last_became_peered":"2024-09-17T08:09:56.503127+0000","last_unstale":"2024-09-17T08:29:55.593178+0000","last_undegraded":"2024-09-17T08:29:55.593178+0000","last_fullsized":"2024-09-17T08:29:55.593178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'19","last_scrub_stamp":"2024-09-17T08:28:11.338595+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:28:11.338595+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026129119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":328,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576564+0000","last_change":"2024-09-17T08:27:30.096764+0000","last_active":"2024-09-17T08:29:54.576564+0000","last_peered":"2024-09-17T08:29:54.576564+0000","last_clean":"2024-09-17T08:29:54.576564+0000","last_became_active":"2024-09-17T08:14:58.716760+0000","last_became_peered":"2024-09-17T08:14:58.716760+0000","last_unstale":"2024-09-17T08:29:54.576564+0000","last_undegraded":"2024-09-17T08:29:54.576564+0000","last_fullsized":"2024-09-17T08:29:54.576564+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:30.096633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:27:30.096633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048718219999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":453,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575987+0000","last_change":"2024-09-17T08:29:33.641078+0000","last_active":"2024-09-17T08:29:54.575987+0000","last_peered":"2024-09-17T08:29:54.575987+0000","last_clean":"2024-09-17T08:29:54.575987+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:29:54.575987+0000","last_undegraded":"2024-09-17T08:29:54.575987+0000","last_fullsized":"2024-09-17T08:29:54.575987+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:33.640953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:26.220022+0000","last_clean_scrub_stamp":"2024-09-17T08:29:33.640953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:47.357514+0000","scrub_duration":0.023004501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"389'8975","reported_seq":9182,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:56.891190+0000","last_change":"2024-09-17T08:29:37.811128+0000","last_active":"2024-09-17T08:29:56.891190+0000","last_peered":"2024-09-17T08:29:56.891190+0000","last_clean":"2024-09-17T08:29:56.891190+0000","last_became_active":"2024-09-17T08:28:00.528334+0000","last_became_peered":"2024-09-17T08:28:00.528334+0000","last_unstale":"2024-09-17T08:29:56.891190+0000","last_undegraded":"2024-09-17T08:29:56.891190+0000","last_fullsized":"2024-09-17T08:29:56.891190+0000","mapping_epoch":359,"log_start":"371'3400","ondisk_log_start":"371'3400","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'7721","last_scrub_stamp":"2024-09-17T08:29:37.810920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:29:37.810920+0000","objects_scrubbed":7644,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:58.107422+0000","scrub_duration":2.7748271309999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":488177664,"num_objects":7449,"num_object_clones":0,"num_object_copies":22347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8975,"num_write_kb":525568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'130","reported_seq":368,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585566+0000","last_change":"2024-09-17T08:29:16.920258+0000","last_active":"2024-09-17T08:29:54.585566+0000","last_peered":"2024-09-17T08:29:54.585566+0000","last_clean":"2024-09-17T08:29:54.585566+0000","last_became_active":"2024-09-17T08:26:56.652585+0000","last_became_peered":"2024-09-17T08:26:56.652585+0000","last_unstale":"2024-09-17T08:29:54.585566+0000","last_undegraded":"2024-09-17T08:29:54.585566+0000","last_fullsized":"2024-09-17T08:29:54.585566+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:29:16.920124+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:29:16.920124+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:24.588651+0000","scrub_duration":0.0032363380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"220'31","reported_seq":563,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575300+0000","last_change":"2024-09-17T08:29:32.287520+0000","last_active":"2024-09-17T08:29:54.575300+0000","last_peered":"2024-09-17T08:29:54.575300+0000","last_clean":"2024-09-17T08:29:54.575300+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:29:54.575300+0000","last_undegraded":"2024-09-17T08:29:54.575300+0000","last_fullsized":"2024-09-17T08:29:54.575300+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'31","last_scrub_stamp":"2024-09-17T08:29:32.287324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:32.287324+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:35.351770+0000","scrub_duration":0.012142918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":451,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.579867+0000","last_change":"2024-09-17T08:29:29.461826+0000","last_active":"2024-09-17T08:29:54.579867+0000","last_peered":"2024-09-17T08:29:54.579867+0000","last_clean":"2024-09-17T08:29:54.579867+0000","last_became_active":"2024-09-17T08:09:54.503643+0000","last_became_peered":"2024-09-17T08:09:54.503643+0000","last_unstale":"2024-09-17T08:29:54.579867+0000","last_undegraded":"2024-09-17T08:29:54.579867+0000","last_fullsized":"2024-09-17T08:29:54.579867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:29.461740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:29:29.461740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:36.989620+0000","scrub_duration":0.0014828809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.579378+0000","last_change":"2024-09-17T08:28:26.403888+0000","last_active":"2024-09-17T08:29:54.579378+0000","last_peered":"2024-09-17T08:29:54.579378+0000","last_clean":"2024-09-17T08:29:54.579378+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:29:54.579378+0000","last_undegraded":"2024-09-17T08:29:54.579378+0000","last_fullsized":"2024-09-17T08:29:54.579378+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:26.403765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.403765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072874840000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":420,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.589965+0000","last_change":"2024-09-17T08:28:20.191865+0000","last_active":"2024-09-17T08:29:54.589965+0000","last_peered":"2024-09-17T08:29:54.589965+0000","last_clean":"2024-09-17T08:29:54.589965+0000","last_became_active":"2024-09-17T08:20:23.487157+0000","last_became_peered":"2024-09-17T08:20:23.487157+0000","last_unstale":"2024-09-17T08:29:54.589965+0000","last_undegraded":"2024-09-17T08:29:54.589965+0000","last_fullsized":"2024-09-17T08:29:54.589965+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:20.191734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.191734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041900449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"389'11797","reported_seq":12152,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.594472+0000","last_change":"2024-09-17T08:29:54.606960+0000","last_active":"2024-09-17T08:29:57.594472+0000","last_peered":"2024-09-17T08:29:57.594472+0000","last_clean":"2024-09-17T08:29:57.594472+0000","last_became_active":"2024-09-17T08:28:00.527680+0000","last_became_peered":"2024-09-17T08:28:00.527680+0000","last_unstale":"2024-09-17T08:29:57.594472+0000","last_undegraded":"2024-09-17T08:29:57.594472+0000","last_fullsized":"2024-09-17T08:29:57.594472+0000","mapping_epoch":359,"log_start":"382'6602","ondisk_log_start":"382'6602","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":3796,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":478216192,"num_objects":7297,"num_object_clones":0,"num_object_copies":21891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8903,"num_write_kb":518400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'130","reported_seq":741,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580482+0000","last_change":"2024-09-17T08:29:35.531220+0000","last_active":"2024-09-17T08:29:54.580482+0000","last_peered":"2024-09-17T08:29:54.580482+0000","last_clean":"2024-09-17T08:29:54.580482+0000","last_became_active":"2024-09-17T08:26:56.652142+0000","last_became_peered":"2024-09-17T08:26:56.652142+0000","last_unstale":"2024-09-17T08:29:54.580482+0000","last_undegraded":"2024-09-17T08:29:54.580482+0000","last_fullsized":"2024-09-17T08:29:54.580482+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:29:35.531105+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:29:35.531105+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:40.954299+0000","scrub_duration":0.0050196399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"367'25","reported_seq":463,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574994+0000","last_change":"2024-09-17T08:28:25.297611+0000","last_active":"2024-09-17T08:29:54.574994+0000","last_peered":"2024-09-17T08:29:54.574994+0000","last_clean":"2024-09-17T08:29:54.574994+0000","last_became_active":"2024-09-17T08:09:56.503324+0000","last_became_peered":"2024-09-17T08:09:56.503324+0000","last_unstale":"2024-09-17T08:29:54.574994+0000","last_undegraded":"2024-09-17T08:29:54.574994+0000","last_fullsized":"2024-09-17T08:29:54.574994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'25","last_scrub_stamp":"2024-09-17T08:28:25.297479+0000","last_deep_scrub":"367'25","last_deep_scrub_stamp":"2024-09-17T08:28:25.297479+0000","last_clean_scrub_stamp":"2024-09-17T08:28:25.297479+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095790600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":309,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577593+0000","last_change":"2024-09-17T08:28:15.377573+0000","last_active":"2024-09-17T08:29:54.577593+0000","last_peered":"2024-09-17T08:29:54.577593+0000","last_clean":"2024-09-17T08:29:54.577593+0000","last_became_active":"2024-09-17T08:15:29.376635+0000","last_became_peered":"2024-09-17T08:15:29.376635+0000","last_unstale":"2024-09-17T08:29:54.577593+0000","last_undegraded":"2024-09-17T08:29:54.577593+0000","last_fullsized":"2024-09-17T08:29:54.577593+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:15.377472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:28:15.377472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001907822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"228'11","reported_seq":639,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.589700+0000","last_change":"2024-09-17T08:28:39.178481+0000","last_active":"2024-09-17T08:29:54.589700+0000","last_peered":"2024-09-17T08:29:54.589700+0000","last_clean":"2024-09-17T08:29:54.589700+0000","last_became_active":"2024-09-17T08:20:23.478902+0000","last_became_peered":"2024-09-17T08:20:23.478902+0000","last_unstale":"2024-09-17T08:29:54.589700+0000","last_undegraded":"2024-09-17T08:29:54.589700+0000","last_fullsized":"2024-09-17T08:29:54.589700+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:28:39.178203+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.178203+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010732917,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":447,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575159+0000","last_change":"2024-09-17T08:28:35.289687+0000","last_active":"2024-09-17T08:29:54.575159+0000","last_peered":"2024-09-17T08:29:54.575159+0000","last_clean":"2024-09-17T08:29:54.575159+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:29:54.575159+0000","last_undegraded":"2024-09-17T08:29:54.575159+0000","last_fullsized":"2024-09-17T08:29:54.575159+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.289539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.289539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086312880000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"389'8932","reported_seq":8969,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.033500+0000","last_change":"2024-09-17T08:28:00.529640+0000","last_active":"2024-09-17T08:29:59.033500+0000","last_peered":"2024-09-17T08:29:59.033500+0000","last_clean":"2024-09-17T08:29:59.033500+0000","last_became_active":"2024-09-17T08:28:00.529365+0000","last_became_peered":"2024-09-17T08:28:00.529365+0000","last_unstale":"2024-09-17T08:29:59.033500+0000","last_undegraded":"2024-09-17T08:29:59.033500+0000","last_fullsized":"2024-09-17T08:29:59.033500+0000","mapping_epoch":359,"log_start":"373'4000","ondisk_log_start":"373'4000","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4932,"log_dups_size":0,"ondisk_log_size":4932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461242368,"num_objects":7038,"num_object_clones":0,"num_object_copies":21114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8932,"num_write_kb":511040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"75'130","reported_seq":369,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585579+0000","last_change":"2024-09-17T08:29:25.938601+0000","last_active":"2024-09-17T08:29:54.585579+0000","last_peered":"2024-09-17T08:29:54.585579+0000","last_clean":"2024-09-17T08:29:54.585579+0000","last_became_active":"2024-09-17T08:26:56.658579+0000","last_became_peered":"2024-09-17T08:26:56.658579+0000","last_unstale":"2024-09-17T08:29:54.585579+0000","last_undegraded":"2024-09-17T08:29:54.585579+0000","last_fullsized":"2024-09-17T08:29:54.585579+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:29:25.938240+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:29:25.938240+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:35.360700+0000","scrub_duration":0.0055357510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"220'34","reported_seq":322,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.589474+0000","last_change":"2024-09-17T08:29:23.968926+0000","last_active":"2024-09-17T08:29:54.589474+0000","last_peered":"2024-09-17T08:29:54.589474+0000","last_clean":"2024-09-17T08:29:54.589474+0000","last_became_active":"2024-09-17T08:20:23.489818+0000","last_became_peered":"2024-09-17T08:20:23.489818+0000","last_unstale":"2024-09-17T08:29:54.589474+0000","last_undegraded":"2024-09-17T08:29:54.589474+0000","last_fullsized":"2024-09-17T08:29:54.589474+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:29:23.968603+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:19.750582+0000","last_clean_scrub_stamp":"2024-09-17T08:29:23.968603+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:31.564239+0000","scrub_duration":0.007522913,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":309,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577039+0000","last_change":"2024-09-17T08:27:14.214573+0000","last_active":"2024-09-17T08:29:54.577039+0000","last_peered":"2024-09-17T08:29:54.577039+0000","last_clean":"2024-09-17T08:29:54.577039+0000","last_became_active":"2024-09-17T08:15:29.387480+0000","last_became_peered":"2024-09-17T08:15:29.387480+0000","last_unstale":"2024-09-17T08:29:54.577039+0000","last_undegraded":"2024-09-17T08:29:54.577039+0000","last_fullsized":"2024-09-17T08:29:54.577039+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:14.214464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:14.214464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002397092,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":675,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575252+0000","last_change":"2024-09-17T08:29:24.320640+0000","last_active":"2024-09-17T08:29:54.575252+0000","last_peered":"2024-09-17T08:29:54.575252+0000","last_clean":"2024-09-17T08:29:54.575252+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:29:54.575252+0000","last_undegraded":"2024-09-17T08:29:54.575252+0000","last_fullsized":"2024-09-17T08:29:54.575252+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:29:24.320565+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:21:35.884516+0000","last_clean_scrub_stamp":"2024-09-17T08:29:24.320565+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:26.930034+0000","scrub_duration":0.00138751,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":309,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.578120+0000","last_change":"2024-09-17T08:27:25.238121+0000","last_active":"2024-09-17T08:29:54.578120+0000","last_peered":"2024-09-17T08:29:54.578120+0000","last_clean":"2024-09-17T08:29:54.578120+0000","last_became_active":"2024-09-17T08:15:29.368192+0000","last_became_peered":"2024-09-17T08:15:29.368192+0000","last_unstale":"2024-09-17T08:29:54.578120+0000","last_undegraded":"2024-09-17T08:29:54.578120+0000","last_fullsized":"2024-09-17T08:29:54.578120+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:25.238008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.238008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.074631296999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"389'8837","reported_seq":8970,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.637427+0000","last_change":"2024-09-17T08:29:40.948517+0000","last_active":"2024-09-17T08:29:57.637427+0000","last_peered":"2024-09-17T08:29:57.637427+0000","last_clean":"2024-09-17T08:29:57.637427+0000","last_became_active":"2024-09-17T08:28:00.529491+0000","last_became_peered":"2024-09-17T08:28:00.529491+0000","last_unstale":"2024-09-17T08:29:57.637427+0000","last_undegraded":"2024-09-17T08:29:57.637427+0000","last_fullsized":"2024-09-17T08:29:57.637427+0000","mapping_epoch":359,"log_start":"372'3600","ondisk_log_start":"372'3600","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":4449,"log_size":5237,"log_dups_size":0,"ondisk_log_size":5237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463536128,"num_objects":7073,"num_object_clones":0,"num_object_copies":21219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8837,"num_write_kb":509120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"220'24","reported_seq":507,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575293+0000","last_change":"2024-09-17T08:29:35.221921+0000","last_active":"2024-09-17T08:29:54.575293+0000","last_peered":"2024-09-17T08:29:54.575293+0000","last_clean":"2024-09-17T08:29:54.575293+0000","last_became_active":"2024-09-17T08:15:29.362034+0000","last_became_peered":"2024-09-17T08:15:29.362034+0000","last_unstale":"2024-09-17T08:29:54.575293+0000","last_undegraded":"2024-09-17T08:29:54.575293+0000","last_fullsized":"2024-09-17T08:29:54.575293+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'24","last_scrub_stamp":"2024-09-17T08:29:35.221792+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:29:35.221792+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:48.668511+0000","scrub_duration":0.0064767949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":448,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580211+0000","last_change":"2024-09-17T08:28:18.569946+0000","last_active":"2024-09-17T08:29:54.580211+0000","last_peered":"2024-09-17T08:29:54.580211+0000","last_clean":"2024-09-17T08:29:54.580211+0000","last_became_active":"2024-09-17T08:15:29.362964+0000","last_became_peered":"2024-09-17T08:15:29.362964+0000","last_unstale":"2024-09-17T08:29:54.580211+0000","last_undegraded":"2024-09-17T08:29:54.580211+0000","last_fullsized":"2024-09-17T08:29:54.580211+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.569818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.569818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054611859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":319,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575273+0000","last_change":"2024-09-17T08:27:21.239377+0000","last_active":"2024-09-17T08:29:54.575273+0000","last_peered":"2024-09-17T08:29:54.575273+0000","last_clean":"2024-09-17T08:29:54.575273+0000","last_became_active":"2024-09-17T08:14:58.716611+0000","last_became_peered":"2024-09-17T08:14:58.716611+0000","last_unstale":"2024-09-17T08:29:54.575273+0000","last_undegraded":"2024-09-17T08:29:54.575273+0000","last_fullsized":"2024-09-17T08:29:54.575273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:21.239266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:27:21.239266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":413,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584599+0000","last_change":"2024-09-17T08:28:21.692398+0000","last_active":"2024-09-17T08:29:54.584599+0000","last_peered":"2024-09-17T08:29:54.584599+0000","last_clean":"2024-09-17T08:29:54.584599+0000","last_became_active":"2024-09-17T08:10:00.597698+0000","last_became_peered":"2024-09-17T08:10:00.597698+0000","last_unstale":"2024-09-17T08:29:54.584599+0000","last_undegraded":"2024-09-17T08:29:54.584599+0000","last_fullsized":"2024-09-17T08:29:54.584599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:21.692268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071970940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"389'8911","reported_seq":8950,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.796232+0000","last_change":"2024-09-17T08:28:00.528378+0000","last_active":"2024-09-17T08:29:57.796232+0000","last_peered":"2024-09-17T08:29:57.796232+0000","last_clean":"2024-09-17T08:29:57.796232+0000","last_became_active":"2024-09-17T08:28:00.527934+0000","last_became_peered":"2024-09-17T08:28:00.527934+0000","last_unstale":"2024-09-17T08:29:57.796232+0000","last_undegraded":"2024-09-17T08:29:57.796232+0000","last_fullsized":"2024-09-17T08:29:57.796232+0000","mapping_epoch":359,"log_start":"373'3900","ondisk_log_start":"373'3900","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464715776,"num_objects":7091,"num_object_clones":0,"num_object_copies":21273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8911,"num_write_kb":512064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'130","reported_seq":358,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584467+0000","last_change":"2024-09-17T08:28:22.668840+0000","last_active":"2024-09-17T08:29:54.584467+0000","last_peered":"2024-09-17T08:29:54.584467+0000","last_clean":"2024-09-17T08:29:54.584467+0000","last_became_active":"2024-09-17T08:26:56.649122+0000","last_became_peered":"2024-09-17T08:26:56.649122+0000","last_unstale":"2024-09-17T08:29:54.584467+0000","last_undegraded":"2024-09-17T08:29:54.584467+0000","last_fullsized":"2024-09-17T08:29:54.584467+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.668701+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073890500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"349'35","reported_seq":467,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574564+0000","last_change":"2024-09-17T08:27:39.245204+0000","last_active":"2024-09-17T08:29:54.574564+0000","last_peered":"2024-09-17T08:29:54.574564+0000","last_clean":"2024-09-17T08:29:54.574564+0000","last_became_active":"2024-09-17T08:09:56.501408+0000","last_became_peered":"2024-09-17T08:09:56.501408+0000","last_unstale":"2024-09-17T08:29:54.574564+0000","last_undegraded":"2024-09-17T08:29:54.574564+0000","last_fullsized":"2024-09-17T08:29:54.574564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'35","last_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_deep_scrub":"349'35","last_deep_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_clean_scrub_stamp":"2024-09-17T08:27:39.245084+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057119750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":306,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.598597+0000","last_change":"2024-09-17T08:28:16.192010+0000","last_active":"2024-09-17T08:29:55.598597+0000","last_peered":"2024-09-17T08:29:55.598597+0000","last_clean":"2024-09-17T08:29:55.598597+0000","last_became_active":"2024-09-17T08:27:47.563576+0000","last_became_peered":"2024-09-17T08:27:47.563576+0000","last_unstale":"2024-09-17T08:29:55.598597+0000","last_undegraded":"2024-09-17T08:29:55.598597+0000","last_fullsized":"2024-09-17T08:29:55.598597+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:16.191909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:28:16.191909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002546974,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"176'11","reported_seq":708,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.601987+0000","last_change":"2024-09-17T08:29:37.007716+0000","last_active":"2024-09-17T08:29:55.601987+0000","last_peered":"2024-09-17T08:29:55.601987+0000","last_clean":"2024-09-17T08:29:55.601987+0000","last_became_active":"2024-09-17T08:15:29.383777+0000","last_became_peered":"2024-09-17T08:15:29.383777+0000","last_unstale":"2024-09-17T08:29:55.601987+0000","last_undegraded":"2024-09-17T08:29:55.601987+0000","last_fullsized":"2024-09-17T08:29:55.601987+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'11","last_scrub_stamp":"2024-09-17T08:29:37.007454+0000","last_deep_scrub":"176'11","last_deep_scrub_stamp":"2024-09-17T08:29:37.007454+0000","last_clean_scrub_stamp":"2024-09-17T08:29:37.007454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:56.438485+0000","scrub_duration":0.012265297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":451,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580378+0000","last_change":"2024-09-17T08:29:20.388389+0000","last_active":"2024-09-17T08:29:54.580378+0000","last_peered":"2024-09-17T08:29:54.580378+0000","last_clean":"2024-09-17T08:29:54.580378+0000","last_became_active":"2024-09-17T08:20:23.483478+0000","last_became_peered":"2024-09-17T08:20:23.483478+0000","last_unstale":"2024-09-17T08:29:54.580378+0000","last_undegraded":"2024-09-17T08:29:54.580378+0000","last_fullsized":"2024-09-17T08:29:54.580378+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:20.388302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:35.495779+0000","last_clean_scrub_stamp":"2024-09-17T08:29:20.388302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:23.295836+0000","scrub_duration":0.0046718990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"389'8973","reported_seq":9172,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.920762+0000","last_change":"2024-09-17T08:29:33.740886+0000","last_active":"2024-09-17T08:29:57.920762+0000","last_peered":"2024-09-17T08:29:57.920762+0000","last_clean":"2024-09-17T08:29:57.920762+0000","last_became_active":"2024-09-17T08:28:00.530036+0000","last_became_peered":"2024-09-17T08:28:00.530036+0000","last_unstale":"2024-09-17T08:29:57.920762+0000","last_undegraded":"2024-09-17T08:29:57.920762+0000","last_fullsized":"2024-09-17T08:29:57.920762+0000","mapping_epoch":359,"log_start":"373'4300","ondisk_log_start":"373'4300","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'7391","last_scrub_stamp":"2024-09-17T08:29:33.740666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:29:33.740666+0000","objects_scrubbed":7309,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:42.200063+0000","scrub_duration":2.2591419660000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474939392,"num_objects":7247,"num_object_clones":0,"num_object_copies":21741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8973,"num_write_kb":519040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'130","reported_seq":357,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574940+0000","last_change":"2024-09-17T08:28:26.291855+0000","last_active":"2024-09-17T08:29:54.574940+0000","last_peered":"2024-09-17T08:29:54.574940+0000","last_clean":"2024-09-17T08:29:54.574940+0000","last_became_active":"2024-09-17T08:26:56.649679+0000","last_became_peered":"2024-09-17T08:26:56.649679+0000","last_unstale":"2024-09-17T08:29:54.574940+0000","last_undegraded":"2024-09-17T08:29:54.574940+0000","last_fullsized":"2024-09-17T08:29:54.574940+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:26.291665+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.291665+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00173146,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"220'30","reported_seq":479,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582933+0000","last_change":"2024-09-17T08:28:26.667711+0000","last_active":"2024-09-17T08:29:54.582933+0000","last_peered":"2024-09-17T08:29:54.582933+0000","last_clean":"2024-09-17T08:29:54.582933+0000","last_became_active":"2024-09-17T08:09:57.043229+0000","last_became_peered":"2024-09-17T08:09:57.043229+0000","last_unstale":"2024-09-17T08:29:54.582933+0000","last_undegraded":"2024-09-17T08:29:54.582933+0000","last_fullsized":"2024-09-17T08:29:54.582933+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:28:26.666614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.666614+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013055815,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580430+0000","last_change":"2024-09-17T08:29:37.364874+0000","last_active":"2024-09-17T08:29:54.580430+0000","last_peered":"2024-09-17T08:29:54.580430+0000","last_clean":"2024-09-17T08:29:54.580430+0000","last_became_active":"2024-09-17T08:15:29.382723+0000","last_became_peered":"2024-09-17T08:15:29.382723+0000","last_unstale":"2024-09-17T08:29:54.580430+0000","last_undegraded":"2024-09-17T08:29:54.580430+0000","last_fullsized":"2024-09-17T08:29:54.580430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:37.364612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:29:37.364612+0000","last_clean_scrub_stamp":"2024-09-17T08:29:37.364612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:45.031004+0000","scrub_duration":0.010427674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":427,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574385+0000","last_change":"2024-09-17T08:28:37.288508+0000","last_active":"2024-09-17T08:29:54.574385+0000","last_peered":"2024-09-17T08:29:54.574385+0000","last_clean":"2024-09-17T08:29:54.574385+0000","last_became_active":"2024-09-17T08:09:58.504430+0000","last_became_peered":"2024-09-17T08:09:58.504430+0000","last_unstale":"2024-09-17T08:29:54.574385+0000","last_undegraded":"2024-09-17T08:29:54.574385+0000","last_fullsized":"2024-09-17T08:29:54.574385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:37.288379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:28:37.288379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:04.707593+0000","scrub_duration":0.0062446639999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":440,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574756+0000","last_change":"2024-09-17T08:28:23.270750+0000","last_active":"2024-09-17T08:29:54.574756+0000","last_peered":"2024-09-17T08:29:54.574756+0000","last_clean":"2024-09-17T08:29:54.574756+0000","last_became_active":"2024-09-17T08:20:23.481844+0000","last_became_peered":"2024-09-17T08:20:23.481844+0000","last_unstale":"2024-09-17T08:29:54.574756+0000","last_undegraded":"2024-09-17T08:29:54.574756+0000","last_fullsized":"2024-09-17T08:29:54.574756+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:23.270616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:23.270616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048781409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"389'9028","reported_seq":9066,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.052039+0000","last_change":"2024-09-17T08:28:00.523359+0000","last_active":"2024-09-17T08:29:59.052039+0000","last_peered":"2024-09-17T08:29:59.052039+0000","last_clean":"2024-09-17T08:29:59.052039+0000","last_became_active":"2024-09-17T08:28:00.523053+0000","last_became_peered":"2024-09-17T08:28:00.523053+0000","last_unstale":"2024-09-17T08:29:59.052039+0000","last_undegraded":"2024-09-17T08:29:59.052039+0000","last_fullsized":"2024-09-17T08:29:59.052039+0000","mapping_epoch":359,"log_start":"373'4100","ondisk_log_start":"373'4100","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4928,"log_dups_size":0,"ondisk_log_size":4928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469499904,"num_objects":7164,"num_object_clones":0,"num_object_copies":21492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9028,"num_write_kb":518144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"75'130","reported_seq":340,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.598917+0000","last_change":"2024-09-17T08:28:29.091771+0000","last_active":"2024-09-17T08:29:55.598917+0000","last_peered":"2024-09-17T08:29:55.598917+0000","last_clean":"2024-09-17T08:29:55.598917+0000","last_became_active":"2024-09-17T08:27:47.568726+0000","last_became_peered":"2024-09-17T08:27:47.568726+0000","last_unstale":"2024-09-17T08:29:55.598917+0000","last_undegraded":"2024-09-17T08:29:55.598917+0000","last_fullsized":"2024-09-17T08:29:55.598917+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:29.091638+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:29.091638+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.091638+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083305140000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"220'29","reported_seq":491,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576499+0000","last_change":"2024-09-17T08:28:42.048805+0000","last_active":"2024-09-17T08:29:54.576499+0000","last_peered":"2024-09-17T08:29:54.576499+0000","last_clean":"2024-09-17T08:29:54.576499+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:29:54.576499+0000","last_undegraded":"2024-09-17T08:29:54.576499+0000","last_fullsized":"2024-09-17T08:29:54.576499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'29","last_scrub_stamp":"2024-09-17T08:28:42.048628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:42.048628+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.248846+0000","scrub_duration":0.0076376279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":447,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580781+0000","last_change":"2024-09-17T08:28:20.547961+0000","last_active":"2024-09-17T08:29:54.580781+0000","last_peered":"2024-09-17T08:29:54.580781+0000","last_clean":"2024-09-17T08:29:54.580781+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:29:54.580781+0000","last_undegraded":"2024-09-17T08:29:54.580781+0000","last_fullsized":"2024-09-17T08:29:54.580781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:28:20.547814+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.547814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021902169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":114,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585189+0000","last_change":"2024-09-17T08:29:34.451238+0000","last_active":"2024-09-17T08:29:54.585189+0000","last_peered":"2024-09-17T08:29:54.585189+0000","last_clean":"2024-09-17T08:29:54.585189+0000","last_became_active":"2024-09-17T08:27:47.562839+0000","last_became_peered":"2024-09-17T08:27:47.562839+0000","last_unstale":"2024-09-17T08:29:54.585189+0000","last_undegraded":"2024-09-17T08:29:54.585189+0000","last_fullsized":"2024-09-17T08:29:54.585189+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:34.451138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:29:34.451138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:54.655647+0000","scrub_duration":0.0026313539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":335,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577177+0000","last_change":"2024-09-17T08:29:18.531201+0000","last_active":"2024-09-17T08:29:54.577177+0000","last_peered":"2024-09-17T08:29:54.577177+0000","last_clean":"2024-09-17T08:29:54.577177+0000","last_became_active":"2024-09-17T08:20:23.494585+0000","last_became_peered":"2024-09-17T08:20:23.494585+0000","last_unstale":"2024-09-17T08:29:54.577177+0000","last_undegraded":"2024-09-17T08:29:54.577177+0000","last_fullsized":"2024-09-17T08:29:54.577177+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:18.531099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:29:18.531099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:32.913005+0000","scrub_duration":0.011140373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.d","version":"389'8748","reported_seq":8947,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:56.924241+0000","last_change":"2024-09-17T08:29:36.497239+0000","last_active":"2024-09-17T08:29:56.924241+0000","last_peered":"2024-09-17T08:29:56.924241+0000","last_clean":"2024-09-17T08:29:56.924241+0000","last_became_active":"2024-09-17T08:28:00.529861+0000","last_became_peered":"2024-09-17T08:28:00.529861+0000","last_unstale":"2024-09-17T08:29:56.924241+0000","last_undegraded":"2024-09-17T08:29:56.924241+0000","last_fullsized":"2024-09-17T08:29:56.924241+0000","mapping_epoch":359,"log_start":"371'3100","ondisk_log_start":"371'3100","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'7437","last_scrub_stamp":"2024-09-17T08:29:36.496881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:29:36.496881+0000","objects_scrubbed":7366,"log_size":5648,"log_dups_size":0,"ondisk_log_size":5648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:57.630369+0000","scrub_duration":2.5146595229999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470876184,"num_objects":7186,"num_object_clones":0,"num_object_copies":21558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7186,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8748,"num_write_kb":509825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"75'130","reported_seq":340,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.598341+0000","last_change":"2024-09-17T08:28:32.005811+0000","last_active":"2024-09-17T08:29:55.598341+0000","last_peered":"2024-09-17T08:29:55.598341+0000","last_clean":"2024-09-17T08:29:55.598341+0000","last_became_active":"2024-09-17T08:27:47.567040+0000","last_became_peered":"2024-09-17T08:27:47.567040+0000","last_unstale":"2024-09-17T08:29:55.598341+0000","last_undegraded":"2024-09-17T08:29:55.598341+0000","last_fullsized":"2024-09-17T08:29:55.598341+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:32.005689+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.005689+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051363199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"220'25","reported_seq":352,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577324+0000","last_change":"2024-09-17T08:29:30.429472+0000","last_active":"2024-09-17T08:29:54.577324+0000","last_peered":"2024-09-17T08:29:54.577324+0000","last_clean":"2024-09-17T08:29:54.577324+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:29:54.577324+0000","last_undegraded":"2024-09-17T08:29:54.577324+0000","last_fullsized":"2024-09-17T08:29:54.577324+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'25","last_scrub_stamp":"2024-09-17T08:29:30.429348+0000","last_deep_scrub":"220'25","last_deep_scrub_stamp":"2024-09-17T08:29:30.429348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:30.429348+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:37.964260+0000","scrub_duration":0.0069458860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":309,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.579841+0000","last_change":"2024-09-17T08:27:19.209183+0000","last_active":"2024-09-17T08:29:54.579841+0000","last_peered":"2024-09-17T08:29:54.579841+0000","last_clean":"2024-09-17T08:29:54.579841+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:29:54.579841+0000","last_undegraded":"2024-09-17T08:29:54.579841+0000","last_fullsized":"2024-09-17T08:29:54.579841+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:19.209066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:27:19.209066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003693712,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"213'9","reported_seq":687,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576355+0000","last_change":"2024-09-17T08:29:20.820671+0000","last_active":"2024-09-17T08:29:54.576355+0000","last_peered":"2024-09-17T08:29:54.576355+0000","last_clean":"2024-09-17T08:29:54.576355+0000","last_became_active":"2024-09-17T08:09:58.505652+0000","last_became_peered":"2024-09-17T08:09:58.505652+0000","last_unstale":"2024-09-17T08:29:54.576355+0000","last_undegraded":"2024-09-17T08:29:54.576355+0000","last_fullsized":"2024-09-17T08:29:54.576355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'9","last_scrub_stamp":"2024-09-17T08:29:20.820541+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:29:20.820541+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:33.103244+0000","scrub_duration":0.010265026,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":441,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575872+0000","last_change":"2024-09-17T08:28:28.040603+0000","last_active":"2024-09-17T08:29:54.575872+0000","last_peered":"2024-09-17T08:29:54.575872+0000","last_clean":"2024-09-17T08:29:54.575872+0000","last_became_active":"2024-09-17T08:20:23.487600+0000","last_became_peered":"2024-09-17T08:20:23.487600+0000","last_unstale":"2024-09-17T08:29:54.575872+0000","last_undegraded":"2024-09-17T08:29:54.575872+0000","last_fullsized":"2024-09-17T08:29:54.575872+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:28.040487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:28.040487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047520069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"389'8967","reported_seq":9006,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.052976+0000","last_change":"2024-09-17T08:28:00.529238+0000","last_active":"2024-09-17T08:29:59.052976+0000","last_peered":"2024-09-17T08:29:59.052976+0000","last_clean":"2024-09-17T08:29:59.052976+0000","last_became_active":"2024-09-17T08:28:00.528278+0000","last_became_peered":"2024-09-17T08:28:00.528278+0000","last_unstale":"2024-09-17T08:29:59.052976+0000","last_undegraded":"2024-09-17T08:29:59.052976+0000","last_fullsized":"2024-09-17T08:29:59.052976+0000","mapping_epoch":359,"log_start":"373'4300","ondisk_log_start":"373'4300","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4667,"log_dups_size":0,"ondisk_log_size":4667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462618624,"num_objects":7059,"num_object_clones":0,"num_object_copies":21177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":512832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"75'130","reported_seq":366,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576975+0000","last_change":"2024-09-17T08:28:31.043325+0000","last_active":"2024-09-17T08:29:54.576975+0000","last_peered":"2024-09-17T08:29:54.576975+0000","last_clean":"2024-09-17T08:29:54.576975+0000","last_became_active":"2024-09-17T08:26:56.663984+0000","last_became_peered":"2024-09-17T08:26:56.663984+0000","last_unstale":"2024-09-17T08:29:54.576975+0000","last_undegraded":"2024-09-17T08:29:54.576975+0000","last_fullsized":"2024-09-17T08:29:54.576975+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:31.042950+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:31.042950+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.825421+0000","scrub_duration":0.0016036220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"228'44","reported_seq":555,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.593463+0000","last_change":"2024-09-17T08:29:25.512932+0000","last_active":"2024-09-17T08:29:55.593463+0000","last_peered":"2024-09-17T08:29:55.593463+0000","last_clean":"2024-09-17T08:29:55.593463+0000","last_became_active":"2024-09-17T08:20:23.497619+0000","last_became_peered":"2024-09-17T08:20:23.497619+0000","last_unstale":"2024-09-17T08:29:55.593463+0000","last_undegraded":"2024-09-17T08:29:55.593463+0000","last_fullsized":"2024-09-17T08:29:55.593463+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'44","last_scrub_stamp":"2024-09-17T08:29:25.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:25.512251+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:27.456264+0000","scrub_duration":0.025106175000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577472+0000","last_change":"2024-09-17T08:28:35.378069+0000","last_active":"2024-09-17T08:29:54.577472+0000","last_peered":"2024-09-17T08:29:54.577472+0000","last_clean":"2024-09-17T08:29:54.577472+0000","last_became_active":"2024-09-17T08:15:29.358576+0000","last_became_peered":"2024-09-17T08:15:29.358576+0000","last_unstale":"2024-09-17T08:29:54.577472+0000","last_undegraded":"2024-09-17T08:29:54.577472+0000","last_fullsized":"2024-09-17T08:29:54.577472+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.377950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.377950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.428689+0000","scrub_duration":0.004717578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":456,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576036+0000","last_change":"2024-09-17T08:29:17.831160+0000","last_active":"2024-09-17T08:29:54.576036+0000","last_peered":"2024-09-17T08:29:54.576036+0000","last_clean":"2024-09-17T08:29:54.576036+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:29:54.576036+0000","last_undegraded":"2024-09-17T08:29:54.576036+0000","last_fullsized":"2024-09-17T08:29:54.576036+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:17.831013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:48.455290+0000","last_clean_scrub_stamp":"2024-09-17T08:29:17.831013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.734445+0000","scrub_duration":0.013257501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":436,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585281+0000","last_change":"2024-09-17T08:29:38.040692+0000","last_active":"2024-09-17T08:29:54.585281+0000","last_peered":"2024-09-17T08:29:54.585281+0000","last_clean":"2024-09-17T08:29:54.585281+0000","last_became_active":"2024-09-17T08:10:00.524167+0000","last_became_peered":"2024-09-17T08:10:00.524167+0000","last_unstale":"2024-09-17T08:29:54.585281+0000","last_undegraded":"2024-09-17T08:29:54.585281+0000","last_fullsized":"2024-09-17T08:29:54.585281+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:38.040577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:29:38.040577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:04.016066+0000","scrub_duration":0.002284824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"389'8972","reported_seq":9011,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.055017+0000","last_change":"2024-09-17T08:28:00.633965+0000","last_active":"2024-09-17T08:29:59.055017+0000","last_peered":"2024-09-17T08:29:59.055017+0000","last_clean":"2024-09-17T08:29:59.055017+0000","last_became_active":"2024-09-17T08:28:00.633649+0000","last_became_peered":"2024-09-17T08:28:00.633649+0000","last_unstale":"2024-09-17T08:29:59.055017+0000","last_undegraded":"2024-09-17T08:29:59.055017+0000","last_fullsized":"2024-09-17T08:29:59.055017+0000","mapping_epoch":359,"log_start":"373'4300","ondisk_log_start":"373'4300","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4672,"log_dups_size":0,"ondisk_log_size":4672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460455936,"num_objects":7026,"num_object_clones":0,"num_object_copies":21078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8972,"num_write_kb":511936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'130","reported_seq":741,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580101+0000","last_change":"2024-09-17T08:29:27.476114+0000","last_active":"2024-09-17T08:29:54.580101+0000","last_peered":"2024-09-17T08:29:54.580101+0000","last_clean":"2024-09-17T08:29:54.580101+0000","last_became_active":"2024-09-17T08:26:56.652768+0000","last_became_peered":"2024-09-17T08:26:56.652768+0000","last_unstale":"2024-09-17T08:29:54.580101+0000","last_undegraded":"2024-09-17T08:29:54.580101+0000","last_fullsized":"2024-09-17T08:29:54.580101+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:29:27.476015+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:29:27.476015+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:44.867147+0000","scrub_duration":0.00240077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"220'17","reported_seq":338,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580618+0000","last_change":"2024-09-17T08:29:38.315526+0000","last_active":"2024-09-17T08:29:54.580618+0000","last_peered":"2024-09-17T08:29:54.580618+0000","last_clean":"2024-09-17T08:29:54.580618+0000","last_became_active":"2024-09-17T08:15:29.369661+0000","last_became_peered":"2024-09-17T08:15:29.369661+0000","last_unstale":"2024-09-17T08:29:54.580618+0000","last_undegraded":"2024-09-17T08:29:54.580618+0000","last_fullsized":"2024-09-17T08:29:54.580618+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'17","last_scrub_stamp":"2024-09-17T08:29:38.315197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:38.315197+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:43.239381+0000","scrub_duration":0.0078904609999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"74'5","reported_seq":1206,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.246356+0000","last_change":"2024-09-17T08:29:33.386195+0000","last_active":"2024-09-17T08:29:57.246356+0000","last_peered":"2024-09-17T08:29:57.246356+0000","last_clean":"2024-09-17T08:29:57.246356+0000","last_became_active":"2024-09-17T08:20:23.489235+0000","last_became_peered":"2024-09-17T08:20:23.489235+0000","last_unstale":"2024-09-17T08:29:57.246356+0000","last_undegraded":"2024-09-17T08:29:57.246356+0000","last_fullsized":"2024-09-17T08:29:57.246356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'5","last_scrub_stamp":"2024-09-17T08:29:33.385962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:29:33.385962+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:57.774956+0000","scrub_duration":0.0035456189999999999,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":865,"num_read_kb":860,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":426,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.586091+0000","last_change":"2024-09-17T08:29:36.014297+0000","last_active":"2024-09-17T08:29:54.586091+0000","last_peered":"2024-09-17T08:29:54.586091+0000","last_clean":"2024-09-17T08:29:54.586091+0000","last_became_active":"2024-09-17T08:20:23.486411+0000","last_became_peered":"2024-09-17T08:20:23.486411+0000","last_unstale":"2024-09-17T08:29:54.586091+0000","last_undegraded":"2024-09-17T08:29:54.586091+0000","last_fullsized":"2024-09-17T08:29:54.586091+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:36.014178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:29:36.014178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:52.416775+0000","scrub_duration":0.0096787479999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":423,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588879+0000","last_change":"2024-09-17T08:28:18.226962+0000","last_active":"2024-09-17T08:29:54.588879+0000","last_peered":"2024-09-17T08:29:54.588879+0000","last_clean":"2024-09-17T08:29:54.588879+0000","last_became_active":"2024-09-17T08:20:23.484976+0000","last_became_peered":"2024-09-17T08:20:23.484976+0000","last_unstale":"2024-09-17T08:29:54.588879+0000","last_undegraded":"2024-09-17T08:29:54.588879+0000","last_fullsized":"2024-09-17T08:29:54.588879+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.226847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.226847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026678819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":446,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580604+0000","last_change":"2024-09-17T08:28:33.423821+0000","last_active":"2024-09-17T08:29:54.580604+0000","last_peered":"2024-09-17T08:29:54.580604+0000","last_clean":"2024-09-17T08:29:54.580604+0000","last_became_active":"2024-09-17T08:10:00.599518+0000","last_became_peered":"2024-09-17T08:10:00.599518+0000","last_unstale":"2024-09-17T08:29:54.580604+0000","last_undegraded":"2024-09-17T08:29:54.580604+0000","last_fullsized":"2024-09-17T08:29:54.580604+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:33.423708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:12:45.574097+0000","last_clean_scrub_stamp":"2024-09-17T08:28:33.423708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:59.644830+0000","scrub_duration":0.0040275149999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"389'9018","reported_seq":9057,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.015960+0000","last_change":"2024-09-17T08:28:00.529084+0000","last_active":"2024-09-17T08:29:59.015960+0000","last_peered":"2024-09-17T08:29:59.015960+0000","last_clean":"2024-09-17T08:29:59.015960+0000","last_became_active":"2024-09-17T08:28:00.528828+0000","last_became_peered":"2024-09-17T08:28:00.528828+0000","last_unstale":"2024-09-17T08:29:59.015960+0000","last_undegraded":"2024-09-17T08:29:59.015960+0000","last_fullsized":"2024-09-17T08:29:59.015960+0000","mapping_epoch":359,"log_start":"373'4400","ondisk_log_start":"373'4400","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":467795968,"num_objects":7138,"num_object_clones":0,"num_object_copies":21414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9018,"num_write_kb":516992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"75'130","reported_seq":358,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583884+0000","last_change":"2024-09-17T08:28:13.346758+0000","last_active":"2024-09-17T08:29:54.583884+0000","last_peered":"2024-09-17T08:29:54.583884+0000","last_clean":"2024-09-17T08:29:54.583884+0000","last_became_active":"2024-09-17T08:26:56.645417+0000","last_became_peered":"2024-09-17T08:26:56.645417+0000","last_unstale":"2024-09-17T08:29:54.583884+0000","last_undegraded":"2024-09-17T08:29:54.583884+0000","last_fullsized":"2024-09-17T08:29:54.583884+0000","mapping_epoch":337,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":338,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:28:13.346675+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.346675+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018158759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"220'30","reported_seq":547,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577953+0000","last_change":"2024-09-17T08:28:27.034085+0000","last_active":"2024-09-17T08:29:54.577953+0000","last_peered":"2024-09-17T08:29:54.577953+0000","last_clean":"2024-09-17T08:29:54.577953+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:29:54.577953+0000","last_undegraded":"2024-09-17T08:29:54.577953+0000","last_fullsized":"2024-09-17T08:29:54.577953+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'30","last_scrub_stamp":"2024-09-17T08:28:27.033942+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-17T08:11:11.895677+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.033942+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:56.987816+0000","scrub_duration":0.0076002580000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":449,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577449+0000","last_change":"2024-09-17T08:28:24.124112+0000","last_active":"2024-09-17T08:29:54.577449+0000","last_peered":"2024-09-17T08:29:54.577449+0000","last_clean":"2024-09-17T08:29:54.577449+0000","last_became_active":"2024-09-17T08:09:54.500435+0000","last_became_peered":"2024-09-17T08:09:54.500435+0000","last_unstale":"2024-09-17T08:29:54.577449+0000","last_undegraded":"2024-09-17T08:29:54.577449+0000","last_fullsized":"2024-09-17T08:29:54.577449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:28:24.123939+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.123939+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056523010000000002,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":471,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.595847+0000","last_change":"2024-09-17T08:28:35.399624+0000","last_active":"2024-09-17T08:29:55.595847+0000","last_peered":"2024-09-17T08:29:55.595847+0000","last_clean":"2024-09-17T08:29:55.595847+0000","last_became_active":"2024-09-17T08:15:29.371214+0000","last_became_peered":"2024-09-17T08:15:29.371214+0000","last_unstale":"2024-09-17T08:29:55.595847+0000","last_undegraded":"2024-09-17T08:29:55.595847+0000","last_fullsized":"2024-09-17T08:29:55.595847+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.399515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043382050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580596+0000","last_change":"2024-09-17T08:28:29.458578+0000","last_active":"2024-09-17T08:29:54.580596+0000","last_peered":"2024-09-17T08:29:54.580596+0000","last_clean":"2024-09-17T08:29:54.580596+0000","last_became_active":"2024-09-17T08:15:29.370120+0000","last_became_peered":"2024-09-17T08:15:29.370120+0000","last_unstale":"2024-09-17T08:29:54.580596+0000","last_undegraded":"2024-09-17T08:29:54.580596+0000","last_fullsized":"2024-09-17T08:29:54.580596+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.458289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.458289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063383909999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.579664+0000","last_change":"2024-09-17T08:29:29.474236+0000","last_active":"2024-09-17T08:29:54.579664+0000","last_peered":"2024-09-17T08:29:54.579664+0000","last_clean":"2024-09-17T08:29:54.579664+0000","last_became_active":"2024-09-17T08:15:29.385325+0000","last_became_peered":"2024-09-17T08:15:29.385325+0000","last_unstale":"2024-09-17T08:29:54.579664+0000","last_undegraded":"2024-09-17T08:29:54.579664+0000","last_fullsized":"2024-09-17T08:29:54.579664+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:29.474116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:29:29.474116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:40.615876+0000","scrub_duration":0.0060215620000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"389'8963","reported_seq":9002,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.026838+0000","last_change":"2024-09-17T08:28:00.633997+0000","last_active":"2024-09-17T08:29:59.026838+0000","last_peered":"2024-09-17T08:29:59.026838+0000","last_clean":"2024-09-17T08:29:59.026838+0000","last_became_active":"2024-09-17T08:28:00.633745+0000","last_became_peered":"2024-09-17T08:28:00.633745+0000","last_unstale":"2024-09-17T08:29:59.026838+0000","last_undegraded":"2024-09-17T08:29:59.026838+0000","last_fullsized":"2024-09-17T08:29:59.026838+0000","mapping_epoch":359,"log_start":"373'4300","ondisk_log_start":"373'4300","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4663,"log_dups_size":0,"ondisk_log_size":4663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464060416,"num_objects":7081,"num_object_clones":0,"num_object_copies":21243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":513408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"220'33","reported_seq":271,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584966+0000","last_change":"2024-09-17T08:28:35.037720+0000","last_active":"2024-09-17T08:29:54.584966+0000","last_peered":"2024-09-17T08:29:54.584966+0000","last_clean":"2024-09-17T08:29:54.584966+0000","last_became_active":"2024-09-17T08:27:47.562580+0000","last_became_peered":"2024-09-17T08:27:47.562580+0000","last_unstale":"2024-09-17T08:29:54.584966+0000","last_undegraded":"2024-09-17T08:29:54.584966+0000","last_fullsized":"2024-09-17T08:29:54.584966+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'33","last_scrub_stamp":"2024-09-17T08:28:35.037586+0000","last_deep_scrub":"139'21","last_deep_scrub_stamp":"2024-09-17T08:14:46.101949+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.037586+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003341709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"69'2","reported_seq":438,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584247+0000","last_change":"2024-09-17T08:28:35.737604+0000","last_active":"2024-09-17T08:29:54.584247+0000","last_peered":"2024-09-17T08:29:54.584247+0000","last_clean":"2024-09-17T08:29:54.584247+0000","last_became_active":"2024-09-17T08:09:54.498234+0000","last_became_peered":"2024-09-17T08:09:54.498234+0000","last_unstale":"2024-09-17T08:29:54.584247+0000","last_undegraded":"2024-09-17T08:29:54.584247+0000","last_fullsized":"2024-09-17T08:29:54.584247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-17T08:28:35.737203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:35.737203+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041867500000000004,"objects_trimmed":0,"snaptrim_duration":0.0010035739999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":324,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588083+0000","last_change":"2024-09-17T08:29:21.934962+0000","last_active":"2024-09-17T08:29:54.588083+0000","last_peered":"2024-09-17T08:29:54.588083+0000","last_clean":"2024-09-17T08:29:54.588083+0000","last_became_active":"2024-09-17T08:20:23.486593+0000","last_became_peered":"2024-09-17T08:20:23.486593+0000","last_unstale":"2024-09-17T08:29:54.588083+0000","last_undegraded":"2024-09-17T08:29:54.588083+0000","last_fullsized":"2024-09-17T08:29:54.588083+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:21.934837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:29:21.934837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:35.909269+0000","scrub_duration":0.0029338290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":725,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580324+0000","last_change":"2024-09-17T08:29:23.431678+0000","last_active":"2024-09-17T08:29:54.580324+0000","last_peered":"2024-09-17T08:29:54.580324+0000","last_clean":"2024-09-17T08:29:54.580324+0000","last_became_active":"2024-09-17T08:26:51.613195+0000","last_became_peered":"2024-09-17T08:26:51.613195+0000","last_unstale":"2024-09-17T08:29:54.580324+0000","last_undegraded":"2024-09-17T08:29:54.580324+0000","last_fullsized":"2024-09-17T08:29:54.580324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:29:23.431558+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:29:23.431558+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:39.442137+0000","scrub_duration":0.0074188489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":441,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580525+0000","last_change":"2024-09-17T08:28:14.537148+0000","last_active":"2024-09-17T08:29:54.580525+0000","last_peered":"2024-09-17T08:29:54.580525+0000","last_clean":"2024-09-17T08:29:54.580525+0000","last_became_active":"2024-09-17T08:20:23.488568+0000","last_became_peered":"2024-09-17T08:20:23.488568+0000","last_unstale":"2024-09-17T08:29:54.580525+0000","last_undegraded":"2024-09-17T08:29:54.580525+0000","last_fullsized":"2024-09-17T08:29:54.580525+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:14.537050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:38.817546+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.537050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001583777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":329,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583474+0000","last_change":"2024-09-17T08:28:19.752472+0000","last_active":"2024-09-17T08:29:54.583474+0000","last_peered":"2024-09-17T08:29:54.583474+0000","last_clean":"2024-09-17T08:29:54.583474+0000","last_became_active":"2024-09-17T08:14:58.718617+0000","last_became_peered":"2024-09-17T08:14:58.718617+0000","last_unstale":"2024-09-17T08:29:54.583474+0000","last_undegraded":"2024-09-17T08:29:54.583474+0000","last_fullsized":"2024-09-17T08:29:54.583474+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:19.752090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.752090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024911651999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"389'8874","reported_seq":8957,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.917518+0000","last_change":"2024-09-17T08:29:40.959890+0000","last_active":"2024-09-17T08:29:57.917518+0000","last_peered":"2024-09-17T08:29:57.917518+0000","last_clean":"2024-09-17T08:29:57.917518+0000","last_became_active":"2024-09-17T08:28:00.634149+0000","last_became_peered":"2024-09-17T08:28:00.634149+0000","last_unstale":"2024-09-17T08:29:57.917518+0000","last_undegraded":"2024-09-17T08:29:57.917518+0000","last_fullsized":"2024-09-17T08:29:57.917518+0000","mapping_epoch":359,"log_start":"373'4200","ondisk_log_start":"373'4200","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":1999,"log_size":4674,"log_dups_size":0,"ondisk_log_size":4674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472907776,"num_objects":7216,"num_object_clones":0,"num_object_copies":21648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8874,"num_write_kb":514880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"220'35","reported_seq":273,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.598652+0000","last_change":"2024-09-17T08:29:28.343538+0000","last_active":"2024-09-17T08:29:55.598652+0000","last_peered":"2024-09-17T08:29:55.598652+0000","last_clean":"2024-09-17T08:29:55.598652+0000","last_became_active":"2024-09-17T08:27:47.559010+0000","last_became_peered":"2024-09-17T08:27:47.559010+0000","last_unstale":"2024-09-17T08:29:55.598652+0000","last_undegraded":"2024-09-17T08:29:55.598652+0000","last_fullsized":"2024-09-17T08:29:55.598652+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'35","last_scrub_stamp":"2024-09-17T08:29:28.343439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:28.343439+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:43.192016+0000","scrub_duration":0.0032224770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":404,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588962+0000","last_change":"2024-09-17T08:28:10.080401+0000","last_active":"2024-09-17T08:29:54.588962+0000","last_peered":"2024-09-17T08:29:54.588962+0000","last_clean":"2024-09-17T08:29:54.588962+0000","last_became_active":"2024-09-17T08:20:23.491661+0000","last_became_peered":"2024-09-17T08:20:23.491661+0000","last_unstale":"2024-09-17T08:29:54.588962+0000","last_undegraded":"2024-09-17T08:29:54.588962+0000","last_fullsized":"2024-09-17T08:29:54.588962+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:10.080312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:10.080312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019214270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":421,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588020+0000","last_change":"2024-09-17T08:28:28.198962+0000","last_active":"2024-09-17T08:29:54.588020+0000","last_peered":"2024-09-17T08:29:54.588020+0000","last_clean":"2024-09-17T08:29:54.588020+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:29:54.588020+0000","last_undegraded":"2024-09-17T08:29:54.588020+0000","last_fullsized":"2024-09-17T08:29:54.588020+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_clean_scrub_stamp":"2024-09-17T08:28:28.198833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035093979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":439,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580395+0000","last_change":"2024-09-17T08:28:27.487067+0000","last_active":"2024-09-17T08:29:54.580395+0000","last_peered":"2024-09-17T08:29:54.580395+0000","last_clean":"2024-09-17T08:29:54.580395+0000","last_became_active":"2024-09-17T08:20:23.484235+0000","last_became_peered":"2024-09-17T08:20:23.484235+0000","last_unstale":"2024-09-17T08:29:54.580395+0000","last_undegraded":"2024-09-17T08:29:54.580395+0000","last_fullsized":"2024-09-17T08:29:54.580395+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.486978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.486978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022327200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.4","version":"389'9161","reported_seq":9359,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.914456+0000","last_change":"2024-09-17T08:29:32.538676+0000","last_active":"2024-09-17T08:29:57.914456+0000","last_peered":"2024-09-17T08:29:57.914456+0000","last_clean":"2024-09-17T08:29:57.914456+0000","last_became_active":"2024-09-17T08:28:00.530013+0000","last_became_peered":"2024-09-17T08:28:00.530013+0000","last_unstale":"2024-09-17T08:29:57.914456+0000","last_undegraded":"2024-09-17T08:29:57.914456+0000","last_fullsized":"2024-09-17T08:29:57.914456+0000","mapping_epoch":359,"log_start":"373'4500","ondisk_log_start":"373'4500","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'7431","last_scrub_stamp":"2024-09-17T08:29:32.538531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:29:32.538531+0000","objects_scrubbed":7357,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:37.582313+0000","scrub_duration":2.0408124349999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483065856,"num_objects":7371,"num_object_clones":0,"num_object_copies":22113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9161,"num_write_kb":529024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"220'15","reported_seq":491,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576429+0000","last_change":"2024-09-17T08:28:26.065898+0000","last_active":"2024-09-17T08:29:54.576429+0000","last_peered":"2024-09-17T08:29:54.576429+0000","last_clean":"2024-09-17T08:29:54.576429+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:29:54.576429+0000","last_undegraded":"2024-09-17T08:29:54.576429+0000","last_fullsized":"2024-09-17T08:29:54.576429+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:28:26.065739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:26.065739+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032677489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":320,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577974+0000","last_change":"2024-09-17T08:28:19.462647+0000","last_active":"2024-09-17T08:29:54.577974+0000","last_peered":"2024-09-17T08:29:54.577974+0000","last_clean":"2024-09-17T08:29:54.577974+0000","last_became_active":"2024-09-17T08:15:29.381062+0000","last_became_peered":"2024-09-17T08:15:29.381062+0000","last_unstale":"2024-09-17T08:29:54.577974+0000","last_undegraded":"2024-09-17T08:29:54.577974+0000","last_fullsized":"2024-09-17T08:29:54.577974+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:19.462533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.462533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034847369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":424,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.586990+0000","last_change":"2024-09-17T08:28:13.138380+0000","last_active":"2024-09-17T08:29:54.586990+0000","last_peered":"2024-09-17T08:29:54.586990+0000","last_clean":"2024-09-17T08:29:54.586990+0000","last_became_active":"2024-09-17T08:20:23.481879+0000","last_became_peered":"2024-09-17T08:20:23.481879+0000","last_unstale":"2024-09-17T08:29:54.586990+0000","last_undegraded":"2024-09-17T08:29:54.586990+0000","last_fullsized":"2024-09-17T08:29:54.586990+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.138294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:25.124549+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.138294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001677385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":440,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577368+0000","last_change":"2024-09-17T08:28:17.190360+0000","last_active":"2024-09-17T08:29:54.577368+0000","last_peered":"2024-09-17T08:29:54.577368+0000","last_clean":"2024-09-17T08:29:54.577368+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:29:54.577368+0000","last_undegraded":"2024-09-17T08:29:54.577368+0000","last_fullsized":"2024-09-17T08:29:54.577368+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:17.190228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:24.572550+0000","last_clean_scrub_stamp":"2024-09-17T08:28:17.190228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003265783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"389'8926","reported_seq":8965,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:59.023886+0000","last_change":"2024-09-17T08:28:00.634479+0000","last_active":"2024-09-17T08:29:59.023886+0000","last_peered":"2024-09-17T08:29:59.023886+0000","last_clean":"2024-09-17T08:29:59.023886+0000","last_became_active":"2024-09-17T08:28:00.634232+0000","last_became_peered":"2024-09-17T08:28:00.634232+0000","last_unstale":"2024-09-17T08:29:59.023886+0000","last_undegraded":"2024-09-17T08:29:59.023886+0000","last_fullsized":"2024-09-17T08:29:59.023886+0000","mapping_epoch":359,"log_start":"373'4000","ondisk_log_start":"373'4000","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466223104,"num_objects":7114,"num_object_clones":0,"num_object_copies":21342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8926,"num_write_kb":513280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"220'34","reported_seq":494,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580089+0000","last_change":"2024-09-17T08:29:25.455385+0000","last_active":"2024-09-17T08:29:54.580089+0000","last_peered":"2024-09-17T08:29:54.580089+0000","last_clean":"2024-09-17T08:29:54.580089+0000","last_became_active":"2024-09-17T08:14:58.716309+0000","last_became_peered":"2024-09-17T08:14:58.716309+0000","last_unstale":"2024-09-17T08:29:54.580089+0000","last_undegraded":"2024-09-17T08:29:54.580089+0000","last_fullsized":"2024-09-17T08:29:54.580089+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:29:25.455224+0000","last_deep_scrub":"140'22","last_deep_scrub_stamp":"2024-09-17T08:18:38.041603+0000","last_clean_scrub_stamp":"2024-09-17T08:29:25.455224+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:40.566639+0000","scrub_duration":0.002002424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":436,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.589304+0000","last_change":"2024-09-17T08:29:18.945715+0000","last_active":"2024-09-17T08:29:54.589304+0000","last_peered":"2024-09-17T08:29:54.589304+0000","last_clean":"2024-09-17T08:29:54.589304+0000","last_became_active":"2024-09-17T08:20:23.493231+0000","last_became_peered":"2024-09-17T08:20:23.493231+0000","last_unstale":"2024-09-17T08:29:54.589304+0000","last_undegraded":"2024-09-17T08:29:54.589304+0000","last_fullsized":"2024-09-17T08:29:54.589304+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:18.945270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:29:18.945270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:24.348318+0000","scrub_duration":0.0079758580000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"267'9","reported_seq":686,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580171+0000","last_change":"2024-09-17T08:28:39.372979+0000","last_active":"2024-09-17T08:29:54.580171+0000","last_peered":"2024-09-17T08:29:54.580171+0000","last_clean":"2024-09-17T08:29:54.580171+0000","last_became_active":"2024-09-17T08:09:58.513415+0000","last_became_peered":"2024-09-17T08:09:58.513415+0000","last_unstale":"2024-09-17T08:29:54.580171+0000","last_undegraded":"2024-09-17T08:29:54.580171+0000","last_fullsized":"2024-09-17T08:29:54.580171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'9","last_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.372822+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003653173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":183,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.597794+0000","last_change":"2024-09-17T08:28:24.092358+0000","last_active":"2024-09-17T08:29:55.597794+0000","last_peered":"2024-09-17T08:29:55.597794+0000","last_clean":"2024-09-17T08:29:55.597794+0000","last_became_active":"2024-09-17T08:27:47.569625+0000","last_became_peered":"2024-09-17T08:27:47.569625+0000","last_unstale":"2024-09-17T08:29:55.597794+0000","last_undegraded":"2024-09-17T08:29:55.597794+0000","last_fullsized":"2024-09-17T08:29:55.597794+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:24.092273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.092273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034287760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"389'8866","reported_seq":8904,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:57.648798+0000","last_change":"2024-09-17T08:28:00.529206+0000","last_active":"2024-09-17T08:29:57.648798+0000","last_peered":"2024-09-17T08:29:57.648798+0000","last_clean":"2024-09-17T08:29:57.648798+0000","last_became_active":"2024-09-17T08:28:00.528548+0000","last_became_peered":"2024-09-17T08:28:00.528548+0000","last_unstale":"2024-09-17T08:29:57.648798+0000","last_undegraded":"2024-09-17T08:29:57.648798+0000","last_fullsized":"2024-09-17T08:29:57.648798+0000","mapping_epoch":359,"log_start":"371'3300","ondisk_log_start":"371'3300","created":359,"last_epoch_clean":360,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:59.507021+0000","last_clean_scrub_stamp":"2024-09-17T08:27:59.507021+0000","objects_scrubbed":0,"log_size":5566,"log_dups_size":0,"ondisk_log_size":5566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468451328,"num_objects":7148,"num_object_clones":0,"num_object_copies":21444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8866,"num_write_kb":512448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"220'18","reported_seq":503,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574475+0000","last_change":"2024-09-17T08:28:19.238927+0000","last_active":"2024-09-17T08:29:54.574475+0000","last_peered":"2024-09-17T08:29:54.574475+0000","last_clean":"2024-09-17T08:29:54.574475+0000","last_became_active":"2024-09-17T08:15:29.372321+0000","last_became_peered":"2024-09-17T08:15:29.372321+0000","last_unstale":"2024-09-17T08:29:54.574475+0000","last_undegraded":"2024-09-17T08:29:54.574475+0000","last_fullsized":"2024-09-17T08:29:54.574475+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:28:19.238796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.238796+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084402430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"65'2","reported_seq":452,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575497+0000","last_change":"2024-09-17T08:29:02.564173+0000","last_active":"2024-09-17T08:29:54.575497+0000","last_peered":"2024-09-17T08:29:54.575497+0000","last_clean":"2024-09-17T08:29:54.575497+0000","last_became_active":"2024-09-17T08:29:02.550936+0000","last_became_peered":"2024-09-17T08:29:02.550936+0000","last_unstale":"2024-09-17T08:29:54.575497+0000","last_undegraded":"2024-09-17T08:29:54.575497+0000","last_fullsized":"2024-09-17T08:29:54.575497+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T08:27:22.244827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:27:22.244827+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052538849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":440,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.587816+0000","last_change":"2024-09-17T08:29:19.919615+0000","last_active":"2024-09-17T08:29:54.587816+0000","last_peered":"2024-09-17T08:29:54.587816+0000","last_clean":"2024-09-17T08:29:54.587816+0000","last_became_active":"2024-09-17T08:20:23.493503+0000","last_became_peered":"2024-09-17T08:20:23.493503+0000","last_unstale":"2024-09-17T08:29:54.587816+0000","last_undegraded":"2024-09-17T08:29:54.587816+0000","last_fullsized":"2024-09-17T08:29:54.587816+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:19.919482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:29:19.919482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.753894+0000","scrub_duration":0.0041697000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":244,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.587956+0000","last_change":"2024-09-17T08:28:14.165430+0000","last_active":"2024-09-17T08:29:54.587956+0000","last_peered":"2024-09-17T08:29:54.587956+0000","last_clean":"2024-09-17T08:29:54.587956+0000","last_became_active":"2024-09-17T08:20:23.485707+0000","last_became_peered":"2024-09-17T08:20:23.485707+0000","last_unstale":"2024-09-17T08:29:54.587956+0000","last_undegraded":"2024-09-17T08:29:54.587956+0000","last_fullsized":"2024-09-17T08:29:54.587956+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:14.165350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.165350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015954630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"220'19","reported_seq":457,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577876+0000","last_change":"2024-09-17T08:28:33.991107+0000","last_active":"2024-09-17T08:29:54.577876+0000","last_peered":"2024-09-17T08:29:54.577876+0000","last_clean":"2024-09-17T08:29:54.577876+0000","last_became_active":"2024-09-17T08:09:57.040870+0000","last_became_peered":"2024-09-17T08:09:57.040870+0000","last_unstale":"2024-09-17T08:29:54.577876+0000","last_undegraded":"2024-09-17T08:29:54.577876+0000","last_fullsized":"2024-09-17T08:29:54.577876+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:33.990923+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:28:33.990923+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037671190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":454,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.586384+0000","last_change":"2024-09-17T08:28:27.202142+0000","last_active":"2024-09-17T08:29:54.586384+0000","last_peered":"2024-09-17T08:29:54.586384+0000","last_clean":"2024-09-17T08:29:54.586384+0000","last_became_active":"2024-09-17T08:20:23.493635+0000","last_became_peered":"2024-09-17T08:20:23.493635+0000","last_unstale":"2024-09-17T08:29:54.586384+0000","last_undegraded":"2024-09-17T08:29:54.586384+0000","last_fullsized":"2024-09-17T08:29:54.586384+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.202005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.202005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008746116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":433,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583691+0000","last_change":"2024-09-17T08:28:08.835470+0000","last_active":"2024-09-17T08:29:54.583691+0000","last_peered":"2024-09-17T08:29:54.583691+0000","last_clean":"2024-09-17T08:29:54.583691+0000","last_became_active":"2024-09-17T08:15:29.384768+0000","last_became_peered":"2024-09-17T08:15:29.384768+0000","last_unstale":"2024-09-17T08:29:54.583691+0000","last_undegraded":"2024-09-17T08:29:54.583691+0000","last_fullsized":"2024-09-17T08:29:54.583691+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:08.835384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:08.835384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034811569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":427,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588818+0000","last_change":"2024-09-17T08:29:26.956712+0000","last_active":"2024-09-17T08:29:54.588818+0000","last_peered":"2024-09-17T08:29:54.588818+0000","last_clean":"2024-09-17T08:29:54.588818+0000","last_became_active":"2024-09-17T08:20:23.486505+0000","last_became_peered":"2024-09-17T08:20:23.486505+0000","last_unstale":"2024-09-17T08:29:54.588818+0000","last_undegraded":"2024-09-17T08:29:54.588818+0000","last_fullsized":"2024-09-17T08:29:54.588818+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:26.956417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:29:26.956417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:42.773075+0000","scrub_duration":0.010154165999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":241,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583166+0000","last_change":"2024-09-17T08:28:10.117649+0000","last_active":"2024-09-17T08:29:54.583166+0000","last_peered":"2024-09-17T08:29:54.583166+0000","last_clean":"2024-09-17T08:29:54.583166+0000","last_became_active":"2024-09-17T08:27:47.561372+0000","last_became_peered":"2024-09-17T08:27:47.561372+0000","last_unstale":"2024-09-17T08:29:54.583166+0000","last_undegraded":"2024-09-17T08:29:54.583166+0000","last_fullsized":"2024-09-17T08:29:54.583166+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:10.117557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:10.117557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015163640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"220'12","reported_seq":480,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583895+0000","last_change":"2024-09-17T08:28:24.657204+0000","last_active":"2024-09-17T08:29:54.583895+0000","last_peered":"2024-09-17T08:29:54.583895+0000","last_clean":"2024-09-17T08:29:54.583895+0000","last_became_active":"2024-09-17T08:15:29.384990+0000","last_became_peered":"2024-09-17T08:15:29.384990+0000","last_unstale":"2024-09-17T08:29:54.583895+0000","last_undegraded":"2024-09-17T08:29:54.583895+0000","last_fullsized":"2024-09-17T08:29:54.583895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'12","last_scrub_stamp":"2024-09-17T08:28:24.657080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:24.657080+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093465079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":336,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585161+0000","last_change":"2024-09-17T08:27:28.616429+0000","last_active":"2024-09-17T08:29:54.585161+0000","last_peered":"2024-09-17T08:29:54.585161+0000","last_clean":"2024-09-17T08:29:54.585161+0000","last_became_active":"2024-09-17T08:15:29.383484+0000","last_became_peered":"2024-09-17T08:15:29.383484+0000","last_unstale":"2024-09-17T08:29:54.585161+0000","last_undegraded":"2024-09-17T08:29:54.585161+0000","last_fullsized":"2024-09-17T08:29:54.585161+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:28.616307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:28.616307+0000","last_clean_scrub_stamp":"2024-09-17T08:27:28.616307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050860280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":445,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580383+0000","last_change":"2024-09-17T08:28:32.421385+0000","last_active":"2024-09-17T08:29:54.580383+0000","last_peered":"2024-09-17T08:29:54.580383+0000","last_clean":"2024-09-17T08:29:54.580383+0000","last_became_active":"2024-09-17T08:10:00.598013+0000","last_became_peered":"2024-09-17T08:10:00.598013+0000","last_unstale":"2024-09-17T08:29:54.580383+0000","last_undegraded":"2024-09-17T08:29:54.580383+0000","last_fullsized":"2024-09-17T08:29:54.580383+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:32.421272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.421272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:56.260584+0000","scrub_duration":0.0016128049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"220'20","reported_seq":500,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584942+0000","last_change":"2024-09-17T08:29:24.002005+0000","last_active":"2024-09-17T08:29:54.584942+0000","last_peered":"2024-09-17T08:29:54.584942+0000","last_clean":"2024-09-17T08:29:54.584942+0000","last_became_active":"2024-09-17T08:15:29.392287+0000","last_became_peered":"2024-09-17T08:15:29.392287+0000","last_unstale":"2024-09-17T08:29:54.584942+0000","last_undegraded":"2024-09-17T08:29:54.584942+0000","last_fullsized":"2024-09-17T08:29:54.584942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:29:24.001881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:24.001881+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:26.090635+0000","scrub_duration":0.0046296009999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":376,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583679+0000","last_change":"2024-09-17T08:28:25.643306+0000","last_active":"2024-09-17T08:29:54.583679+0000","last_peered":"2024-09-17T08:29:54.583679+0000","last_clean":"2024-09-17T08:29:54.583679+0000","last_became_active":"2024-09-17T08:20:23.483911+0000","last_became_peered":"2024-09-17T08:20:23.483911+0000","last_unstale":"2024-09-17T08:29:54.583679+0000","last_undegraded":"2024-09-17T08:29:54.583679+0000","last_fullsized":"2024-09-17T08:29:54.583679+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:25.643230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:28:25.643230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001106164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"201'12","reported_seq":499,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.591011+0000","last_change":"2024-09-17T08:29:34.411428+0000","last_active":"2024-09-17T08:29:55.591011+0000","last_peered":"2024-09-17T08:29:55.591011+0000","last_clean":"2024-09-17T08:29:55.591011+0000","last_became_active":"2024-09-17T08:15:29.370669+0000","last_became_peered":"2024-09-17T08:15:29.370669+0000","last_unstale":"2024-09-17T08:29:55.591011+0000","last_undegraded":"2024-09-17T08:29:55.591011+0000","last_fullsized":"2024-09-17T08:29:55.591011+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'12","last_scrub_stamp":"2024-09-17T08:29:34.411246+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:29:34.411246+0000","last_clean_scrub_stamp":"2024-09-17T08:29:34.411246+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:42.511124+0000","scrub_duration":0.0069095370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":424,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575647+0000","last_change":"2024-09-17T08:27:25.259320+0000","last_active":"2024-09-17T08:29:54.575647+0000","last_peered":"2024-09-17T08:29:54.575647+0000","last_clean":"2024-09-17T08:29:54.575647+0000","last_became_active":"2024-09-17T08:10:00.521624+0000","last_became_peered":"2024-09-17T08:10:00.521624+0000","last_unstale":"2024-09-17T08:29:54.575647+0000","last_undegraded":"2024-09-17T08:29:54.575647+0000","last_fullsized":"2024-09-17T08:29:54.575647+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:25.259208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:25.259208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021771546999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"220'19","reported_seq":381,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580707+0000","last_change":"2024-09-17T08:28:15.581250+0000","last_active":"2024-09-17T08:29:54.580707+0000","last_peered":"2024-09-17T08:29:54.580707+0000","last_clean":"2024-09-17T08:29:54.580707+0000","last_became_active":"2024-09-17T08:14:58.720897+0000","last_became_peered":"2024-09-17T08:14:58.720897+0000","last_unstale":"2024-09-17T08:29:54.580707+0000","last_undegraded":"2024-09-17T08:29:54.580707+0000","last_fullsized":"2024-09-17T08:29:54.580707+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:15.581112+0000","last_deep_scrub":"177'13","last_deep_scrub_stamp":"2024-09-17T08:17:00.945463+0000","last_clean_scrub_stamp":"2024-09-17T08:28:15.581112+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044324229999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":430,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.576653+0000","last_change":"2024-09-17T08:27:37.220337+0000","last_active":"2024-09-17T08:29:54.576653+0000","last_peered":"2024-09-17T08:29:54.576653+0000","last_clean":"2024-09-17T08:29:54.576653+0000","last_became_active":"2024-09-17T08:09:54.500307+0000","last_became_peered":"2024-09-17T08:09:54.500307+0000","last_unstale":"2024-09-17T08:29:54.576653+0000","last_undegraded":"2024-09-17T08:29:54.576653+0000","last_fullsized":"2024-09-17T08:29:54.576653+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:37.220206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:27:37.220206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076403579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"228'11","reported_seq":682,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.601874+0000","last_change":"2024-09-17T08:28:14.817503+0000","last_active":"2024-09-17T08:29:55.601874+0000","last_peered":"2024-09-17T08:29:55.601874+0000","last_clean":"2024-09-17T08:29:55.601874+0000","last_became_active":"2024-09-17T08:20:23.490028+0000","last_became_peered":"2024-09-17T08:20:23.490028+0000","last_unstale":"2024-09-17T08:29:55.601874+0000","last_undegraded":"2024-09-17T08:29:55.601874+0000","last_fullsized":"2024-09-17T08:29:55.601874+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:28:14.817416+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.817416+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001610317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":429,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575203+0000","last_change":"2024-09-17T08:28:38.263488+0000","last_active":"2024-09-17T08:29:54.575203+0000","last_peered":"2024-09-17T08:29:54.575203+0000","last_clean":"2024-09-17T08:29:54.575203+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:29:54.575203+0000","last_undegraded":"2024-09-17T08:29:54.575203+0000","last_fullsized":"2024-09-17T08:29:54.575203+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_clean_scrub_stamp":"2024-09-17T08:28:38.263303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049197570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"220'15","reported_seq":341,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577401+0000","last_change":"2024-09-17T08:27:07.276176+0000","last_active":"2024-09-17T08:29:54.577401+0000","last_peered":"2024-09-17T08:29:54.577401+0000","last_clean":"2024-09-17T08:29:54.577401+0000","last_became_active":"2024-09-17T08:20:23.496509+0000","last_became_peered":"2024-09-17T08:20:23.496509+0000","last_unstale":"2024-09-17T08:29:54.577401+0000","last_undegraded":"2024-09-17T08:29:54.577401+0000","last_fullsized":"2024-09-17T08:29:54.577401+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'15","last_scrub_stamp":"2024-09-17T08:27:07.275632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:07.275632+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073795509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":223,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582639+0000","last_change":"2024-09-17T08:29:33.504198+0000","last_active":"2024-09-17T08:29:54.582639+0000","last_peered":"2024-09-17T08:29:54.582639+0000","last_clean":"2024-09-17T08:29:54.582639+0000","last_became_active":"2024-09-17T08:27:47.564870+0000","last_became_peered":"2024-09-17T08:27:47.564870+0000","last_unstale":"2024-09-17T08:29:54.582639+0000","last_undegraded":"2024-09-17T08:29:54.582639+0000","last_fullsized":"2024-09-17T08:29:54.582639+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:33.504048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:21.683096+0000","last_clean_scrub_stamp":"2024-09-17T08:29:33.504048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:38.508644+0000","scrub_duration":0.013492753,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":433,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584887+0000","last_change":"2024-09-17T08:27:26.647218+0000","last_active":"2024-09-17T08:29:54.584887+0000","last_peered":"2024-09-17T08:29:54.584887+0000","last_clean":"2024-09-17T08:29:54.584887+0000","last_became_active":"2024-09-17T08:15:29.383852+0000","last_became_peered":"2024-09-17T08:15:29.383852+0000","last_unstale":"2024-09-17T08:29:54.584887+0000","last_undegraded":"2024-09-17T08:29:54.584887+0000","last_fullsized":"2024-09-17T08:29:54.584887+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:26.646984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:27:26.646984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001558357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":442,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575777+0000","last_change":"2024-09-17T08:27:16.007484+0000","last_active":"2024-09-17T08:29:54.575777+0000","last_peered":"2024-09-17T08:29:54.575777+0000","last_clean":"2024-09-17T08:29:54.575777+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:29:54.575777+0000","last_undegraded":"2024-09-17T08:29:54.575777+0000","last_fullsized":"2024-09-17T08:29:54.575777+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:16.007026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:27:16.007026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076085880000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"220'19","reported_seq":502,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574610+0000","last_change":"2024-09-17T08:28:22.312962+0000","last_active":"2024-09-17T08:29:54.574610+0000","last_peered":"2024-09-17T08:29:54.574610+0000","last_clean":"2024-09-17T08:29:54.574610+0000","last_became_active":"2024-09-17T08:15:29.384440+0000","last_became_peered":"2024-09-17T08:15:29.384440+0000","last_unstale":"2024-09-17T08:29:54.574610+0000","last_undegraded":"2024-09-17T08:29:54.574610+0000","last_fullsized":"2024-09-17T08:29:54.574610+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:22.312822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.312822+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031592030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":308,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.578868+0000","last_change":"2024-09-17T08:28:18.412470+0000","last_active":"2024-09-17T08:29:54.578868+0000","last_peered":"2024-09-17T08:29:54.578868+0000","last_clean":"2024-09-17T08:29:54.578868+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:29:54.578868+0000","last_undegraded":"2024-09-17T08:29:54.578868+0000","last_fullsized":"2024-09-17T08:29:54.578868+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.412336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.412336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034612430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":309,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577663+0000","last_change":"2024-09-17T08:28:30.480847+0000","last_active":"2024-09-17T08:29:54.577663+0000","last_peered":"2024-09-17T08:29:54.577663+0000","last_clean":"2024-09-17T08:29:54.577663+0000","last_became_active":"2024-09-17T08:15:29.377320+0000","last_became_peered":"2024-09-17T08:15:29.377320+0000","last_unstale":"2024-09-17T08:29:54.577663+0000","last_undegraded":"2024-09-17T08:29:54.577663+0000","last_fullsized":"2024-09-17T08:29:54.577663+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:30.480721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:30.480721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099748570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":424,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582248+0000","last_change":"2024-09-17T08:28:34.729325+0000","last_active":"2024-09-17T08:29:54.582248+0000","last_peered":"2024-09-17T08:29:54.582248+0000","last_clean":"2024-09-17T08:29:54.582248+0000","last_became_active":"2024-09-17T08:10:00.519016+0000","last_became_peered":"2024-09-17T08:10:00.519016+0000","last_unstale":"2024-09-17T08:29:54.582248+0000","last_undegraded":"2024-09-17T08:29:54.582248+0000","last_fullsized":"2024-09-17T08:29:54.582248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:34.725717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:34.725717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070493120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"220'8","reported_seq":142,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574226+0000","last_change":"2024-09-17T08:27:20.254549+0000","last_active":"2024-09-17T08:29:54.574226+0000","last_peered":"2024-09-17T08:29:54.574226+0000","last_clean":"2024-09-17T08:29:54.574226+0000","last_became_active":"2024-09-17T08:24:04.446708+0000","last_became_peered":"2024-09-17T08:24:04.446708+0000","last_unstale":"2024-09-17T08:29:54.574226+0000","last_undegraded":"2024-09-17T08:29:54.574226+0000","last_fullsized":"2024-09-17T08:29:54.574226+0000","mapping_epoch":287,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'8","last_scrub_stamp":"2024-09-17T08:27:20.254413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:20.254413+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032240540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":182,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582924+0000","last_change":"2024-09-17T08:29:19.309305+0000","last_active":"2024-09-17T08:29:54.582924+0000","last_peered":"2024-09-17T08:29:54.582924+0000","last_clean":"2024-09-17T08:29:54.582924+0000","last_became_active":"2024-09-17T08:27:47.566307+0000","last_became_peered":"2024-09-17T08:27:47.566307+0000","last_unstale":"2024-09-17T08:29:54.582924+0000","last_undegraded":"2024-09-17T08:29:54.582924+0000","last_fullsized":"2024-09-17T08:29:54.582924+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:19.309181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:29:19.309181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:31.089723+0000","scrub_duration":0.0020030260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":223,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582754+0000","last_change":"2024-09-17T08:28:13.150686+0000","last_active":"2024-09-17T08:29:54.582754+0000","last_peered":"2024-09-17T08:29:54.582754+0000","last_clean":"2024-09-17T08:29:54.582754+0000","last_became_active":"2024-09-17T08:27:47.561205+0000","last_became_peered":"2024-09-17T08:27:47.561205+0000","last_unstale":"2024-09-17T08:29:54.582754+0000","last_undegraded":"2024-09-17T08:29:54.582754+0000","last_fullsized":"2024-09-17T08:29:54.582754+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.150568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.150568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023719209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":449,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580112+0000","last_change":"2024-09-17T08:28:36.410539+0000","last_active":"2024-09-17T08:29:54.580112+0000","last_peered":"2024-09-17T08:29:54.580112+0000","last_clean":"2024-09-17T08:29:54.580112+0000","last_became_active":"2024-09-17T08:20:23.484165+0000","last_became_peered":"2024-09-17T08:20:23.484165+0000","last_unstale":"2024-09-17T08:29:54.580112+0000","last_undegraded":"2024-09-17T08:29:54.580112+0000","last_fullsized":"2024-09-17T08:29:54.580112+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:36.410466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:28:36.410466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036944650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"220'19","reported_seq":459,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.582453+0000","last_change":"2024-09-17T08:29:25.017089+0000","last_active":"2024-09-17T08:29:54.582453+0000","last_peered":"2024-09-17T08:29:54.582453+0000","last_clean":"2024-09-17T08:29:54.582453+0000","last_became_active":"2024-09-17T08:09:57.041963+0000","last_became_peered":"2024-09-17T08:09:57.041963+0000","last_unstale":"2024-09-17T08:29:54.582453+0000","last_undegraded":"2024-09-17T08:29:54.582453+0000","last_fullsized":"2024-09-17T08:29:54.582453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:29:25.016896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:29:25.016896+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:28.910165+0000","scrub_duration":0.0046246280000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":244,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583962+0000","last_change":"2024-09-17T08:28:39.133714+0000","last_active":"2024-09-17T08:29:54.583962+0000","last_peered":"2024-09-17T08:29:54.583962+0000","last_clean":"2024-09-17T08:29:54.583962+0000","last_became_active":"2024-09-17T08:27:47.564429+0000","last_became_peered":"2024-09-17T08:27:47.564429+0000","last_unstale":"2024-09-17T08:29:54.583962+0000","last_undegraded":"2024-09-17T08:29:54.583962+0000","last_fullsized":"2024-09-17T08:29:54.583962+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:28:39.133574+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:11:19.313315+0000","last_clean_scrub_stamp":"2024-09-17T08:28:39.133574+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:58.000527+0000","scrub_duration":0.0064456499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":436,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.579955+0000","last_change":"2024-09-17T08:28:22.551519+0000","last_active":"2024-09-17T08:29:54.579955+0000","last_peered":"2024-09-17T08:29:54.579955+0000","last_clean":"2024-09-17T08:29:54.579955+0000","last_became_active":"2024-09-17T08:09:58.513420+0000","last_became_peered":"2024-09-17T08:09:58.513420+0000","last_unstale":"2024-09-17T08:29:54.579955+0000","last_undegraded":"2024-09-17T08:29:54.579955+0000","last_fullsized":"2024-09-17T08:29:54.579955+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:22.551401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:22.551401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030279500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":413,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.584755+0000","last_change":"2024-09-17T08:27:29.641693+0000","last_active":"2024-09-17T08:29:54.584755+0000","last_peered":"2024-09-17T08:29:54.584755+0000","last_clean":"2024-09-17T08:29:54.584755+0000","last_became_active":"2024-09-17T08:10:00.598772+0000","last_became_peered":"2024-09-17T08:10:00.598772+0000","last_unstale":"2024-09-17T08:29:54.584755+0000","last_undegraded":"2024-09-17T08:29:54.584755+0000","last_fullsized":"2024-09-17T08:29:54.584755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_clean_scrub_stamp":"2024-09-17T08:27:29.641570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060037839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"72'2","reported_seq":450,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.580505+0000","last_change":"2024-09-17T08:29:18.410850+0000","last_active":"2024-09-17T08:29:54.580505+0000","last_peered":"2024-09-17T08:29:54.580505+0000","last_clean":"2024-09-17T08:29:54.580505+0000","last_became_active":"2024-09-17T08:20:23.489251+0000","last_became_peered":"2024-09-17T08:20:23.489251+0000","last_unstale":"2024-09-17T08:29:54.580505+0000","last_undegraded":"2024-09-17T08:29:54.580505+0000","last_fullsized":"2024-09-17T08:29:54.580505+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'2","last_scrub_stamp":"2024-09-17T08:29:18.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:29:18.410694+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:44.856880+0000","scrub_duration":0.0069667799999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":330,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.574437+0000","last_change":"2024-09-17T08:28:27.267072+0000","last_active":"2024-09-17T08:29:54.574437+0000","last_peered":"2024-09-17T08:29:54.574437+0000","last_clean":"2024-09-17T08:29:54.574437+0000","last_became_active":"2024-09-17T08:14:58.714735+0000","last_became_peered":"2024-09-17T08:14:58.714735+0000","last_unstale":"2024-09-17T08:29:54.574437+0000","last_undegraded":"2024-09-17T08:29:54.574437+0000","last_fullsized":"2024-09-17T08:29:54.574437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:27.266982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:27.266982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011529369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"220'20","reported_seq":459,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577803+0000","last_change":"2024-09-17T08:27:11.144999+0000","last_active":"2024-09-17T08:29:54.577803+0000","last_peered":"2024-09-17T08:29:54.577803+0000","last_clean":"2024-09-17T08:29:54.577803+0000","last_became_active":"2024-09-17T08:09:57.042271+0000","last_became_peered":"2024-09-17T08:09:57.042271+0000","last_unstale":"2024-09-17T08:29:54.577803+0000","last_undegraded":"2024-09-17T08:29:54.577803+0000","last_fullsized":"2024-09-17T08:29:54.577803+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'20","last_scrub_stamp":"2024-09-17T08:27:11.144705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:27:11.144705+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013738378000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"353'10","reported_seq":196,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583800+0000","last_change":"2024-09-17T08:28:20.143550+0000","last_active":"2024-09-17T08:29:54.583800+0000","last_peered":"2024-09-17T08:29:54.583800+0000","last_clean":"2024-09-17T08:29:54.583800+0000","last_became_active":"2024-09-17T08:27:47.561825+0000","last_became_peered":"2024-09-17T08:27:47.561825+0000","last_unstale":"2024-09-17T08:29:54.583800+0000","last_undegraded":"2024-09-17T08:29:54.583800+0000","last_fullsized":"2024-09-17T08:29:54.583800+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'10","last_scrub_stamp":"2024-09-17T08:28:20.143416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.143416+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035354610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":319,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.578049+0000","last_change":"2024-09-17T08:29:21.488534+0000","last_active":"2024-09-17T08:29:54.578049+0000","last_peered":"2024-09-17T08:29:54.578049+0000","last_clean":"2024-09-17T08:29:54.578049+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:29:54.578049+0000","last_undegraded":"2024-09-17T08:29:54.578049+0000","last_fullsized":"2024-09-17T08:29:54.578049+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:21.488419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:29:21.488419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:36.446433+0000","scrub_duration":0.0038677669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":430,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583054+0000","last_change":"2024-09-17T08:28:32.738220+0000","last_active":"2024-09-17T08:29:54.583054+0000","last_peered":"2024-09-17T08:29:54.583054+0000","last_clean":"2024-09-17T08:29:54.583054+0000","last_became_active":"2024-09-17T08:09:54.504526+0000","last_became_peered":"2024-09-17T08:09:54.504526+0000","last_unstale":"2024-09-17T08:29:54.583054+0000","last_undegraded":"2024-09-17T08:29:54.583054+0000","last_fullsized":"2024-09-17T08:29:54.583054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:32.738028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:28:32.738028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014925677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"220'19","reported_seq":238,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583884+0000","last_change":"2024-09-17T08:28:14.189235+0000","last_active":"2024-09-17T08:29:54.583884+0000","last_peered":"2024-09-17T08:29:54.583884+0000","last_clean":"2024-09-17T08:29:54.583884+0000","last_became_active":"2024-09-17T08:27:47.563331+0000","last_became_peered":"2024-09-17T08:27:47.563331+0000","last_unstale":"2024-09-17T08:29:54.583884+0000","last_undegraded":"2024-09-17T08:29:54.583884+0000","last_fullsized":"2024-09-17T08:29:54.583884+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'19","last_scrub_stamp":"2024-09-17T08:28:14.189064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:14.189064+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022061400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":455,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.577638+0000","last_change":"2024-09-17T08:29:16.813853+0000","last_active":"2024-09-17T08:29:54.577638+0000","last_peered":"2024-09-17T08:29:54.577638+0000","last_clean":"2024-09-17T08:29:54.577638+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:29:54.577638+0000","last_undegraded":"2024-09-17T08:29:54.577638+0000","last_fullsized":"2024-09-17T08:29:54.577638+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:29:16.813680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:29:16.813680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:40.127925+0000","scrub_duration":0.0063677899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":414,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575208+0000","last_change":"2024-09-17T08:28:13.307062+0000","last_active":"2024-09-17T08:29:54.575208+0000","last_peered":"2024-09-17T08:29:54.575208+0000","last_clean":"2024-09-17T08:29:54.575208+0000","last_became_active":"2024-09-17T08:10:00.597588+0000","last_became_peered":"2024-09-17T08:10:00.597588+0000","last_unstale":"2024-09-17T08:29:54.575208+0000","last_undegraded":"2024-09-17T08:29:54.575208+0000","last_fullsized":"2024-09-17T08:29:54.575208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:13.306936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:28:13.306936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001457446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":117,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.585598+0000","last_change":"2024-09-17T08:28:18.200418+0000","last_active":"2024-09-17T08:29:54.585598+0000","last_peered":"2024-09-17T08:29:54.585598+0000","last_clean":"2024-09-17T08:29:54.585598+0000","last_became_active":"2024-09-17T08:27:47.563398+0000","last_became_peered":"2024-09-17T08:27:47.563398+0000","last_unstale":"2024-09-17T08:29:54.585598+0000","last_undegraded":"2024-09-17T08:29:54.585598+0000","last_fullsized":"2024-09-17T08:29:54.585598+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:18.200302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:28:18.200302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003324605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"220'34","reported_seq":546,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.588558+0000","last_change":"2024-09-17T08:28:19.204373+0000","last_active":"2024-09-17T08:29:54.588558+0000","last_peered":"2024-09-17T08:29:54.588558+0000","last_clean":"2024-09-17T08:29:54.588558+0000","last_became_active":"2024-09-17T08:20:23.493953+0000","last_became_peered":"2024-09-17T08:20:23.493953+0000","last_unstale":"2024-09-17T08:29:54.588558+0000","last_undegraded":"2024-09-17T08:29:54.588558+0000","last_fullsized":"2024-09-17T08:29:54.588558+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'34","last_scrub_stamp":"2024-09-17T08:28:19.204197+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:28:19.204197+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095927449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":308,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.578192+0000","last_change":"2024-09-17T08:27:39.265326+0000","last_active":"2024-09-17T08:29:54.578192+0000","last_peered":"2024-09-17T08:29:54.578192+0000","last_clean":"2024-09-17T08:29:54.578192+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:29:54.578192+0000","last_undegraded":"2024-09-17T08:29:54.578192+0000","last_fullsized":"2024-09-17T08:29:54.578192+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:27:39.265233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:34.089010+0000","last_clean_scrub_stamp":"2024-09-17T08:27:39.265233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013430779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":414,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583607+0000","last_change":"2024-09-17T08:28:23.645255+0000","last_active":"2024-09-17T08:29:54.583607+0000","last_peered":"2024-09-17T08:29:54.583607+0000","last_clean":"2024-09-17T08:29:54.583607+0000","last_became_active":"2024-09-17T08:10:00.597687+0000","last_became_peered":"2024-09-17T08:10:00.597687+0000","last_unstale":"2024-09-17T08:29:54.583607+0000","last_undegraded":"2024-09-17T08:29:54.583607+0000","last_fullsized":"2024-09-17T08:29:54.583607+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:23.645106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:28:23.645106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036194209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":430,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-17T08:29:55.600419+0000","last_change":"2024-09-17T08:28:20.695325+0000","last_active":"2024-09-17T08:29:55.600419+0000","last_peered":"2024-09-17T08:29:55.600419+0000","last_clean":"2024-09-17T08:29:55.600419+0000","last_became_active":"2024-09-17T08:09:54.498338+0000","last_became_peered":"2024-09-17T08:09:54.498338+0000","last_unstale":"2024-09-17T08:29:55.600419+0000","last_undegraded":"2024-09-17T08:29:55.600419+0000","last_fullsized":"2024-09-17T08:29:55.600419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:20.695195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:28:20.695195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052696519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"220'18","reported_seq":501,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.583579+0000","last_change":"2024-09-17T08:28:31.708556+0000","last_active":"2024-09-17T08:29:54.583579+0000","last_peered":"2024-09-17T08:29:54.583579+0000","last_clean":"2024-09-17T08:29:54.583579+0000","last_became_active":"2024-09-17T08:15:29.384606+0000","last_became_peered":"2024-09-17T08:15:29.384606+0000","last_unstale":"2024-09-17T08:29:54.583579+0000","last_undegraded":"2024-09-17T08:29:54.583579+0000","last_fullsized":"2024-09-17T08:29:54.583579+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"220'18","last_scrub_stamp":"2024-09-17T08:28:31.708312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:28:31.708312+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060376420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":441,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-17T08:29:54.575035+0000","last_change":"2024-09-17T08:28:29.221115+0000","last_active":"2024-09-17T08:29:54.575035+0000","last_peered":"2024-09-17T08:29:54.575035+0000","last_clean":"2024-09-17T08:29:54.575035+0000","last_became_active":"2024-09-17T08:20:23.490472+0000","last_became_peered":"2024-09-17T08:20:23.490472+0000","last_unstale":"2024-09-17T08:29:54.575035+0000","last_undegraded":"2024-09-17T08:29:54.575035+0000","last_fullsized":"2024-09-17T08:29:54.575035+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:28:29.221035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:29.221035+0000","last_clean_scrub_stamp":"2024-09-17T08:28:29.221035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014868539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":7516782616,"num_objects":114698,"num_object_clones":0,"num_object_copies":344094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143054,"num_write_kb":8248001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22596038656,"data_stored":22596026440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79646,"ondisk_log_size":79646,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1120,"num_read_kb":877,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":4236,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":865,"num_read_kb":860,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1430,"ondisk_log_size":1430,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070710,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3501224,"kb_used_data":2788380,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":90243928,"statfs":{"total":95995035648,"available":92409782272,"internally_reserved":0,"allocated":2855301120,"data_stored":2854979731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":729938379},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":1.1579999999999999,"15min":1.3839999999999999},"min":{"1min":0.23599999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":1.3049999999999999,"5min":6.2460000000000004,"15min":62.018000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.0289999999999999,"15min":1.508},"min":{"1min":0.249,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":1.246,"5min":9.4169999999999998,"15min":61.896999999999998},"last":0.47799999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.85699999999999998,"15min":1.4810000000000001},"min":{"1min":0.26700000000000002,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":2.7389999999999999,"5min":5.8369999999999997,"15min":62.067},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.755,"15min":1.472},"min":{"1min":0.26100000000000001,"5min":0.252,"15min":0.23599999999999999},"max":{"1min":2.1819999999999999,"5min":5.5179999999999998,"15min":62.194000000000003},"last":1.571}]},{"osd":2,"last update":"Tue Sep 17 08:29:38 2024","interfaces":[{"interface":"back","average":{"1min":4.2779999999999996,"5min":1.621,"15min":1.149},"min":{"1min":0.30199999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":42.829999999999998,"5min":42.829999999999998,"15min":42.829999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":3.7450000000000001,"5min":1.5860000000000001,"15min":1.0069999999999999},"min":{"1min":0.28399999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":42.459000000000003,"5min":42.459000000000003,"15min":42.459000000000003},"last":1.8500000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.8460000000000001,"15min":1.6439999999999999},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":2.0049999999999999,"5min":74.376000000000005,"15min":74.376000000000005},"last":1.952},{"interface":"front","average":{"1min":1.575,"5min":1.792,"15min":1.7470000000000001},"min":{"1min":0.36399999999999999,"5min":0.27600000000000002,"15min":0.27300000000000002},"max":{"1min":8.9570000000000007,"5min":75.183000000000007,"15min":75.183000000000007},"last":2.2679999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.623,"15min":0.72499999999999998},"min":{"1min":0.28499999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.1240000000000001,"5min":4.3719999999999999,"15min":5.335},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.66100000000000003,"15min":0.64900000000000002},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.23899999999999999},"max":{"1min":1.593,"5min":2.798,"15min":5.9669999999999996},"last":1.8240000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.89700000000000002,"15min":0.88300000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.84399999999999997,"5min":34.436,"15min":34.436},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.93600000000000005,"15min":0.88300000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.294,"5min":34.965000000000003,"15min":34.965000000000003},"last":0.43099999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.80100000000000005},"min":{"1min":0.29199999999999998,"5min":0.28799999999999998,"15min":0.26600000000000001},"max":{"1min":1.264,"5min":2.4199999999999999,"15min":7.2359999999999998},"last":0.316},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.246},"max":{"1min":1.865,"5min":3.1819999999999999,"15min":5.9400000000000004},"last":1.7889999999999999}]}]},{"osd":6,"up_from":50,"seq":214748365051,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4340968,"kb_used_data":3732312,"kb_used_omap":1,"kb_used_meta":608638,"kb_avail":89404184,"statfs":{"total":95995035648,"available":91549884416,"internally_reserved":0,"allocated":3821887488,"data_stored":3821569571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":623245767},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,3,1,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:54 2024","interfaces":[{"interface":"back","average":{"1min":2.081,"5min":3.1480000000000001,"15min":2.3300000000000001},"min":{"1min":0.29499999999999998,"5min":0.214,"15min":0.20100000000000001},"max":{"1min":6.2469999999999999,"5min":104.185,"15min":104.185},"last":1.05},{"interface":"front","average":{"1min":1.1899999999999999,"5min":2.7669999999999999,"15min":2.2360000000000002},"min":{"1min":0.26800000000000002,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":4.9320000000000004,"5min":104.15300000000001,"15min":104.15300000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:29:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.133,"15min":1.6539999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.22500000000000001},"max":{"1min":5.9240000000000004,"5min":6.3029999999999999,"15min":80.813000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.3839999999999999,"5min":0.91200000000000003,"15min":1.653},"min":{"1min":0.25900000000000001,"5min":0.255,"15min":0.24299999999999999},"max":{"1min":4.5129999999999999,"5min":4.8890000000000002,"15min":80.281000000000006},"last":1.016}]},{"osd":2,"last update":"Tue Sep 17 08:29:39 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.0569999999999999,"15min":1.1739999999999999},"min":{"1min":0.28100000000000003,"5min":0.254,"15min":0.254},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":70.451999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.133},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":4.4299999999999997,"5min":4.9180000000000001,"15min":70.488},"last":2.6829999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:29:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.121,"15min":1.5880000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":6.4199999999999999,"5min":6.4199999999999999,"15min":80.427000000000007},"last":0.629},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.1679999999999999,"15min":1.54},"min":{"1min":0.308,"5min":0.29799999999999999,"15min":0.26000000000000001},"max":{"1min":4.5350000000000001,"5min":6.1580000000000004,"15min":80.379999999999995},"last":0.98299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:29:54 2024","interfaces":[{"interface":"back","average":{"1min":8.6039999999999992,"5min":2.5110000000000001,"15min":1.27},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.215},"max":{"1min":125.25700000000001,"5min":125.25700000000001,"15min":125.25700000000001},"last":0.496},{"interface":"front","average":{"1min":9.1349999999999998,"5min":2.7349999999999999,"15min":1.3129999999999999},"min":{"1min":0.313,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":126.673,"5min":126.673,"15min":126.673},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:29:54 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.76800000000000002,"15min":0.746},"min":{"1min":0.32500000000000001,"5min":0.25600000000000001,"15min":0.245},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":6.1859999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.121,"5min":0.73299999999999998,"15min":0.68400000000000005},"min":{"1min":0.28699999999999998,"5min":0.26900000000000002,"15min":0.26200000000000001},"max":{"1min":5.2149999999999999,"5min":5.2240000000000002,"15min":5.2240000000000002},"last":0.68500000000000005}]},{"osd":7,"last update":"Tue Sep 17 08:29:17 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.69099999999999995,"15min":0.72999999999999998},"min":{"1min":0.39300000000000002,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":4.8090000000000002,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.35899999999999999,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":4.6399999999999997,"5min":4.6399999999999997,"15min":5.0599999999999996},"last":0.82699999999999996}]}]},{"osd":5,"up_from":41,"seq":176093659392,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2546280,"kb_used_data":1861340,"kb_used_omap":1,"kb_used_meta":684926,"kb_avail":91198872,"statfs":{"total":95995035648,"available":93387644928,"internally_reserved":0,"allocated":1906012160,"data_stored":1905690616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":701364678},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":3.2469999999999999,"15min":2.423},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":5.4829999999999997,"5min":64.218999999999994,"15min":65.941000000000003},"last":1.085},{"interface":"front","average":{"1min":0.79100000000000004,"5min":3.2069999999999999,"15min":2.3969999999999998},"min":{"1min":0.28100000000000003,"5min":0.218,"15min":0.218},"max":{"1min":4.851,"5min":65.405000000000001,"15min":66.135000000000005},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:29:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.9950000000000001,"15min":1.7709999999999999},"min":{"1min":0.33500000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.8100000000000001,"5min":50.426000000000002,"15min":66.981999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.238,"5min":2.1389999999999998,"15min":1.8899999999999999},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.22},"max":{"1min":4.8799999999999999,"5min":50.744,"15min":66.316000000000003},"last":0.29999999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.97,"15min":1.8109999999999999},"min":{"1min":0.25700000000000001,"5min":0.255,"15min":0.221},"max":{"1min":2.9929999999999999,"5min":50.518000000000001,"15min":131.727},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":2.056,"15min":2.032},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.27200000000000002},"max":{"1min":3.1309999999999998,"5min":50.204000000000001,"15min":131.50399999999999},"last":0.44}]},{"osd":3,"last update":"Tue Sep 17 08:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":2.298,"15min":2.1179999999999999},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":3.5910000000000002,"5min":50.450000000000003,"15min":75.286000000000001},"last":1.163},{"interface":"front","average":{"1min":1.446,"5min":2.3140000000000001,"15min":2.1110000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.1470000000000002,"5min":50.469000000000001,"15min":77.206999999999994},"last":1.236}]},{"osd":4,"last update":"Tue Sep 17 08:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.81999999999999995,"15min":0.85499999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":1.02,"5min":6.3600000000000003,"15min":42.421999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":2.085,"5min":6.0899999999999999,"15min":42.191000000000003},"last":0.38600000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.79700000000000004,"15min":0.71099999999999997},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.4199999999999999,"5min":7.8399999999999999,"15min":7.8399999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.85099999999999998,"15min":0.74399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26700000000000002},"max":{"1min":1.081,"5min":5.883,"15min":5.883},"last":0.47399999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":1.0900000000000001,"15min":0.91000000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":25.085000000000001,"5min":25.085000000000001,"15min":25.085000000000001},"last":0.503},{"interface":"front","average":{"1min":1.7769999999999999,"5min":0.97799999999999998,"15min":0.83399999999999996},"min":{"1min":0.32100000000000001,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":23.561,"5min":23.561,"15min":23.561},"last":0.59699999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953733,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2387568,"kb_used_data":1837160,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":91357584,"statfs":{"total":95995035648,"available":93550166016,"internally_reserved":0,"allocated":1881251840,"data_stored":1880918117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":563608006},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":1.2070000000000001,"15min":1.048},"min":{"1min":0.24399999999999999,"5min":0.21099999999999999,"15min":0.19600000000000001},"max":{"1min":1.8400000000000001,"5min":20.271000000000001,"15min":23.786000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.224,"15min":1.1020000000000001},"min":{"1min":0.28999999999999998,"5min":0.23499999999999999,"15min":0.219},"max":{"1min":2.2919999999999998,"5min":19.798999999999999,"15min":23.245000000000001},"last":2.7519999999999998}]},{"osd":1,"last update":"Tue Sep 17 08:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.872,"15min":1.2190000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.23200000000000001},"max":{"1min":3.2879999999999998,"5min":5.5839999999999996,"15min":23.135000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.2170000000000001},"min":{"1min":0.29799999999999999,"5min":0.25800000000000001,"15min":0.246},"max":{"1min":2.5059999999999998,"5min":8.1959999999999997,"15min":23.434999999999999},"last":4.2329999999999997}]},{"osd":2,"last update":"Tue Sep 17 08:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.008,"15min":0.75},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":3.1349999999999998,"5min":3.9910000000000001,"15min":3.9910000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.874,"15min":0.77400000000000002},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":2.2989999999999999,"5min":3.6499999999999999,"15min":3.6499999999999999},"last":2.004}]},{"osd":3,"last update":"Tue Sep 17 08:29:01 2024","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.3009999999999999,"15min":1.9159999999999999},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":12.984999999999999,"5min":28.138000000000002,"15min":154.137},"last":2.8010000000000002},{"interface":"front","average":{"1min":1.343,"5min":1.274,"15min":1.8480000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":13.205,"5min":28.117000000000001,"15min":153.54400000000001},"last":1.6819999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:29:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72499999999999998,"15min":0.68999999999999995},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.23000000000000001},"max":{"1min":3.1600000000000001,"5min":4.093,"15min":6.2329999999999997},"last":4.2640000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.70099999999999996},"min":{"1min":0.29799999999999999,"5min":0.27400000000000002,"15min":0.26700000000000002},"max":{"1min":2.3300000000000001,"5min":3.9409999999999998,"15min":4.5300000000000002},"last":2.718}]},{"osd":6,"last update":"Tue Sep 17 08:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70599999999999996,"15min":0.66800000000000004},"min":{"1min":0.316,"5min":0.25900000000000001,"15min":0.252},"max":{"1min":2.9649999999999999,"5min":4.2880000000000003,"15min":4.827},"last":1.7050000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.80500000000000005,"15min":0.71799999999999997},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.28000000000000003},"max":{"1min":3.8929999999999998,"5min":3.8929999999999998,"15min":6.0010000000000003},"last":2.379}]},{"osd":7,"last update":"Tue Sep 17 08:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67700000000000005,"15min":0.75600000000000001},"min":{"1min":0.33100000000000002,"5min":0.27400000000000002,"15min":0.26800000000000002},"max":{"1min":2.3540000000000001,"5min":4.1719999999999997,"15min":4.875},"last":2.7789999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.67600000000000005,"15min":0.73399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28199999999999997},"max":{"1min":1.0369999999999999,"5min":2.0249999999999999,"15min":4.6950000000000003},"last":3.3069999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117257,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3943104,"kb_used_data":3208116,"kb_used_omap":1,"kb_used_meta":734974,"kb_avail":89802048,"statfs":{"total":95995035648,"available":91957297152,"internally_reserved":0,"allocated":3285110784,"data_stored":3284764770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":752613832},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:47 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":2.972,"15min":2.2519999999999998},"min":{"1min":0.152,"5min":0.152,"15min":0.152},"max":{"1min":4.7000000000000002,"5min":130.40299999999999,"15min":130.40299999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.649,"5min":2.6059999999999999,"15min":2.0329999999999999},"min":{"1min":0.186,"5min":0.186,"15min":0.17399999999999999},"max":{"1min":5.4000000000000004,"5min":130.70099999999999,"15min":130.70099999999999},"last":0.86799999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:29:47 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":0.92500000000000004,"15min":1.5940000000000001},"min":{"1min":0.308,"5min":0.214,"15min":0.214},"max":{"1min":5.4560000000000004,"5min":5.4560000000000004,"15min":71.459999999999994},"last":2.0299999999999998},{"interface":"front","average":{"1min":1.661,"5min":1.2749999999999999,"15min":1.7729999999999999},"min":{"1min":0.28799999999999998,"5min":0.218,"15min":0.192},"max":{"1min":7.6790000000000003,"5min":7.6790000000000003,"15min":71.438999999999993},"last":0.79800000000000004}]},{"osd":2,"last update":"Tue Sep 17 08:29:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.613,"15min":1.4279999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.7130000000000001,"5min":17.097999999999999,"15min":60.423999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.6379999999999999,"15min":1.726},"min":{"1min":0.51700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.7370000000000001,"5min":14.925000000000001,"15min":60.368000000000002},"last":1.109}]},{"osd":4,"last update":"Tue Sep 17 08:29:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.524,"15min":1.5169999999999999},"min":{"1min":0.33700000000000002,"5min":0.32300000000000001,"15min":0.27200000000000002},"max":{"1min":2.819,"5min":30.722999999999999,"15min":35.841000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.363,"15min":1.3819999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.27200000000000002},"max":{"1min":3.7570000000000001,"5min":30.884,"15min":35.857999999999997},"last":0.93700000000000006}]},{"osd":5,"last update":"Tue Sep 17 08:29:26 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.2809999999999999},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.29799999999999999},"max":{"1min":6.9089999999999998,"5min":6.9089999999999998,"15min":8.2240000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2390000000000001,"15min":1.4370000000000001},"min":{"1min":0.33800000000000002,"5min":0.32600000000000001,"15min":0.29499999999999998},"max":{"1min":4.8769999999999998,"5min":5.5759999999999996,"15min":9.7289999999999992},"last":1.0449999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:28:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.92600000000000005,"15min":1.151},"min":{"1min":0.33200000000000002,"5min":0.32300000000000001,"15min":0.29499999999999998},"max":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":8.4550000000000001},"last":0.997},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.264,"15min":1.5109999999999999},"min":{"1min":0.39300000000000002,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":3.5219999999999998,"5min":5.5430000000000001,"15min":9.6600000000000001},"last":1.0089999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.90500000000000003,"15min":1.7609999999999999},"min":{"1min":0.41899999999999998,"5min":0.35399999999999998,"15min":0.316},"max":{"1min":1.7549999999999999,"5min":4.4720000000000004,"15min":163.006},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2569999999999999,"15min":1.903},"min":{"1min":0.432,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":6.2809999999999997,"5min":6.2809999999999997,"15min":162.83699999999999},"last":0.90700000000000003}]}]},{"osd":2,"up_from":227,"seq":974957576308,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3195640,"kb_used_data":2842988,"kb_used_omap":4,"kb_used_meta":352635,"kb_avail":90549512,"statfs":{"total":95995035648,"available":92722700288,"internally_reserved":0,"allocated":2911219712,"data_stored":2910882298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":361098915},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.7290000000000001,"15min":1.256},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":4.0750000000000002,"5min":9.6389999999999993,"15min":9.6389999999999993},"last":4.5620000000000003},{"interface":"front","average":{"1min":1.756,"5min":1.8260000000000001,"15min":1.4419999999999999},"min":{"1min":0.32300000000000001,"5min":0.223,"15min":0.184},"max":{"1min":6.2220000000000004,"5min":10.237,"15min":10.237},"last":1.4319999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":2.375,"5min":1.4159999999999999,"15min":1.4750000000000001},"min":{"1min":0.34699999999999998,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":6.6420000000000003,"5min":6.6420000000000003,"15min":41.405000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.712,"5min":1.5249999999999999,"15min":1.522},"min":{"1min":0.217,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":4.1779999999999999,"5min":5.617,"15min":41.25},"last":1.9850000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6879999999999999,"15min":1.5580000000000001},"min":{"1min":0.22600000000000001,"5min":0.218,"15min":0.21299999999999999},"max":{"1min":5.702,"5min":21.477,"15min":44.185000000000002},"last":1.464},{"interface":"front","average":{"1min":2.2669999999999999,"5min":1.921,"15min":1.631},"min":{"1min":0.374,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":11.852,"5min":21.422000000000001,"15min":43.994},"last":1.2290000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":2.54,"15min":1.403},"min":{"1min":0.27800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.0709999999999997,"5min":62.298999999999999,"15min":62.298999999999999},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":2.339,"15min":1.3240000000000001},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":2.8580000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":1.528}]},{"osd":5,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1859999999999999,"15min":1.01},"min":{"1min":0.33000000000000002,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":4.0940000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"last":1.964},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0600000000000001,"15min":0.96399999999999997},"min":{"1min":0.33700000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.9180000000000001,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":1.55}]},{"osd":6,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.3149999999999999,"15min":0.94299999999999995},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":5.1760000000000002,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3400000000000001,"15min":1.03},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.5179999999999998,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":1.9119999999999999}]},{"osd":7,"last update":"Tue Sep 17 08:29:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2330000000000001,"15min":1.0509999999999999},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.742,"5min":8.9849999999999994,"15min":8.9849999999999994},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.694,"5min":1.323,"15min":1.044},"min":{"1min":0.45100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.1109999999999998,"5min":7.468,"15min":7.468},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834575123,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4462272,"kb_used_data":3659192,"kb_used_omap":1,"kb_used_meta":803070,"kb_avail":89282880,"statfs":{"total":95995035648,"available":91425669120,"internally_reserved":0,"allocated":3747012608,"data_stored":3746662544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":822344134},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:29:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.923,"15min":1.7509999999999999},"min":{"1min":0.26900000000000002,"5min":0.16200000000000001,"15min":0.155},"max":{"1min":3.6019999999999999,"5min":27.670999999999999,"15min":31.585999999999999},"last":1.696},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.72,"15min":1.857},"min":{"1min":0.34999999999999998,"5min":0.215,"15min":0.17199999999999999},"max":{"1min":2.8530000000000002,"5min":26.120999999999999,"15min":35.807000000000002},"last":1.427}]},{"osd":2,"last update":"Tue Sep 17 08:29:39 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.6259999999999999,"15min":1.1990000000000001},"min":{"1min":0.29999999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":6,"5min":26.297999999999998,"15min":26.297999999999998},"last":1.71},{"interface":"front","average":{"1min":2.5310000000000001,"5min":1.879,"15min":1.5669999999999999},"min":{"1min":0.31900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":8.9900000000000002,"5min":26.602,"15min":26.602},"last":0.85099999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:29:50 2024","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":1.2569999999999999,"15min":1.401},"min":{"1min":0.25600000000000001,"5min":0.223,"15min":0.184},"max":{"1min":4.9450000000000003,"5min":5.3739999999999997,"15min":8.0559999999999992},"last":4.5279999999999996},{"interface":"front","average":{"1min":1.538,"5min":1.129,"15min":1.571},"min":{"1min":0.40600000000000003,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":5.9749999999999996,"5min":5.9749999999999996,"15min":22.984999999999999},"last":2.5960000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:29:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.196,"15min":1.3640000000000001},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.25800000000000001},"max":{"1min":2.7330000000000001,"5min":14.109,"15min":16.884},"last":1.635},{"interface":"front","average":{"1min":1.389,"5min":1.131,"15min":1.2250000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":3.552,"5min":14.151999999999999,"15min":14.151999999999999},"last":1.6719999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:29:31 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.98499999999999999,"15min":1.2030000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.254},"max":{"1min":3.0249999999999999,"5min":4.2460000000000004,"15min":7.9139999999999997},"last":4.4909999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":0.96899999999999997,"15min":1.3500000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":4.3479999999999999,"5min":6.1520000000000001,"15min":16.853999999999999},"last":1.6850000000000001}]},{"osd":6,"last update":"Tue Sep 17 08:29:50 2024","interfaces":[{"interface":"back","average":{"1min":3.8730000000000002,"5min":1.665,"15min":1.4770000000000001},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":37.799999999999997,"5min":37.799999999999997,"15min":37.799999999999997},"last":1.6100000000000001},{"interface":"front","average":{"1min":3.5910000000000002,"5min":1.5580000000000001,"15min":1.4379999999999999},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.27200000000000002},"max":{"1min":37.835000000000001,"5min":37.835000000000001,"15min":37.835000000000001},"last":4.5590000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.0609999999999999,"15min":1.5800000000000001},"min":{"1min":0.46300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.3920000000000003,"5min":6.1260000000000003,"15min":47.442},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.772,"5min":1.1579999999999999,"15min":1.492},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":5.242,"5min":5.242,"15min":47.475000000000001},"last":1.651}]}]},{"osd":0,"up_from":8,"seq":34359738645,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3048168,"kb_used_data":2309088,"kb_used_omap":1,"kb_used_meta":739070,"kb_avail":90696984,"statfs":{"total":95995035648,"available":92873711616,"internally_reserved":0,"allocated":2364506112,"data_stored":2364180403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":756808134},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:29:12 2024","interfaces":[{"interface":"back","average":{"1min":2.496,"5min":1.4490000000000001,"15min":2.0449999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":21.106000000000002,"5min":21.106000000000002,"15min":61.649000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":2.399,"5min":1.659,"15min":2.214},"min":{"1min":0.182,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":20.667000000000002,"5min":20.667000000000002,"15min":61.281999999999996},"last":1.2689999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:29:41 2024","interfaces":[{"interface":"back","average":{"1min":3.113,"5min":3.2480000000000002,"15min":2.262},"min":{"1min":0.22,"5min":0.19900000000000001,"15min":0.17799999999999999},"max":{"1min":40.359000000000002,"5min":114.771,"15min":114.771},"last":0.35099999999999998},{"interface":"front","average":{"1min":2.7770000000000001,"5min":3.2040000000000002,"15min":2.1850000000000001},"min":{"1min":0.20899999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":40.417000000000002,"5min":114.732,"15min":114.732},"last":1.1180000000000001}]},{"osd":3,"last update":"Tue Sep 17 08:29:50 2024","interfaces":[{"interface":"back","average":{"1min":4.8920000000000003,"5min":2.5819999999999999,"15min":2.3820000000000001},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20399999999999999},"max":{"1min":44.506,"5min":44.506,"15min":61.926000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":4.3920000000000003,"5min":1.9159999999999999,"15min":1.99},"min":{"1min":0.26300000000000001,"5min":0.22,"15min":0.22},"max":{"1min":45.286000000000001,"5min":45.286000000000001,"15min":61.616},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 17 08:29:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.7689999999999999,"15min":1.5669999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26300000000000001},"max":{"1min":2.6829999999999998,"5min":54.176000000000002,"15min":54.176000000000002},"last":1.24},{"interface":"front","average":{"1min":1.24,"5min":1.9159999999999999,"15min":1.6439999999999999},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":3.3679999999999999,"5min":54.097000000000001,"15min":54.097000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:29:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":1.1699999999999999,"15min":1.3129999999999999},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.472,"5min":10.805999999999999,"15min":39.058},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":1.7589999999999999,"15min":1.6359999999999999},"min":{"1min":0.32700000000000001,"5min":0.307,"15min":0.25600000000000001},"max":{"1min":1.742,"5min":54.116,"15min":54.116},"last":1.1739999999999999}]},{"osd":6,"last update":"Tue Sep 17 08:29:56 2024","interfaces":[{"interface":"back","average":{"1min":4.3650000000000002,"5min":1.873,"15min":1.534},"min":{"1min":0.312,"5min":0.30599999999999999,"15min":0.27700000000000002},"max":{"1min":72.004000000000005,"5min":72.004000000000005,"15min":72.004000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":4.5940000000000003,"5min":2.1230000000000002,"15min":1.774},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":72.203999999999994,"5min":72.203999999999994,"15min":72.203999999999994},"last":1.2070000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":2.0910000000000002,"15min":1.8460000000000001},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.29699999999999999},"max":{"1min":3.8940000000000001,"5min":54.161000000000001,"15min":54.161000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.7030000000000001,"15min":1.6619999999999999},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.1760000000000002,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2362441728,"data_stored":2362441728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3718316032,"data_stored":3718316032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2879721472,"data_stored":2879717400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3250388992,"data_stored":3250388992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1879379968,"data_stored":1879375896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1885077504,"data_stored":1885077504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3781758976,"data_stored":3781754904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2838953984,"data_stored":2838953984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:30:01.488 INFO:tasks.thrashosds.thrasher:Setting 4.18 to [6, 3, 0, 7, 1, 2] 2024-09-17T08:30:01.488 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.18', '6', '3', '0', '7', '1', '2'] 2024-09-17T08:30:01.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 4.18 6 3 0 7 1 2 2024-09-17T08:30:01.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:01.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.574 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:01.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:01.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:01.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:01.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:01.716 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:01 smithi173 ceph-mon[33187]: pgmap v901: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:30:01.717 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:01 smithi173 ceph-mon[33187]: from='client.25498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:30:01.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:30:01.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:01 smithi079 ceph-mon[29565]: pgmap v901: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:30:01.752 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:01 smithi079 ceph-mon[29565]: from='client.25498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:30:01.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:01 smithi079 ceph-mon[40649]: pgmap v901: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-17T08:30:01.752 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:01 smithi079 ceph-mon[40649]: from='client.25498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:30:01.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:01.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.835 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:01.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:01.936 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:01.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:01.980 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:01.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:02.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:02.115 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:02.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:02.118 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:02.217 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:02.243 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:02.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:02.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:02.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:02.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:02.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:02.516 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:02.602 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:02.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:02.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:02.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:02.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:02.808 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:02.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:02.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:02 smithi079 ceph-mon[29565]: pgmap v902: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:30:02.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:02 smithi079 ceph-mon[40649]: pgmap v902: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:30:02.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:02.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:02.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:02.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:02.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:02 smithi173 ceph-mon[33187]: pgmap v902: 158 pgs: 158 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s 2024-09-17T08:30:03.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:03.105 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:03.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:03.207 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:03.208 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:03.241 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:03.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:03.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:03.392 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:03.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:03.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:03.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.532 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:03.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:03.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:03.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:03.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:03.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:03.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:03.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:03.786 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:03.786 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:03.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:03.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:03.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:03.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:03.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:03.932 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:03.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:03.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:04.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:04.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:04.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:04.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:04.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:04.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:04.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:04.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.358 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:04.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:04.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:04.460 INFO:teuthology.orchestra.run.smithi173.stdout:ERROR: (22) Invalid argument 2024-09-17T08:30:04.461 INFO:teuthology.orchestra.run.smithi173.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:30:04.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:04.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:04.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:04.635 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:04.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:04.649 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:30:04.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:04.770 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:04.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:04.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:04.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:04.913 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:04.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:04 smithi173 ceph-mon[33187]: pgmap v903: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:04 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1405605322' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:04 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:05.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:05.044 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:05.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[40649]: pgmap v903: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1405605322' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[29565]: pgmap v903: 158 pgs: 158 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1405605322' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:04 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:30:05.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:05.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:30:05.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.180 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:05.180 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:05.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:05.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:05.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:05.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.449 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:05.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:05.478 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:05.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:05.587 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:05.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:05.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:05.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:05.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:05.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:05.854 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:05.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:05.935 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:05 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:05.935 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:05 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:30:05.935 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:05 smithi173 ceph-mon[33187]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T08:30:05.935 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:05 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:30:05.935 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:05 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:30:05.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:05.993 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:05.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:06.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:06.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:30:06.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[29565]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T08:30:06.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[40649]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:30:06.059 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:05 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:30:06.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:06.134 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:06.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:06.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:06.270 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:06.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:06.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:06.397 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:06.407 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:06.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:06.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:06.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:06.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:06.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:06.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[29565]: 11.5 scrub starts 2024-09-17T08:30:06.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[29565]: pgmap v905: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:30:06.667 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:30:06.668 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:30:06.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:06.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:06.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:06 smithi173 ceph-mon[33187]: 11.5 scrub starts 2024-09-17T08:30:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:06 smithi173 ceph-mon[33187]: pgmap v905: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:30:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:06 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:30:06.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:06 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:30:06.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:06.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:06.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:06.930 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:06.937 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[40649]: 11.5 scrub starts 2024-09-17T08:30:06.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[40649]: pgmap v905: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-17T08:30:06.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:30:06.938 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:06 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:30:06.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:06.963 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:06.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:30:07.004 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:30:07.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:30:07.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:30:07.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:30:07.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:30:07.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:30:07.010 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:30:07.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:30:07.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:07.104 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:07.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:07.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:07.243 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:07.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:07.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.378 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:07.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:07.405 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:30:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:30:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:30:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:30:07.405 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:30:07.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:07.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:07.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:07.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:07.657 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:07.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:07.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:07.798 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:07.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:07.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:07.929 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:07.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:07.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:07 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1754318392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:07 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:30:07.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:07 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:30:08.011 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1754318392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:08.012 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:30:08.012 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:30:08.012 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1754318392' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:08.012 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:30:08.012 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:07 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:30:08.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:08.067 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:08.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:08.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:30:08.114 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:30:08.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:08.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:08.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:08.297 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:08.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:08.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:08.401 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:30:08.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:08.485 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:08.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:08.619 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:08.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:08.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:08.761 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:08.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:08.841 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:08 smithi173 ceph-mon[33187]: 11.5 scrub ok 2024-09-17T08:30:08.841 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:08 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:30:08.841 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:08 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:30:08.841 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:08 smithi173 ceph-mon[33187]: pgmap v906: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:30:08.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:08.898 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:08.899 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:09.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:09.036 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:09.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[40649]: 11.5 scrub ok 2024-09-17T08:30:09.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:30:09.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[40649]: pgmap v906: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:30:09.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[29565]: 11.5 scrub ok 2024-09-17T08:30:09.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:30:09.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:30:09.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:08 smithi079 ceph-mon[29565]: pgmap v906: 158 pgs: 158 active+clean; 6.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-17T08:30:09.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:09.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:09.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:09.303 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:09.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:09.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:09.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:09.556 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:09.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:09.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:09.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:09.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:09.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.821 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:09.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:09.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:30:09.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:09.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:09.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:09.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:09 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:30:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:09 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:30:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:09 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:30:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:09 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:30:10.070 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:10.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:10.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:30:10.106 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:09 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:30:10.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:10.275 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:10.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:10.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:10.398 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:10.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:10.414 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:10.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:10.533 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:10.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:10.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:10.655 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:10.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:10.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:10.777 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:10.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:10.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:10.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:10.900 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:10.900 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:10 smithi173 ceph-mon[33187]: pgmap v907: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:10 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:30:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:10 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:30:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:10 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:30:10.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:10 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:30:11.001 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:11.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.027 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:11.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:11.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[40649]: pgmap v907: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:11.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:30:11.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:30:11.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:30:11.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:30:11.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[29565]: pgmap v907: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:11.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:30:11.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:30:11.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:30:11.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:10 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:30:11.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:11.161 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:11.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:11.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:11.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.298 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:11.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:11.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:11.435 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:11.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:11.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:11.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:11.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:11.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:11.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.706 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:11.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:11.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:11.838 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:11.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:11.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:11 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:30:11.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:11 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:30:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:11 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:30:11.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:11 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:30:11.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:11.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:11 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:30:11.954 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:11 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:30:11.969 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:11.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:11.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:12.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:12.104 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:12.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:12.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:12.242 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:30:12.242 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238260 9.c e279/221 9:3ad2fbde:::benchmark_data_smithi079_143589_object107061:head v 279'15028, mlcod=279'15025)", 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.27828591000002, 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.55165437399999995, 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961996+0000", 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.962003+0000", 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6880000000000002e-06 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030074+0000", 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.068071330999999999 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030089+0000", 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4934e-05 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030144+0000", 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4675000000000001e-05 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030167+0000", 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.304e-05 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513620+0000", 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48345298399999997 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513651+0000", 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1421000000000003e-05 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238262 9.c e279/221 9:3e3399a0:::benchmark_data_smithi079_143589_object107063:head v 279'15029, mlcod=279'15025)", 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.21016350100001, 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48366091900000002, 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030118+0000", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030126+0000", 2024-09-17T08:30:12.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7919999999999993e-06 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030136+0000", 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6800000000000005e-06 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030141+0000", 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9599999999999999e-06 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032835+0000", 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002693955 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032854+0000", 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9167000000000001e-05 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513749+0000", 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48089472900000002 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513780+0000", 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1254000000000003e-05 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238269 9.c e279/221 9:300a75d2:::benchmark_data_smithi079_143589_object107070:head v 279'15030, mlcod=279'15026)", 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.21013670600001, 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48365520499999998, 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030150+0000", 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.014e-06 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030155+0000", 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2700000000000004e-06 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030159+0000", 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9009999999999996e-06 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.035254+0000", 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005095558 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.035272+0000", 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7187999999999999e-05 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513783+0000", 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47851158300000002 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513801+0000", 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7895999999999999e-05 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.257 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238270 9.c e279/221 9:3be7327c:::benchmark_data_smithi079_143589_object107071:head v 279'15031, mlcod=279'15026)", 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.21011985600001, 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48365735300000001, 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:30:12.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030170+0000", 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.17e-06 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030175+0000", 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4439999999999999e-06 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030179+0000", 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.031e-06 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.042670+0000", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012490846 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.042689+0000", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9309000000000001e-05 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513803+0000", 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47111345199999999 2024-09-17T08:30:12.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513820+0000", 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7261e-05 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238263 9.9 e279/221 9:970a2f6b:::benchmark_data_smithi079_143589_object107064:head v 279'14960, mlcod=279'14957)", 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.20984347799998, 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48596704299999999, 2024-09-17T08:30:12.262 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030444+0000", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.062e-06 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030451+0000", 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3339999999999996e-06 2024-09-17T08:30:12.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030455+0000", 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0890000000000002e-06 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033010+0000", 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025551879999999999 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033030+0000", 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9850000000000001e-05 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516378+0000", 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48334795200000003 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516406+0000", 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8008e-05 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.266 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238266 9.9 e279/221 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head v 279'14961, mlcod=279'14957)", 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.20737444500003, 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48354262399999998, 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032918+0000", 2024-09-17T08:30:12.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0468999999999999e-05 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032924+0000", 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6760000000000004e-06 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032929+0000", 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9010000000000002e-06 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040159+0000", 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0072303089999999999 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040178+0000", 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.836e-05 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516433+0000", 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47625525400000002 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516451+0000", 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7932999999999999e-05 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238273 9.1 e279/221 9:8dab1ad1:::benchmark_data_smithi079_143589_object107074:head v 279'14726, mlcod=279'14724)", 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.20651747400001, 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.4826628, 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.271 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033764+0000", 2024-09-17T08:30:12.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:30:12.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033771+0000", 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3969999999999999e-06 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033781+0000", 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0417999999999999e-05 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033787+0000", 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6200000000000004e-06 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.037324+0000", 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035372799999999999 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.037343+0000", 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9313999999999998e-05 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516409+0000", 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47906535 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516428+0000", 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9205e-05 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.277 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238276 9.1 e279/221 9:8a6a764e:::benchmark_data_smithi079_143589_object107077:head v 279'14727, mlcod=279'14724)", 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.20407664700002, 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48026569699999999, 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.278 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036205+0000", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036212+0000", 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.55e-06 2024-09-17T08:30:12.279 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036222+0000", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8549999999999997e-06 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036227+0000", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7640000000000002e-06 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.039923+0000", 2024-09-17T08:30:12.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036956379999999998 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.039940+0000", 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7447000000000001e-05 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516453+0000", 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47651224599999997 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516472+0000", 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8930999999999999e-05 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238283 9.e 9:7b3bdc8a:::benchmark_data_smithi079_143589_object107084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.19644395799997, 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47686604199999999, 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.282 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238283 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043837+0000", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043843+0000", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9900000000000002e-06 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043853+0000", 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5910000000000002e-06 2024-09-17T08:30:12.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043859+0000", 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1680000000000001e-06 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043909+0000", 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0373e-05 2024-09-17T08:30:12.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044031+0000", 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121384 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513503+0000", 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46947200700000002 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516350+0000", 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028466569999999998 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.518549+0000", 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021995579999999999 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520469+0000", 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001919511 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520515+0000", 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6544e-05 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520705+0000", 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018956499999999999 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.288 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238285 9.d 9:b4139a65:::benchmark_data_smithi079_143589_object107086:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.19638779399997, 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48801911199999998, 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238285 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043894+0000", 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043898+0000", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043908+0000", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9709999999999999e-06 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043913+0000", 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4070000000000001e-06 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043951+0000", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7672000000000001e-05 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044051+0000", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100319 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513514+0000", 2024-09-17T08:30:12.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46946270299999998 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522917+0000", 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094026289999999992 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531719+0000", 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0088019359999999998 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531752+0000", 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3612000000000003e-05 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531869+0000", 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011707800000000001 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531914+0000", 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4614999999999999e-05 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.294 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238282 9.a e279/221 9:5c132a73:::benchmark_data_smithi079_143589_object107083:head v 279'14822, mlcod=279'14817)", 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.19627491400001, 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47837759200000002, 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.295 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044007+0000", 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044014+0000", 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6279999999999999e-06 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044022+0000", 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2519999999999999e-06 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044027+0000", 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5799999999999999e-06 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044064+0000", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6637000000000002e-05 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044084+0000", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.969e-05 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522318+0000", 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.478234569 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522385+0000", 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6926000000000003e-05 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238286 9.1 e279/221 9:88a162d9:::benchmark_data_smithi079_143589_object107087:head v 279'14728, mlcod=279'14724)", 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.19603614099998, 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47235775800000002, 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.299 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:30:12.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044252+0000", 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0109999999999997e-06 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513439+0000", 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46918737900000002 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513457+0000", 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8286000000000001e-05 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513649+0000", 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019104500000000001 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513678+0000", 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9445e-05 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.302 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516589+0000", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029111139999999998 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516604+0000", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5124e-05 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238289 9.7 9:e13aa397:::benchmark_data_smithi079_143589_object107090:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:30:12.303 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.19343400700001, 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48274150100000002, 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238289 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046848+0000", 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046853+0000", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5659999999999997e-06 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513663+0000", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.466810365 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513676+0000", 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2177e-05 2024-09-17T08:30:12.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514277+0000", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00060180900000000002 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514428+0000", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015038000000000001 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514561+0000", 2024-09-17T08:30:12.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013314300000000001 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517577+0000", 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030164469999999998 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520600+0000", 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030229660000000002 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529514+0000", 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0089139070000000004 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529552+0000", 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7676000000000003e-05 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529590+0000", 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8146999999999999e-05 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.309 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238291 9.a e279/221 9:5e6ad04a:::benchmark_data_smithi079_143589_object107092:head v 279'14823, mlcod=279'14820)", 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 413.187607087, 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46976095600000001, 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:30:12.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052675+0000", 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052683+0000", 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1140000000000006e-06 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513440+0000", 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.460757156 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513458+0000", 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7580000000000001e-05 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517250+0000", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037925939999999998 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517269+0000", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8785999999999999e-05 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522391+0000", 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0051220800000000002 2024-09-17T08:30:12.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522436+0000", 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5509000000000001e-05 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106709 11.5 11:aac0c29c:::benchmark_data_smithi079_199625_object106708:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 47.793432500000002, 2024-09-17T08:30:12.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.30129817399999997, 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106709 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446848+0000", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446923+0000", 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4624999999999998e-05 2024-09-17T08:30:12.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446938+0000", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5270000000000001e-05 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446946+0000", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9440000000000005e-06 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447254+0000", 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000307552 2024-09-17T08:30:12.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447414+0000", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000159918 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447576+0000", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016261800000000001 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.449833+0000", 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022572059999999999 2024-09-17T08:30:12.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.449866+0000", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2344000000000001e-05 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748051+0000", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.298184966 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748102+0000", 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1711e-05 2024-09-17T08:30:12.319 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748148+0000", 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5850000000000003e-05 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106724 11.5 11:aca01d77:::benchmark_data_smithi079_199625_object106723:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 47.777031092000001, 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28498498500000002, 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.320 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106724 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463250+0000", 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463304+0000", 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4355999999999999e-05 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463318+0000", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3997e-05 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463325+0000", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8929999999999999e-06 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463377+0000", 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1746000000000002e-05 2024-09-17T08:30:12.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463540+0000", 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016334100000000001 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463704+0000", 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000163974 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.466429+0000", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027248369999999999 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.466584+0000", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015447700000000001 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748185+0000", 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28160126699999999 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748216+0000", 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0923999999999997e-05 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748236+0000", 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0757000000000001e-05 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158194 11.f 11:f9a88cbf:::benchmark_data_smithi079_199625_object29316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.690730839, 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36185254900000002, 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158194 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549550+0000", 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:12.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549560+0000", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0039e-05 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549574+0000", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4100000000000001e-05 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549581+0000", 2024-09-17T08:30:12.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9720000000000003e-06 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549628+0000", 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6598999999999999e-05 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549787+0000", 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015902200000000001 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549918+0000", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013126199999999999 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552370+0000", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024517179999999999 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.561759+0000", 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093890620000000001 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911298+0000", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.34953903800000002 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911357+0000", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9194000000000003e-05 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911404+0000", 2024-09-17T08:30:12.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7009999999999999e-05 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158196 11.4 11:2df499b9:::benchmark_data_smithi079_199625_object29318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.688083682, 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36381823200000002, 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.333 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158196 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552198+0000", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552203+0000", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.31e-06 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552214+0000", 2024-09-17T08:30:12.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1260000000000001e-05 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552221+0000", 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7410000000000003e-06 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552271+0000", 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9874999999999999e-05 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552393+0000", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000122748 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552521+0000", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127143 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557360+0000", 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004838869 2024-09-17T08:30:12.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557497+0000", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137899 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.915929+0000", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.358431267 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.915977+0000", 2024-09-17T08:30:12.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8420000000000001e-05 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.916017+0000", 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0068000000000003e-05 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158197 11.4 11:2749ddc7:::benchmark_data_smithi079_199625_object29319:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:30:12.339 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.6858451699999999, 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36599574699999998, 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158197 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554436+0000", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554442+0000", 2024-09-17T08:30:12.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6949999999999997e-06 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554455+0000", 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3114e-05 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554462+0000", 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.764e-06 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.342 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554856+0000", 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000394239 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.555013+0000", 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015731600000000001 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.555124+0000", 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011102799999999999 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.343 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557638+0000", 2024-09-17T08:30:12.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025140509999999998 2024-09-17T08:30:12.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.344 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:12.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:12.345 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:12.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.558070+0000", 2024-09-17T08:30:12.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00043175 2024-09-17T08:30:12.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920349+0000", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.362278875 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920400+0000", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0889e-05 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920433+0000", 2024-09-17T08:30:12.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3420999999999997e-05 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158230 11.4 11:2ed1e3f1:::benchmark_data_smithi079_199625_object29352:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 1.6610281570000001, 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.341273365, 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:30:12.347 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158230 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:30:12.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579257+0000", 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.799e-06 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579268+0000", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0529e-05 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579273+0000", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1290000000000004e-06 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579316+0000", 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3248999999999999e-05 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579426+0000", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000109676 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579502+0000", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6363000000000001e-05 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.581538+0000", 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020359100000000002 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.581655+0000", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011646600000000001 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920463+0000", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.33880848200000002 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920493+0000", 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0063e-05 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:30:12.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920528+0000", 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4613999999999998e-05 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:30:12.353 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:30:12.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:12.480 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:12.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:12.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:12.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:12.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:12.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:12.748 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:12.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[29565]: pgmap v908: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[29565]: 11.1 scrub starts 2024-09-17T08:30:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:30:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:30:12.802 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[40649]: pgmap v908: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[40649]: 11.1 scrub starts 2024-09-17T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:12 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:30:12.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:12.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:12.895 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:12.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:12.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:12 smithi173 ceph-mon[33187]: pgmap v908: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-17T08:30:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:12 smithi173 ceph-mon[33187]: 11.1 scrub starts 2024-09-17T08:30:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:12 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:30:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:12 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:30:12.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:12 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:30:12.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:13.037 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:13.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:13.109 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:13.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:13.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:30:13.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:13.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:13.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:13.326 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:13.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:13.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:13.463 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:13.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:13.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:13.602 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:13.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:13.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:13.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:13.713 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:30:13.713 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:30:13.713 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:30:13.714 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:30:13.727 INFO:teuthology.orchestra.run.smithi079.stderr:set 4.18 pg_upmap_items mapping to [6->3,0->7,1->2] 2024-09-17T08:30:13.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:13.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:13.797 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3165389465' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:13.798 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3165389465' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.799 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:13.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:13.886 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:13.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:13.887 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:13.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:30:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3165389465' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]: dispatch 2024-09-17T08:30:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:13.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:14.022 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:14.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.023 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:14.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:14.150 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:14.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:14.252 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:14.283 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:14.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:14.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:14.423 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:14.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:14.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:14.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:14.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:14.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:14.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:14.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:14.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:14.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.828 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:14.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:14.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:14.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:14.964 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:14.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:15.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:15.091 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:15.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:30:15.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:30:15.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: 11.1 scrub ok 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: pgmap v909: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]': finished 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[29565]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:30:15.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: 11.1 scrub ok 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: pgmap v909: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]': finished 2024-09-17T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:14 smithi079 ceph-mon[40649]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T08:30:15.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:15.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: 11.1 scrub ok 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: pgmap v909: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.18", "id": [6, 3, 0, 7, 1, 2]}]': finished 2024-09-17T08:30:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:14 smithi173 ceph-mon[33187]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T08:30:15.235 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:15.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:15.374 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:15.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:15.515 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:15.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:15.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.655 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:15.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:15.789 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:15.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:15.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:15.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:16.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:16.069 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:16.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:16.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:30:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:15 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:30:16.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:16.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.211 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:16.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:30:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:15 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:30:16.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:16.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:16.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:16.439 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:16.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.476 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:16.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:16.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:16.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.603 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:16.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:16.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:16.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:16.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:16.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:16.871 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:16.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:16.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:16.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:17.005 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:17.107 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:17.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:30:17.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[29565]: pgmap v912: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:30:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:16 smithi079 ceph-mon[40649]: pgmap v912: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-17T08:30:17.135 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:17.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:17.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:16 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:30:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:16 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:30:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:16 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:30:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:16 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:30:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:16 smithi173 ceph-mon[33187]: pgmap v912: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-17T08:30:17.236 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:17.268 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:17.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:17.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:17.404 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:17.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:17.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:17.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:17.686 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:17.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:17.831 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:17.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:30:17.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[29565]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:30:17.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:30:17.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:30:17.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:30:17.874 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:17 smithi079 ceph-mon[40649]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T08:30:17.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:17.968 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:17.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:17.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:18.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:18.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:18.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:18.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:30:18.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:30:18.124 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:30:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:17 smithi173 ceph-mon[33187]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T08:30:18.225 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:18.255 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:18.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:18.357 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:18.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:18.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:18.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:18.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.534 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:18.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:18.635 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:18.668 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:18.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:18.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:18.807 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:18.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: pgmap v914: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: pgmap v914: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:30:18.812 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:18.820 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:30:18.820 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:30:18.821 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:18.861 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:30:18.862 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:30:18.902 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:30:18.903 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:30:18.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:18.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:18.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:18.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:18.935 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:30:18.935 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:30:18.971 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:30:18.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:30:19.009 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:30:19.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:30:19.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:19.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:30:19.046 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:30:19.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:19.083 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:30:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:30:19.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:30:19.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:30:19.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:30:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:30:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: pgmap v914: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-17T08:30:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:19.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:19.219 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:19.321 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:19.347 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:19.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:19.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:19.482 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:19.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:19.644 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:19.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:19.772 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:30:19.785 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:19.810 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:30:19.811 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:30:19.811 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:30:19.811 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.477948 2024-09-17T08:30:19.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 7 0.47794827405343654 2024-09-17T08:30:19.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:19.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:19.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:19.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:20.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:20.068 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:20.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:30:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:30:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1955910379' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:20.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3392210259' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:30:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:30:20.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1955910379' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:20.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3392210259' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:19 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.170 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:20.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.206 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:20.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:19 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:30:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:19 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:30:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1955910379' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:19 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3392210259' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:19 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:20.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:20.336 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:20.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:20.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:20.472 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:20.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:20.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:20.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:20.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:20.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:20.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.742 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:20.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:20.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:20.873 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:20.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:20.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:20.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:21.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:21.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:30:21.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: pgmap v915: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.47k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: 11.3 scrub starts 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[40649]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T08:30:21.039 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: pgmap v915: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.47k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: 11.3 scrub starts 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:21.040 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:20 smithi079 ceph-mon[29565]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T08:30:21.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:21.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: pgmap v915: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.47k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: 11.3 scrub starts 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:20 smithi173 ceph-mon[33187]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T08:30:21.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:21.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.287 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:21.388 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:21.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:21.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:21.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:21.557 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:21.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:21.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.709 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:21.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:21.852 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:21.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:21.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:21.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:22.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 5.1d deep-scrub starts 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 5.1d deep-scrub ok 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:30:22.116 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[40649]: 11.3 scrub ok 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 5.1d deep-scrub starts 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 5.1d deep-scrub ok 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T08:30:22.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:21 smithi079 ceph-mon[29565]: 11.3 scrub ok 2024-09-17T08:30:22.127 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:22.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 5.1d deep-scrub starts 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 5.1d deep-scrub ok 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T08:30:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:21 smithi173 ceph-mon[33187]: 11.3 scrub ok 2024-09-17T08:30:22.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:22.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.260 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:22.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:22.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:22.403 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:22.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:22.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:22.542 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:22.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.543 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:22.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:22.654 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:22.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.707 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:22.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:22.808 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:22.829 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:22 smithi079 ceph-mon[29565]: 11.a scrub starts 2024-09-17T08:30:22.829 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:22 smithi079 ceph-mon[29565]: pgmap v918: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:22.830 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:22 smithi079 ceph-mon[40649]: 11.a scrub starts 2024-09-17T08:30:22.830 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:22 smithi079 ceph-mon[40649]: pgmap v918: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:22.845 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:22.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:22.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:22.988 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:22.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:23.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:23.107 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:23.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:22 smithi173 ceph-mon[33187]: 11.a scrub starts 2024-09-17T08:30:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:22 smithi173 ceph-mon[33187]: pgmap v918: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:23.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:23.256 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:23.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:23.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:23.407 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:23.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:23.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:30:23.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:23.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:23.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:23.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:23.665 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:23.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:23.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:23.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:23.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:23.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:23.889 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:30:23.889 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:30:23.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:23.934 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:23.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:23.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:24.036 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:24.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:24.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: 11.b scrub starts 2024-09-17T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: 11.6 scrub starts 2024-09-17T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[29565]: 2.5 deep-scrub starts 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: 11.b scrub starts 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: 11.6 scrub starts 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:24.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:23 smithi079 ceph-mon[40649]: 2.5 deep-scrub starts 2024-09-17T08:30:24.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:30:24.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:24.191 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:24.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: 11.b scrub starts 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: 11.6 scrub starts 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:24.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:23 smithi173 ceph-mon[33187]: 2.5 deep-scrub starts 2024-09-17T08:30:24.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:24.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:24.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:24.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:24.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:24.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:24.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:24.600 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:24.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:24.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:24.729 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:24.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:24.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:30:24.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:24.883 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:24.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:24.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:24.985 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:25.011 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:25.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:25.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[29565]: 11.a scrub ok 2024-09-17T08:30:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[29565]: 11.9 scrub starts 2024-09-17T08:30:25.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[29565]: pgmap v919: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 838 B/s rd, 0 B/s wr, 1.43k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:25.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[29565]: 1.6 scrub starts 2024-09-17T08:30:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[40649]: 11.a scrub ok 2024-09-17T08:30:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[40649]: 11.9 scrub starts 2024-09-17T08:30:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[40649]: pgmap v919: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 838 B/s rd, 0 B/s wr, 1.43k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:25.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:24 smithi079 ceph-mon[40649]: 1.6 scrub starts 2024-09-17T08:30:25.139 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:25.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:25.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:24 smithi173 ceph-mon[33187]: 11.a scrub ok 2024-09-17T08:30:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:24 smithi173 ceph-mon[33187]: 11.9 scrub starts 2024-09-17T08:30:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:24 smithi173 ceph-mon[33187]: pgmap v919: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 5.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 838 B/s rd, 0 B/s wr, 1.43k op/s; 18 B/s, 0 objects/s recovering 2024-09-17T08:30:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:24 smithi173 ceph-mon[33187]: 1.6 scrub starts 2024-09-17T08:30:25.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:25.286 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:25.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:25.388 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:25.407 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:25.412 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:25.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:25.428 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:25.513 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:25.514 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:25.539 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:25.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:25.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:25.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.684 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:25.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:25.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:25.819 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:25.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 11.b scrub ok 2024-09-17T08:30:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 11.c scrub starts 2024-09-17T08:30:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 11.6 scrub ok 2024-09-17T08:30:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 5.10 deep-scrub starts 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[40649]: 5.10 deep-scrub ok 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 11.b scrub ok 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 11.c scrub starts 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 11.6 scrub ok 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 5.10 deep-scrub starts 2024-09-17T08:30:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:25 smithi079 ceph-mon[29565]: 5.10 deep-scrub ok 2024-09-17T08:30:25.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:25.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:25.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:26.060 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:26.081 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:26.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:26.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:26.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:26.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:26.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 11.b scrub ok 2024-09-17T08:30:26.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 11.c scrub starts 2024-09-17T08:30:26.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 11.6 scrub ok 2024-09-17T08:30:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:30:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:30:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 5.10 deep-scrub starts 2024-09-17T08:30:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:25 smithi173 ceph-mon[33187]: 5.10 deep-scrub ok 2024-09-17T08:30:26.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:26.343 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:26.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:26.445 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:26.475 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:26.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:26.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:26.619 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:26.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:26.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:26.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:26.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:26.887 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:26.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:26.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:26.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:27.010 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:27.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:27.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:30:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:30:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:30:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:30:27.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 11.9 scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 1.7 scrub starts 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: 1.7 scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[40649]: pgmap v920: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 11.9 scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 1.7 scrub starts 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: 1.7 scrub ok 2024-09-17T08:30:27.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:26 smithi079 ceph-mon[29565]: pgmap v920: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:30:27.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.145 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:27.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 11.9 scrub ok 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 1.7 scrub starts 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: 1.7 scrub ok 2024-09-17T08:30:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:26 smithi173 ceph-mon[33187]: pgmap v920: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-17T08:30:27.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:27.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.292 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:27.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:27.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:27.414 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:27.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:27.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:27.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:27.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:27.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:27.681 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:27.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:27.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:27.806 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:27.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:27.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:27.944 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:27.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:27.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:28.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:28.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:28.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 11.c scrub ok 2024-09-17T08:30:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 11.7 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 11.8 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 11.c scrub ok 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 11.7 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 11.8 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:30:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:30:28.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:27 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:30:28.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:28.204 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:28.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 11.c scrub ok 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 11.7 scrub starts 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 11.8 scrub starts 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:30:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:27 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:30:28.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:28.341 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:28.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:28.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:28.470 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:28.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:28.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:28.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:28.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:28.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.740 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:28.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:28.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:28.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:28.863 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:28.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:28.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:29.000 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:29.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:29.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:29.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:30:29.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:30:29.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: pgmap v921: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 768 op/s 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 11.7 scrub ok 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:30:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: pgmap v921: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 768 op/s 2024-09-17T08:30:29.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 11.7 scrub ok 2024-09-17T08:30:29.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:30:29.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:28 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:30:29.127 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:29.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:29.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: pgmap v921: 158 pgs: 4 active+clean+scrubbing, 154 active+clean; 5.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 768 op/s 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 11.7 scrub ok 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:30:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:28 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:30:29.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:29.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.265 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:29.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:29.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:29.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.387 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:29.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:29.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:29.520 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:29.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:29.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:29.656 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:29.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:29.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:29.796 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:29.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:29.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:29.936 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:29.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:29.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:30.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:30.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:30.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[40649]: 11.8 scrub ok 2024-09-17T08:30:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[40649]: 4.5 scrub starts 2024-09-17T08:30:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[40649]: 4.5 scrub ok 2024-09-17T08:30:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[29565]: 11.8 scrub ok 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[29565]: 4.5 scrub starts 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[29565]: 4.5 scrub ok 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:30:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:29 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:30:30.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:30.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:30.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:30.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:29 smithi173 ceph-mon[33187]: 11.8 scrub ok 2024-09-17T08:30:30.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:29 smithi173 ceph-mon[33187]: 4.5 scrub starts 2024-09-17T08:30:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:29 smithi173 ceph-mon[33187]: 4.5 scrub ok 2024-09-17T08:30:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:29 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:30:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:29 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:30:30.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:30.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:30.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:30.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:30.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:30.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:30.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:30.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:30.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:30.753 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:30.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:30.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:30.888 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:30.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:30.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:30.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:30 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:30:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:30 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:30:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:30 smithi173 ceph-mon[33187]: pgmap v922: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:30:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:30 smithi173 ceph-mon[33187]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T08:30:30.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:31.029 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:31.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:30:31.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:30:31.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[40649]: pgmap v922: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:30:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[40649]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[29565]: pgmap v922: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-17T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:30 smithi079 ceph-mon[29565]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T08:30:31.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:31.156 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:31.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:31.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:31.295 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:31.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:31.446 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:31.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:31.577 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:31.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:31.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.718 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:31.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:31.847 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:31.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:31.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:31.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:31.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:31.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:32.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:32.118 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:32.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:30:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:30:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:30:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:31 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:30:32.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:31 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:30:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:31 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:30:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:31 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:30:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:31 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:30:32.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.256 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:32.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:32.357 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:32.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:32.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:32.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:32.523 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:32.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:32.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:32.663 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:32.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:32.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:32.804 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:32.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:32.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:32.928 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:32.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:32.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:33.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:33.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:33.102 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: pgmap v924: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2328653847' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:30:33.103 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:30:33.104 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:30:33.104 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: pgmap v924: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:33.104 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2328653847' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.104 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:32 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:33.175 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:33.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:33.219 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.7 primary-affinity to 0.477948 (8313222) 2024-09-17T08:30:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:30:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:30:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:30:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:30:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: pgmap v924: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-17T08:30:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2328653847' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:32 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]: dispatch 2024-09-17T08:30:33.277 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:33.298 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.299 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:33.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:33.421 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:33.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:33.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:33.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:33.669 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:33.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:33.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:33.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:33.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:33.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:33.930 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:33.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:33.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:34.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:34.052 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:30:34.052 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:30:34.052 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]': finished 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[40649]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]': finished 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:33 smithi079 ceph-mon[29565]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T08:30:34.075 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:34.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:34.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:34.216 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:30:34.216 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:30:34.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.223 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:34.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:34.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 0.47794827405343654}]': finished 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:33 smithi173 ceph-mon[33187]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T08:30:34.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:34.364 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:34.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:34.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:34.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.504 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:34.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:34.605 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:34.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:34.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:34.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:34.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.764 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:34.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:34.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:34.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:34.902 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:34.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:35.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:35.042 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:35.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[29565]: pgmap v926: 158 pgs: 1 peering, 157 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s 2024-09-17T08:30:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:30:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:30:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:30:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[40649]: pgmap v926: 158 pgs: 1 peering, 157 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s 2024-09-17T08:30:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:30:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:30:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:34 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:30:35.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:35.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:35.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:35.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:34 smithi173 ceph-mon[33187]: pgmap v926: 158 pgs: 1 peering, 157 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s 2024-09-17T08:30:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:34 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:30:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:34 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:30:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:34 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:30:35.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:35.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.314 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:35.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:35.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:35.452 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:35.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:35.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:35.582 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:35.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:35.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:35.715 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:35.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:35.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:35.840 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:35.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:35.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:35.969 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:35.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:35.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:36.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:36.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.103 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:36.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:36.110 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.5 deep-scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 6.5 deep-scrub ok 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:30:36.111 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:30:36.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.5 deep-scrub starts 2024-09-17T08:30:36.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 6.5 deep-scrub ok 2024-09-17T08:30:36.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:30:36.112 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:35 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:30:36.204 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:30:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:30:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:30:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:30:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.5 deep-scrub starts 2024-09-17T08:30:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 6.5 deep-scrub ok 2024-09-17T08:30:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:30:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:35 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:30:36.233 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:36.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:36.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:36.366 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:36.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:36.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:36.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:36.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:36.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:36.643 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:36.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:36.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:36.792 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:36.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:36.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:36.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:36.931 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:36.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:37.032 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:37.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:37.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:36 smithi079 ceph-mon[40649]: pgmap v928: 158 pgs: 1 peering, 157 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 5.3 KiB/s rd, 36 KiB/s wr, 848 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-17T08:30:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:36 smithi079 ceph-mon[29565]: pgmap v928: 158 pgs: 1 peering, 157 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 5.3 KiB/s rd, 36 KiB/s wr, 848 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-17T08:30:37.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:37.202 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:37.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:37.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:36 smithi173 ceph-mon[33187]: pgmap v928: 158 pgs: 1 peering, 157 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 5.3 KiB/s rd, 36 KiB/s wr, 848 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-17T08:30:37.304 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:37.333 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:37.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:37.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:37.466 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:37.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:37.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:37.605 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:37.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:37.707 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:37.744 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:37.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:37.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:37.885 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:37.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:37.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:37.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:38.018 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 2.e scrub starts 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 2.e scrub ok 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 11.0 scrub starts 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:30:38.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[29565]: 11.e scrub starts 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 2.e scrub starts 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 2.e scrub ok 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 11.0 scrub starts 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:30:38.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:37 smithi079 ceph-mon[40649]: 11.e scrub starts 2024-09-17T08:30:38.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:30:38.120 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:38.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:38.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:38.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 2.e scrub starts 2024-09-17T08:30:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 2.e scrub ok 2024-09-17T08:30:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 11.0 scrub starts 2024-09-17T08:30:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:30:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:30:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:37 smithi173 ceph-mon[33187]: 11.e scrub starts 2024-09-17T08:30:38.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:38.279 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:38.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:38.381 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:38.412 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:38.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:38.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:38.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.553 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:38.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:38.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:38.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:38.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:38.770 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:30:38.771 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:30:38.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:38.810 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:30:38.811 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:30:38.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:30:38.846 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:30:38.847 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:38.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:38.886 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:30:38.886 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:30:38.917 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:30:38.917 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:30:38.949 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:38.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:30:38.951 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:30:38.976 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:38.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:38.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:38.992 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:30:38.992 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 1.8 scrub starts 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 1.8 scrub ok 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 1.4 scrub starts 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 1.4 scrub ok 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 2.16 scrub starts 2024-09-17T08:30:38.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 2.16 scrub ok 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: pgmap v929: 158 pgs: 1 peering, 157 active+clean; 4.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 4.1 KiB/s rd, 28 KiB/s wr, 729 op/s; 20 KiB/s, 0 objects/s recovering 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 11.0 scrub ok 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3528984081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 1.8 scrub starts 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 1.8 scrub ok 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 1.4 scrub starts 2024-09-17T08:30:38.994 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 1.4 scrub ok 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 2.16 scrub starts 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 2.16 scrub ok 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: pgmap v929: 158 pgs: 1 peering, 157 active+clean; 4.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 4.1 KiB/s rd, 28 KiB/s wr, 729 op/s; 20 KiB/s, 0 objects/s recovering 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 11.0 scrub ok 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:30:38.995 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:38 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3528984081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:39.028 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:30:39.028 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:30:39.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:39.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.113 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:39.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:39.214 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 1.8 scrub starts 2024-09-17T08:30:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 1.8 scrub ok 2024-09-17T08:30:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:30:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 1.4 scrub starts 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 1.4 scrub ok 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 2.16 scrub starts 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 2.16 scrub ok 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: pgmap v929: 158 pgs: 1 peering, 157 active+clean; 4.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 4.1 KiB/s rd, 28 KiB/s wr, 729 op/s; 20 KiB/s, 0 objects/s recovering 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 11.0 scrub ok 2024-09-17T08:30:39.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:30:39.226 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:30:39.226 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:38 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3528984081' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:39.244 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:39.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.245 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:39.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:39.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:39.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:39.472 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:39.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:39.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.516 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:39.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:39.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:39.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.654 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:39.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:39.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:39.787 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:39.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:39.887 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:30:39.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:39.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:39.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:39.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:40.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:40.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:40.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:40.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:40.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.196 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:40.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:40.199 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[40649]: 11.e scrub ok 2024-09-17T08:30:40.199 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1001040642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.200 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.200 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[29565]: 11.e scrub ok 2024-09-17T08:30:40.200 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1001040642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.200 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:39 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:39 smithi173 ceph-mon[33187]: 11.e scrub ok 2024-09-17T08:30:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:39 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1001040642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:39 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:30:40.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:40.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.332 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:40.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:40.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:40.461 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:40.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:40.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:40.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.602 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:40.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:40.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:40.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.740 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:40.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:40.773 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:30:40.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:30:40.773 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-17T08:30:40.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd out 3 2024-09-17T08:30:40.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:40.869 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:40.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:40.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:40.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:41.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.014 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:41.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:41.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:30:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:30:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.38k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:41.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[40649]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.38k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:40 smithi079 ceph-mon[29565]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T08:30:41.151 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:41.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.38k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:30:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:40 smithi173 ceph-mon[33187]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T08:30:41.252 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:41.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:41.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:41.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.415 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:41.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:41.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:41.645 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:41.670 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:41.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:41.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.799 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:41.900 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:41.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:30:41.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:41.927 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:41.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:41.974 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:42.028 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:42.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:42.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:42.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:42.203 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:42.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:42.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:42.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:42.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:42.488 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:42.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:42.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:42.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:42.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:42.696 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210605 9.0 e272/230 9:018ae7c4:::benchmark_data_smithi079_143589_object79406:head v 272'12909, mlcod=272'12906)", 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 466.75375927800002, 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37561313400000002, 2024-09-17T08:30:42.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939885+0000", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939884+0000", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939892+0000", 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.187e-06 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939902+0000", 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0917000000000001e-05 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939910+0000", 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3420000000000002e-06 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939949+0000", 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9501e-05 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.939970+0000", 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0380000000000001e-05 2024-09-17T08:30:42.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315468+0000", 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.37549839200000001 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315498+0000", 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0312999999999999e-05 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15327.0:210614 9.2 e272/230 9:448427a4:::benchmark_data_smithi079_143589_object79415:head v 272'13602, mlcod=272'13600)", 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 466.74748183999998, 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36429002500000002, 2024-09-17T08:30:42.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946163+0000", 2024-09-17T08:30:42.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946161+0000", 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946171+0000", 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303862+0000", 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35769168000000001 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303871+0000", 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8189999999999998e-06 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304030+0000", 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158679 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304053+0000", 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3189000000000001e-05 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310428+0000", 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063754440000000001 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310453+0000", 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4318e-05 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210615 9.6 9:6555ccd0:::benchmark_data_smithi079_143589_object79416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 466.746994336, 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36942082399999998, 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210615 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946650+0000", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946648+0000", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946656+0000", 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-17T08:30:42.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303849+0000", 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35719317699999997 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303864+0000", 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5011000000000001e-05 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.303913+0000", 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9240999999999998e-05 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304058+0000", 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014518800000000001 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.304185+0000", 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126952 2024-09-17T08:30:42.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312414+0000", 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008228806 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.312468+0000", 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3854000000000003e-05 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.315985+0000", 2024-09-17T08:30:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035164839999999998 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316026+0000", 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1301000000000003e-05 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.316071+0000", 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5207999999999998e-05 2024-09-17T08:30:42.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213094 9.6 9:6c0b11e2:::benchmark_data_smithi079_143589_object81895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 464.356279454, 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.31323531599999999, 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.716 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213094 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337363+0000", 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337371+0000", 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6359999999999994e-06 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337385+0000", 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4183e-05 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337392+0000", 2024-09-17T08:30:42.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4719999999999998e-06 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337442+0000", 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9378000000000001e-05 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337568+0000", 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012602300000000001 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337689+0000", 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012135599999999999 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341229+0000", 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003539979 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.345283+0000", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040540510000000004 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650515+0000", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30523122800000002 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650560+0000", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5611000000000001e-05 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650600+0000", 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0299000000000002e-05 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213100 9.6 9:65f8bbfc:::benchmark_data_smithi079_143589_object81901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 464.35242567699999, 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30948176700000002, 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:30:42.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213100 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341217+0000", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341225+0000", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.328e-06 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341239+0000", 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4051e-05 2024-09-17T08:30:42.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341247+0000", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3319999999999992e-06 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341301+0000", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3996000000000003e-05 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341442+0000", 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141328 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341562+0000", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011956 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.346195+0000", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046330629999999998 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.347443+0000", 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001248292 2024-09-17T08:30:42.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650634+0000", 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30319073099999999 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650668+0000", 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4360999999999998e-05 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650701+0000", 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2305999999999999e-05 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213109 9.6 9:690187cb:::benchmark_data_smithi079_143589_object81910:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 464.34282313300002, 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29998592099999999, 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213109 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350820+0000", 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350826+0000", 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5889999999999999e-06 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350840+0000", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3474e-05 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350848+0000", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9829999999999996e-06 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350896+0000", 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8618000000000001e-05 2024-09-17T08:30:42.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351004+0000", 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108012 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351110+0000", 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105951 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355185+0000", 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040743740000000004 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355199+0000", 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-05 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650732+0000", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29553320199999999 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650766+0000", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4260000000000001e-05 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650807+0000", 2024-09-17T08:30:42.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0850999999999999e-05 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:158194 11.f e390/359 11:f9a88cbf:::benchmark_data_smithi079_199625_object29316:head v 390'9938, mlcod=390'9934)", 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.143200747999998, 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36035172500000001, 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.736 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550443+0000", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550451+0000", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4999999999999999e-06 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550463+0000", 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1854e-05 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550470+0000", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5899999999999996e-06 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550523+0000", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3628999999999998e-05 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550542+0000", 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8199e-05 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910751+0000", 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36020933599999999 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910796+0000", 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589999999999998e-05 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158198 11.2 11:452bec04:::benchmark_data_smithi079_199625_object29320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.141163865999999, 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.38156971499999998, 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158198 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552479+0000", 2024-09-17T08:30:42.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552487+0000", 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8180000000000004e-06 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552500+0000", 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3468e-05 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552508+0000", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5490000000000001e-06 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552557+0000", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9351999999999998e-05 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552734+0000", 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177116 2024-09-17T08:30:42.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552848+0000", 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114298 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913036+0000", 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36018772500000001 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913068+0000", 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1971000000000003e-05 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.933979+0000", 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020910900999999999 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.934023+0000", 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3476000000000002e-05 2024-09-17T08:30:42.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.934050+0000", 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7875e-05 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:158196 11.4 e390/359 11:2df499b9:::benchmark_data_smithi079_199625_object29318:head v 390'10130, mlcod=390'10128)", 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.141033428, 2024-09-17T08:30:42.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36284417299999999, 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552610+0000", 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552620+0000", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4720000000000001e-06 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552628+0000", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6510000000000001e-06 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552633+0000", 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0549999999999999e-06 2024-09-17T08:30:42.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910783+0000", 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.358150621 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910809+0000", 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6072999999999999e-05 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.915431+0000", 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046219549999999996 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.915455+0000", 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4005000000000001e-05 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158201 11.d 11:baf37dc6:::benchmark_data_smithi079_199625_object29323:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:30:42.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.140440327, 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36441254099999998, 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158201 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553203+0000", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553209+0000", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.356e-06 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910726+0000", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35751760900000001 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910743+0000", 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6948000000000001e-05 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910798+0000", 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4549999999999998e-05 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910929+0000", 2024-09-17T08:30:42.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013176099999999999 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,6", 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.911046+0000", 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116297 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917507+0000", 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064610680000000004 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917536+0000", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8711e-05 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917542+0000", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.241e-06 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917581+0000", 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9060000000000002e-05 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917617+0000", 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5938e-05 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:158198 11.2 e390/359 ondisk, result = 0)", 2024-09-17T08:30:42.759 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:30:42.759 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.136192819999998, 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35561820999999999, 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:30:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557450+0000", 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557458+0000", 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3479999999999998e-06 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910819+0000", 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35336151999999998 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910827+0000", 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7500000000000003e-06 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913061+0000", 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022343430000000002 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913065+0000", 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7079999999999999e-06 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913070+0000", 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8459999999999999e-06 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:158198 11.2 e390/359 ondisk, result = 0)", 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.135923751, 2024-09-17T08:30:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.355317826, 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:30:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557720+0000", 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557726+0000", 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0460000000000002e-06 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910738+0000", 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35301196699999998 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910749+0000", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1367e-05 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913026+0000", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022763359999999999 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913032+0000", 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-17T08:30:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913039+0000", 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.545e-06 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:189643 11.2 11:48f748f6:::benchmark_data_smithi079_199625_object60765:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7504856420000001, 2024-09-17T08:30:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.038635083000000001, 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 189643 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:30:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943158+0000", 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943161+0000", 2024-09-17T08:30:42.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0850000000000001e-06 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943168+0000", 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7120000000000003e-06 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943173+0000", 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.977e-06 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957497+0000", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014323989000000001 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957626+0000", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012899299999999999 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957750+0000", 2024-09-17T08:30:42.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124084 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.967557+0000", 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098063520000000008 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.969091+0000", 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015346940000000001 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981558+0000", 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012466761999999999 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981670+0000", 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011168 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981794+0000", 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012425899999999999 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:189705 11.2 11:4839ca5f:::benchmark_data_smithi079_199625_object60827:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:39.005675+0000", 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6879690699999998, 2024-09-17T08:30:42.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031482597000000001, 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 189705 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005675+0000", 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005675+0000", 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005674+0000", 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005679+0000", 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5770000000000004e-06 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005688+0000", 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7660000000000004e-06 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.005703+0000", 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4975e-05 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.008169+0000", 2024-09-17T08:30:42.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024660139999999999 2024-09-17T08:30:42.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.008273+0000", 2024-09-17T08:30:42.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104349 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.008377+0000", 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103686 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.021910+0000", 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013533671000000001 2024-09-17T08:30:42.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.027679+0000", 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005768848 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.036951+0000", 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092713029999999998 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.037038+0000", 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7519e-05 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.037158+0000", 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119987 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:190490 11.2 11:4fb106e8:::benchmark_data_smithi079_199625_object61612:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:30:42.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:39.668640+0000", 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.025004553, 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031542962000000001, 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 190490 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668640+0000", 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668640+0000", 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668639+0000", 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668644+0000", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6609999999999999e-06 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668666+0000", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2050999999999999e-05 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.668674+0000", 2024-09-17T08:30:42.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1599999999999998e-06 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.672715+0000", 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040408010000000001 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.672829+0000", 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011393 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.672953+0000", 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012369700000000001 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.678785+0000", 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058324359999999999 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.683648+0000", 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048625669999999999 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.699521+0000", 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01587336 2024-09-17T08:30:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.700149+0000", 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062775299999999999 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.700183+0000", 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4344000000000002e-05 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.797 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:190741 11.2 11:4543be5d:::benchmark_data_smithi079_199625_object61863:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8412054630000001, 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052378642000000003, 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 190741 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852431+0000", 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999919 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852444+0000", 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2499e-05 2024-09-17T08:30:42.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852455+0000", 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0869999999999999e-05 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852462+0000", 2024-09-17T08:30:42.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1230000000000002e-06 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.862987+0000", 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010525223 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.863271+0000", 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028371400000000001 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.863568+0000", 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029748900000000001 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.866664+0000", 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030955219999999999 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.867430+0000", 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076666199999999999 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904714+0000", 2024-09-17T08:30:42.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.037283789999999997 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904766+0000", 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1990000000000002e-05 2024-09-17T08:30:42.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904818+0000", 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1706000000000003e-05 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:190797 11.2 11:441d878c:::benchmark_data_smithi079_199625_object61919:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:39.910845+0000", 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7827998389999999, 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.032446322999999999, 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 190797 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910845+0000", 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910845+0000", 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910843+0000", 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910849+0000", 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1649999999999997e-06 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910858+0000", 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6050000000000004e-06 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910865+0000", 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4980000000000003e-06 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.910917+0000", 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2478999999999997e-05 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.911040+0000", 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122994 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.911151+0000", 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110839 2024-09-17T08:30:42.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.916057+0000", 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049061060000000004 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.919894+0000", 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038365399999999998 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.943168+0000", 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023274903 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.943257+0000", 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8556999999999999e-05 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.943291+0000", 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3991e-05 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:193597 11.2 11:41c833c5:::benchmark_data_smithi079_199625_object64719:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:42.142447+0000", 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55119718200000001, 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035712861999999998, 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 193597 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.142447+0000", 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.142447+0000", 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.142447+0000", 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.143030+0000", 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00058325100000000002 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.143042+0000", 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1421e-05 2024-09-17T08:30:42.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.143048+0000", 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8580000000000002e-06 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.149060+0000", 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060119609999999997 2024-09-17T08:30:42.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.149332+0000", 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027190599999999998 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.149654+0000", 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032186399999999999 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154431+0000", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047769980000000002 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.158656+0000", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042246540000000004 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.178073+0000", 2024-09-17T08:30:42.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019417212 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.178119+0000", 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6384999999999997e-05 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.178160+0000", 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0760999999999998e-05 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:193608 11.2 11:45d14cfe:::benchmark_data_smithi079_199625_object64730:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:42.154558+0000", 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.53908665499999997, 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.030790871000000001, 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 193608 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154558+0000", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154558+0000", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154556+0000", 2024-09-17T08:30:42.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154563+0000", 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0079999999999996e-06 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154576+0000", 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2563e-05 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.154584+0000", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9619999999999993e-06 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.158766+0000", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041820199999999998 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.159105+0000", 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033889199999999998 2024-09-17T08:30:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.159766+0000", 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066112699999999996 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.163490+0000", 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003724757 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.178429+0000", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014938379 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.185168+0000", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067387130000000003 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.185284+0000", 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011665199999999999 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.185349+0000", 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4536999999999996e-05 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194151 11.d 11:b507d49f:::benchmark_data_smithi079_199625_object65273:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:42.595868+0000", 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.097776196999999995, 2024-09-17T08:30:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.030780467999999998, 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194151 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595868+0000", 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595868+0000", 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595867+0000", 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595873+0000", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6319999999999996e-06 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595889+0000", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5906e-05 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.595896+0000", 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8410000000000002e-06 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.599856+0000", 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039605899999999999 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.599957+0000", 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010070500000000001 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,6", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.600059+0000", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101568 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.607020+0000", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006961722 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.609461+0000", 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024406279999999998 2024-09-17T08:30:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.623894+0000", 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014433496000000001 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.626443+0000", 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002548916 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:30:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.626649+0000", 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020553699999999999 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:30:42.838 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:30:42.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.839 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:42.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:42.940 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:42.968 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:42.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:42.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:43.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:30:43.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:43.117 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:43.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:43.260 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:43.293 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:43.362 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:43.398 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:43.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:43.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:43.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:43.680 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.681 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:43.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:43.821 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:43 smithi079 ceph-mon[29565]: pgmap v932: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.07k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:43.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:43 smithi079 ceph-mon[40649]: pgmap v932: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.07k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:43.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:43.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:43.966 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:43.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:43 smithi173 ceph-mon[33187]: pgmap v932: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 11 KiB/s rd, 82 KiB/s wr, 1.07k op/s; 59 KiB/s, 0 objects/s recovering 2024-09-17T08:30:44.067 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:44.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:44.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:44.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:44.237 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:44.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:44.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:44.365 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:44.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-17T08:30:44.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:44.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:44.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:44.601 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:30:44.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:44.645 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:44.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:44.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:44.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[29565]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:44 smithi079 ceph-mon[40649]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T08:30:44.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:44.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:44.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:44.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:44 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:30:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:44 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:30:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:44 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:44 smithi173 ceph-mon[33187]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T08:30:45.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:45.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:45.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:45.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:45.234 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:45.247 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:45.265 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:45.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:45.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:45.410 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:45.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:45.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:45.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:45.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:45.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:45.681 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:45.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:45.689 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:45 smithi079 ceph-mon[40649]: pgmap v934: 158 pgs: 158 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.1 KiB/s rd, 55 KiB/s wr, 1.14k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:45.689 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:45 smithi079 ceph-mon[29565]: pgmap v934: 158 pgs: 158 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.1 KiB/s rd, 55 KiB/s wr, 1.14k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:45.783 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:45.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:45.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:45.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:45.942 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:45.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:45.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:45 smithi173 ceph-mon[33187]: pgmap v934: 158 pgs: 158 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.1 KiB/s rd, 55 KiB/s wr, 1.14k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:46.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:46.085 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:46.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:46.187 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:46.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:46.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:46.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:46.354 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:46.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.355 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:46.456 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:46.493 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:46.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:46.595 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:46.616 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.3. 2024-09-17T08:30:46.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:46.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:46.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:46.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.751 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:46.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:46.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["3"]}]: dispatch 2024-09-17T08:30:46.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:46 smithi079 ceph-mon[40649]: Client client.admin marked osd.3 out, while it was still marked up 2024-09-17T08:30:46.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:46 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:30:46.439+0000 7f02ec2c3640 -1 mon.a@0(leader).osd e400 definitely_dead 0 2024-09-17T08:30:46.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["3"]}]: dispatch 2024-09-17T08:30:46.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:46 smithi079 ceph-mon[29565]: Client client.admin marked osd.3 out, while it was still marked up 2024-09-17T08:30:46.873 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:46.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["3"]}]: dispatch 2024-09-17T08:30:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:46 smithi173 ceph-mon[33187]: Client client.admin marked osd.3 out, while it was still marked up 2024-09-17T08:30:46.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:46.997 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:46.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:46.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:47.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:47.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:47.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:47.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:47.244 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:47.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:47.370 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:47.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:47.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:47.493 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:47.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:47.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.624 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:47.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:47.746 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:47.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:47.864 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[29565]: pgmap v935: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.7 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:47.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["3"]}]': finished 2024-09-17T08:30:47.865 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[29565]: osdmap e401: 8 total, 8 up, 7 in 2024-09-17T08:30:47.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[40649]: pgmap v935: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.7 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:47.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["3"]}]': finished 2024-09-17T08:30:47.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:47 smithi079 ceph-mon[40649]: osdmap e401: 8 total, 8 up, 7 in 2024-09-17T08:30:47.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:47.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:47 smithi173 ceph-mon[33187]: pgmap v935: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 7.7 KiB/s rd, 55 KiB/s wr, 1.45k op/s; 39 KiB/s, 0 objects/s recovering 2024-09-17T08:30:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4030997598' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["3"]}]': finished 2024-09-17T08:30:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:47 smithi173 ceph-mon[33187]: osdmap e401: 8 total, 8 up, 7 in 2024-09-17T08:30:47.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:47.995 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:47.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:48.074 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:48.097 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:48.121 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:48.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:48.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:48.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:48.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:48.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:48.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.368 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:48.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:48.469 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:48.495 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:48.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:48.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:48.618 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:48.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:48.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:48.747 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:48.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[40649]: osdmap e402: 8 total, 8 up, 7 in 2024-09-17T08:30:48.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:48.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:48.823 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[40649]: osdmap e403: 8 total, 8 up, 7 in 2024-09-17T08:30:48.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[29565]: osdmap e402: 8 total, 8 up, 7 in 2024-09-17T08:30:48.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:48.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:48.824 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:48 smithi079 ceph-mon[29565]: osdmap e403: 8 total, 8 up, 7 in 2024-09-17T08:30:48.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:48.894 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:48.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:48.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:48.899 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:30:48.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:30:48.936 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:30:48.936 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:30:48.971 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:30:48.971 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:30:48.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:48 smithi173 ceph-mon[33187]: osdmap e402: 8 total, 8 up, 7 in 2024-09-17T08:30:48.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:48.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:30:48.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:48 smithi173 ceph-mon[33187]: osdmap e403: 8 total, 8 up, 7 in 2024-09-17T08:30:48.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:49.009 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:30:49.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:30:49.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.045 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:49.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:49.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:30:49.045 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:30:49.077 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:30:49.078 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:30:49.115 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:30:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:30:49.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:49.156 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:30:49.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:30:49.190 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:49.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.191 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:49.292 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:49.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.324 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:49.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:49.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:49.458 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:49.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:49.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:49.587 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:49.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:49.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:49.736 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:49.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:49.832 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: pgmap v938: 158 pgs: 1 unknown, 12 peering, 145 active+clean; 3.8 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 400'12351 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 400'12413 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.2 starting backfill to osd.7 from (0'0,0'0] MAX to 400'12570 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 400'12334 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.a starting backfill to osd.1 from (0'0,0'0] MAX to 400'12441 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.b starting backfill to osd.4 from (0'0,0'0] MAX to 400'12454 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 400'12294 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4107723053' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[40649]: osdmap e404: 8 total, 8 up, 7 in 2024-09-17T08:30:49.833 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: pgmap v938: 158 pgs: 1 unknown, 12 peering, 145 active+clean; 3.8 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 400'12351 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 400'12413 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.2 starting backfill to osd.7 from (0'0,0'0] MAX to 400'12570 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 400'12334 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.a starting backfill to osd.1 from (0'0,0'0] MAX to 400'12441 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.b starting backfill to osd.4 from (0'0,0'0] MAX to 400'12454 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 400'12294 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4107723053' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:49.834 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:49 smithi079 ceph-mon[29565]: osdmap e404: 8 total, 8 up, 7 in 2024-09-17T08:30:49.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:49.865 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:49.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:49.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:49.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:49.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: pgmap v938: 158 pgs: 1 unknown, 12 peering, 145 active+clean; 3.8 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 400'12351 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.9 starting backfill to osd.1 from (0'0,0'0] MAX to 400'12413 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.2 starting backfill to osd.7 from (0'0,0'0] MAX to 400'12570 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.8 starting backfill to osd.0 from (0'0,0'0] MAX to 400'12334 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.a starting backfill to osd.1 from (0'0,0'0] MAX to 400'12441 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.b starting backfill to osd.4 from (0'0,0'0] MAX to 400'12454 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: 11.1 starting backfill to osd.2 from (0'0,0'0] MAX to 400'12294 2024-09-17T08:30:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4107723053' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:30:49.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:49 smithi173 ceph-mon[33187]: osdmap e404: 8 total, 8 up, 7 in 2024-09-17T08:30:49.999 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:49.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:50.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:50.141 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:50.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:50.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:50.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.283 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:50.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:50.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:50.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.417 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:50.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:50.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:50.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:50.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:50.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:50.676 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:50.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:50.823 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:50.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:50.925 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:50.952 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:50.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:50.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:51.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:51.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:51.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:51.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:51.219 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:51.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:51.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:51.357 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:51.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:51.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:51.497 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:51.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:51.599 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:51.632 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:51.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:51.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:51.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.772 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:51.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:51.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:51.910 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[29565]: pgmap v941: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s; 7104/176106 objects misplaced (4.034%); 576 KiB/s, 26 objects/s recovering 2024-09-17T08:30:51.911 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[29565]: osdmap e405: 8 total, 8 up, 7 in 2024-09-17T08:30:51.911 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[29565]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:51.911 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[40649]: pgmap v941: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s; 7104/176106 objects misplaced (4.034%); 576 KiB/s, 26 objects/s recovering 2024-09-17T08:30:51.911 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[40649]: osdmap e405: 8 total, 8 up, 7 in 2024-09-17T08:30:51.911 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:51 smithi079 ceph-mon[40649]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:51.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:51.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:51.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:51 smithi173 ceph-mon[33187]: pgmap v941: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s; 7104/176106 objects misplaced (4.034%); 576 KiB/s, 26 objects/s recovering 2024-09-17T08:30:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:51 smithi173 ceph-mon[33187]: osdmap e405: 8 total, 8 up, 7 in 2024-09-17T08:30:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:51 smithi173 ceph-mon[33187]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:52.020 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:52.057 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:52.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:52.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:52.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.199 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:52.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:52.321 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:52.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:52.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:52.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:52.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.605 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:52.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:52.748 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:52.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:52.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:52.893 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:52.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:52.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:53.025 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:53.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:53.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:53.159 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:53.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:53.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:53.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:53.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:30:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:30:53.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:53.433 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:53.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.434 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:53.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:53.567 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 1] out_osds: [3] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:30:53.567 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:30:53.567 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T08:30:53.568 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.051026 2024-09-17T08:30:53.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 2 0.05102553357962425 2024-09-17T08:30:53.586 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:53.671 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:30:53.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:53.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.756 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:53.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:53.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[29565]: pgmap v943: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 848 op/s; 7104/176106 objects misplaced (4.034%); 567 KiB/s, 25 objects/s recovering 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[40649]: pgmap v943: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 848 op/s; 7104/176106 objects misplaced (4.034%); 567 KiB/s, 25 objects/s recovering 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:53.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:53.883 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:53.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:53.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:53.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:30:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:30:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:53 smithi173 ceph-mon[33187]: pgmap v943: 158 pgs: 1 active+remapped+backfilling, 6 remapped+peering, 1 unknown, 1 active+remapped, 12 peering, 1 active+remapped+backfill_toofull, 136 active+clean; 3.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 0 B/s wr, 848 op/s; 7104/176106 objects misplaced (4.034%); 567 KiB/s, 25 objects/s recovering 2024-09-17T08:30:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:53 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:30:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:30:53.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:54.014 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:54.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.015 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:54.116 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:54.144 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:54.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:54.152 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:30:54.153 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:30:54.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:54.282 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:54.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:54.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:54.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:54.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:54.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:54.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.555 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:54.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:54.656 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:54.691 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:54.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:54.793 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:54.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.827 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:54.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:54.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:54.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:54.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: osdmap e406: 8 total, 8 up, 7 in 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: pgmap v945: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 404 B/s rd, 0 B/s wr, 750 op/s; 23074/168060 objects misplaced (13.730%); 17 MiB/s, 282 objects/s recovering 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:55.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:55.086 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:55.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:55.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:55.106 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:55.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:55.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: osdmap e406: 8 total, 8 up, 7 in 2024-09-17T08:30:55.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: pgmap v945: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 404 B/s rd, 0 B/s wr, 750 op/s; 23074/168060 objects misplaced (13.730%); 17 MiB/s, 282 objects/s recovering 2024-09-17T08:30:55.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:55.107 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:55.108 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:55.108 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:30:55.108 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:30:55.108 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/352423629' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:30:55.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: osdmap e406: 8 total, 8 up, 7 in 2024-09-17T08:30:55.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: pgmap v945: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 404 B/s rd, 0 B/s wr, 750 op/s; 23074/168060 objects misplaced (13.730%); 17 MiB/s, 282 objects/s recovering 2024-09-17T08:30:55.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:30:55.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:30:55.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:30:55.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:55.213 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:55.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:55.315 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:55.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.340 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:55.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:55.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:55.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.487 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:55.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:55.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:55.611 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:30:55.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.625 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:55.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:55.726 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:55.744 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:55 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 2 pgs peering) 2024-09-17T08:30:55.744 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:55 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 2 pgs peering) 2024-09-17T08:30:55.760 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:55.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:55.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:55.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:55.909 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:55.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:55 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 2 pgs peering) 2024-09-17T08:30:56.010 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:56.037 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:56.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:56.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-17T08:30:56.139 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:56.169 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:56.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:56.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:56.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:56.420 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:56.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.445 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:56.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:56.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:56.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.594 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:56.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:56.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:56.732 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:56.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:56.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:56.879 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:56.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:56.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:56.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:56 smithi173 ceph-mon[33187]: pgmap v946: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 845 B/s rd, 0 B/s wr, 702 op/s; 21944/163356 objects misplaced (13.433%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:56 smithi173 ceph-mon[33187]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:56.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:30:57.028 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:57.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:30:57.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:56 smithi079 ceph-mon[29565]: pgmap v946: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 845 B/s rd, 0 B/s wr, 702 op/s; 21944/163356 objects misplaced (13.433%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:57.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:56 smithi079 ceph-mon[29565]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:57.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:56 smithi079 ceph-mon[40649]: pgmap v946: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 845 B/s rd, 0 B/s wr, 702 op/s; 21944/163356 objects misplaced (13.433%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:57.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:56 smithi079 ceph-mon[40649]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:30:57.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:57.164 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:57.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:57.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:30:57.294 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:57.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:30:57.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:57.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.424 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:57.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:57.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:30:57.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:57.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:57.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:57.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:30:57.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:57.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:57.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:57.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:30:57.857 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:57.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:30:57.959 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:57.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:57.992 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:57.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:58.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:58.128 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:58.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:58.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:58.261 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:58.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:58.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:58.394 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:58.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:58.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:58.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.528 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:58.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:58.629 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:58.662 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:58.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:58.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:30:58.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:58.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.789 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:30:58.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:58.930 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:58.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:58.931 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:58.993 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:30:58 smithi079 ceph-mon[40649]: pgmap v947: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 613 op/s; 21056/161385 objects misplaced (13.047%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:58.993 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:30:58 smithi079 ceph-mon[29565]: pgmap v947: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 613 op/s; 21056/161385 objects misplaced (13.047%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:59.032 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:59.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.067 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:59.168 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:59.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.198 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:59.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:30:58 smithi173 ceph-mon[33187]: pgmap v947: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.3 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 613 op/s; 21056/161385 objects misplaced (13.047%); 20 MiB/s, 318 objects/s recovering 2024-09-17T08:30:59.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:30:59.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:30:59.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:59.474 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:59.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:30:59.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.609 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:30:59.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:30:59.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.745 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:30:59.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:30:59.846 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:30:59.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:30:59.880 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:30:59.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:30:59.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:00.012 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:00.114 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:00.146 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:00.248 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:00.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.281 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:00.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:00.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:00.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:00.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:00.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:00.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:00.679 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:00.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:00.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.804 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:00.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:00.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:00.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:00.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:00.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:01.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:01.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.081 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:01.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:00 smithi079 ceph-mon[40649]: pgmap v948: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s; 17175/148554 objects misplaced (11.561%); 33 MiB/s, 533 objects/s recovering 2024-09-17T08:31:01.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:00 smithi079 ceph-mon[29565]: pgmap v948: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s; 17175/148554 objects misplaced (11.561%); 33 MiB/s, 533 objects/s recovering 2024-09-17T08:31:01.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:01.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:01.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:00 smithi173 ceph-mon[33187]: pgmap v948: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.01k op/s; 17175/148554 objects misplaced (11.561%); 33 MiB/s, 533 objects/s recovering 2024-09-17T08:31:01.299 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:01.312 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:01.341 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:01.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:01.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:01.466 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:01.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:01.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:01.602 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:01.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:01.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:01.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:01.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:01.849 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:01.884 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:01.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:01.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:01.986 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:02.017 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:02.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:02.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:02.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.156 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:02.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:02.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:02.290 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:02.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:02.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:02.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.426 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:02.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:02.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:02.567 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:02.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:02.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:02.696 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:02.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:02.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:02.837 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:02.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:02.939 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:02.960 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:02.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:02.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:03.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:03.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.095 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:03.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:03.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:03.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:03.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:03.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:03.467 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:03.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.505 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:03.606 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:03.637 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:03.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:03.774 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:03.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[40649]: pgmap v949: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 919 op/s; 17175/148554 objects misplaced (11.561%); 30 MiB/s, 483 objects/s recovering 2024-09-17T08:31:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[29565]: pgmap v949: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 919 op/s; 17175/148554 objects misplaced (11.561%); 30 MiB/s, 483 objects/s recovering 2024-09-17T08:31:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:03.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:03.914 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:03.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:03.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:03.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:03 smithi173 ceph-mon[33187]: pgmap v949: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 919 op/s; 17175/148554 objects misplaced (11.561%); 30 MiB/s, 483 objects/s recovering 2024-09-17T08:31:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:03.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:04.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:04.054 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:04.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:04.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:04.187 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:04.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.188 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:04.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:04.329 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:04.431 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:04.463 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:04.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:04.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:04.607 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:04.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:04.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:04.742 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:04.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:04.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:04 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:04.880 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:04.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:04.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:04 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:04 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:04.982 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:05.012 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:05.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:05.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:05.150 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:05.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:05.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:05.286 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:05.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:05.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:05.416 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:05.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:05.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:05.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:05.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:05.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:05.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.680 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:05.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:05.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:05.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:05.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[40649]: pgmap v950: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.8 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1021 B/s rd, 0 B/s wr, 1.02k op/s; 13914/137316 objects misplaced (10.133%); 34 MiB/s, 548 objects/s recovering 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[40649]: osdmap e407: 8 total, 8 up, 7 in 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[29565]: pgmap v950: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.8 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1021 B/s rd, 0 B/s wr, 1.02k op/s; 13914/137316 objects misplaced (10.133%); 34 MiB/s, 548 objects/s recovering 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:05 smithi079 ceph-mon[29565]: osdmap e407: 8 total, 8 up, 7 in 2024-09-17T08:31:05.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:05.929 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:05.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:05.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:05 smithi173 ceph-mon[33187]: pgmap v950: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.8 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1021 B/s rd, 0 B/s wr, 1.02k op/s; 13914/137316 objects misplaced (10.133%); 34 MiB/s, 548 objects/s recovering 2024-09-17T08:31:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:05 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:05 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:05 smithi173 ceph-mon[33187]: osdmap e407: 8 total, 8 up, 7 in 2024-09-17T08:31:06.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:06.057 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:06.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:06.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:06.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:06.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:06.308 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:06.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:06.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:06.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:06.538 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:06.568 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:06.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:06.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:06.704 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:06.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:06.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:06.842 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:06.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:06.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:06.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:06.990 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:06.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:07.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:07.122 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:07.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:07.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:07.246 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:07.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:07.348 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:07.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.380 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:07.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:07.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:07.517 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:07.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:07.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:07.651 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:07.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:07.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:07.787 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:07.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:07 smithi079 ceph-mon[29565]: pgmap v952: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 12876/132498 objects misplaced (9.718%); 35 MiB/s, 552 objects/s recovering 2024-09-17T08:31:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:07 smithi079 ceph-mon[40649]: pgmap v952: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 12876/132498 objects misplaced (9.718%); 35 MiB/s, 552 objects/s recovering 2024-09-17T08:31:07.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:07.921 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:07.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:07.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:07 smithi173 ceph-mon[33187]: pgmap v952: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 12876/132498 objects misplaced (9.718%); 35 MiB/s, 552 objects/s recovering 2024-09-17T08:31:08.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:08.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.056 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:08.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:08.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:08.211 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:08.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:08.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:08.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.359 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:08.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:08.459 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:08.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:08.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:08.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:08.621 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 0.0510255 (833442) 2024-09-17T08:31:08.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.632 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:08.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:08.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:08.764 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:08.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:08.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2892336267' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2892336267' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:08 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.889 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:08.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:08.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:08.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:08 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:08 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2892336267' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:08 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]: dispatch 2024-09-17T08:31:08.988 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:08.991 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:09.013 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:09.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:09.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:09.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.140 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:09.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:09.241 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:09.263 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:09.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:09.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:09.386 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:09.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:09.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:09.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.510 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:09.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:09.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:09.648 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:09.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:09.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:09.770 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:09.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[29565]: pgmap v953: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 11943/130479 objects misplaced (9.153%); 36 MiB/s, 570 objects/s recovering 2024-09-17T08:31:09.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]': finished 2024-09-17T08:31:09.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[29565]: osdmap e408: 8 total, 8 up, 7 in 2024-09-17T08:31:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[40649]: pgmap v953: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 11943/130479 objects misplaced (9.153%); 36 MiB/s, 570 objects/s recovering 2024-09-17T08:31:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]': finished 2024-09-17T08:31:09.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:09 smithi079 ceph-mon[40649]: osdmap e408: 8 total, 8 up, 7 in 2024-09-17T08:31:09.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:09.893 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:09.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:09.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:09.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:09 smithi173 ceph-mon[33187]: pgmap v953: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s; 11943/130479 objects misplaced (9.153%); 36 MiB/s, 570 objects/s recovering 2024-09-17T08:31:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:09 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.05102553357962425}]': finished 2024-09-17T08:31:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:09 smithi173 ceph-mon[33187]: osdmap e408: 8 total, 8 up, 7 in 2024-09-17T08:31:09.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:10.017 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:10.119 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:10.140 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:10.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:10.264 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:10.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:10.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:10.388 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:10.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:10.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.512 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:31:10.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:10.638 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:10.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:10.773 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:10.793 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:10 smithi079 ceph-mon[29565]: osdmap e409: 8 total, 8 up, 7 in 2024-09-17T08:31:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:10 smithi079 ceph-mon[29565]: osdmap e410: 8 total, 8 up, 7 in 2024-09-17T08:31:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:10 smithi079 ceph-mon[40649]: osdmap e409: 8 total, 8 up, 7 in 2024-09-17T08:31:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:10 smithi079 ceph-mon[40649]: osdmap e410: 8 total, 8 up, 7 in 2024-09-17T08:31:10.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:10.915 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:10.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:10.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:10 smithi173 ceph-mon[33187]: osdmap e409: 8 total, 8 up, 7 in 2024-09-17T08:31:10.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:10 smithi173 ceph-mon[33187]: osdmap e410: 8 total, 8 up, 7 in 2024-09-17T08:31:11.017 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:11.047 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:11.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:11.149 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:11.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.184 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:11.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:11.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:11.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-17T08:31:11.324 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:11.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:11.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:11.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.480 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:11.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:11.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:11.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:11.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.613 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:11.713 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:11.744 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:11.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:11.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:11.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:11 smithi079 ceph-mon[29565]: pgmap v956: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 8389/117600 objects misplaced (7.134%); 39 MiB/s, 623 objects/s recovering 2024-09-17T08:31:11.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:11 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:31:11.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:11 smithi079 ceph-mon[40649]: pgmap v956: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 8389/117600 objects misplaced (7.134%); 39 MiB/s, 623 objects/s recovering 2024-09-17T08:31:11.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:11 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:31:11.885 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:11.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:11.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:11.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:11 smithi173 ceph-mon[33187]: pgmap v956: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 8389/117600 objects misplaced (7.134%); 39 MiB/s, 623 objects/s recovering 2024-09-17T08:31:11.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:11 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-17T08:31:11.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:12.026 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:12.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:12.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.170 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:12.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:12.271 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:12.301 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:12.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:12.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:12.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:12.548 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:12.581 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:12.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.582 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:12.683 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:12.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.719 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:12.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:12.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:12.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:12.860 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:12.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:12.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:13.000 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:13.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:13.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:13.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.133 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:13.234 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:13.261 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:13.321 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:13.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:13.395 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:13.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:13.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:13.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:13.637 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:13.665 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:31:13.667 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:13.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:13.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:13.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:13.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:13.942 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:13.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:13.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: pgmap v958: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 828 op/s; 8389/117600 objects misplaced (7.134%); 33 MiB/s, 525 objects/s recovering 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: osdmap e411: 8 total, 8 up, 7 in 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/815647362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:13.981 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:14.043 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: pgmap v958: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 828 op/s; 8389/117600 objects misplaced (7.134%); 33 MiB/s, 525 objects/s recovering 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: osdmap e411: 8 total, 8 up, 7 in 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/815647362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:14.054 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: pgmap v958: 158 pgs: 1 active+clean+remapped, 3 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 148 active+clean; 2.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 682 B/s rd, 0 B/s wr, 828 op/s; 8389/117600 objects misplaced (7.134%); 33 MiB/s, 525 objects/s recovering 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: osdmap e411: 8 total, 8 up, 7 in 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/815647362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:14.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:31:14.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:14.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:14.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:14.216 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:14.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:14.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:14.355 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:14.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:14.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:14.494 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:14.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:14.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:14.630 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:14.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:14.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:14.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.780 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:14.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:14.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:14.901 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:14.901 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:14.901 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:31:14.901 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[29565]: osdmap e412: 8 total, 8 up, 7 in 2024-09-17T08:31:14.901 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[29565]: pgmap v961: 158 pgs: 1 active+clean+remapped, 3 peering, 4 active+remapped+backfill_toofull, 150 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 0 B/s wr, 711 op/s; 6974/108906 objects misplaced (6.404%); 14 MiB/s, 221 objects/s recovering 2024-09-17T08:31:14.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:14.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:14.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:31:14.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[40649]: osdmap e412: 8 total, 8 up, 7 in 2024-09-17T08:31:14.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:14 smithi079 ceph-mon[40649]: pgmap v961: 158 pgs: 1 active+clean+remapped, 3 peering, 4 active+remapped+backfill_toofull, 150 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 0 B/s wr, 711 op/s; 6974/108906 objects misplaced (6.404%); 14 MiB/s, 221 objects/s recovering 2024-09-17T08:31:14.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:14.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:14.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:14.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:14 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:14 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:31:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:14 smithi173 ceph-mon[33187]: osdmap e412: 8 total, 8 up, 7 in 2024-09-17T08:31:14.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:14 smithi173 ceph-mon[33187]: pgmap v961: 158 pgs: 1 active+clean+remapped, 3 peering, 4 active+remapped+backfill_toofull, 150 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 0 B/s wr, 711 op/s; 6974/108906 objects misplaced (6.404%); 14 MiB/s, 221 objects/s recovering 2024-09-17T08:31:15.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:15.055 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:15.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:15.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:15.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.190 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:15.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:15.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:15.324 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:15.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.325 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:15.404 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 1] out_osds: [3] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:31:15.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:31:15.404 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T08:31:15.404 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T08:31:15.405 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T08:31:15.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-17T08:31:15.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:15.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.492 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:15.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:15.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:15.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.636 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:31:15.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:31:15.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:31:15.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:31:15.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/176993701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:15.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:31:15.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:15 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:31:15.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:15.738 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:31:15.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.774 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:15.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:15.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:15.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:31:15.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:31:15.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:31:15.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:31:15.880 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:31:15.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:31:15.918 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:15.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:15.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:16.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:16.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.059 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:16.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:16.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/176993701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:16.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/176993701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:31:16.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:15 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:31:16.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:16.199 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:16.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:16.301 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:16.330 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:16.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:16.432 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:16.464 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:16.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:16.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:16.606 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:16.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:16.707 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:16.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: pgmap v962: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 341 B/s rd, 0 B/s wr, 743 op/s; 6688/104220 objects misplaced (6.417%); 9.5 MiB/s, 150 objects/s recovering 2024-09-17T08:31:16.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:31:16.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:31:16.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:31:16.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:31:16.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:31:16.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:16 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:31:16.754 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:16.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:16.856 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:16.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: pgmap v962: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 341 B/s rd, 0 B/s wr, 743 op/s; 6688/104220 objects misplaced (6.417%); 9.5 MiB/s, 150 objects/s recovering 2024-09-17T08:31:16.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:31:16.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: pgmap v962: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 341 B/s rd, 0 B/s wr, 743 op/s; 6688/104220 objects misplaced (6.417%); 9.5 MiB/s, 150 objects/s recovering 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:31:16.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:31:16.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:16 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:31:16.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:16.885 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:16.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:16.986 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:17.020 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:17.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:17.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:17.154 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:17.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:17.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:17.287 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:17.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:17.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:17.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.426 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:17.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:17.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:17.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.578 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:17.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238263 9.9 9:970a2f6b:::benchmark_data_smithi079_143589_object107064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:31:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 478.705694333, 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55651207300000005, 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238263 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961768+0000", 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961766+0000", 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961772+0000", 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9970000000000004e-06 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961796+0000", 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3638e-05 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961803+0000", 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9949999999999999e-06 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961847+0000", 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3470999999999998e-05 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961955+0000", 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108051 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962124+0000", 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016913200000000001 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.968042+0000", 2024-09-17T08:31:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059184399999999996 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033570+0000", 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065527335000000006 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518190+0000", 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48461996699999998 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518239+0000", 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9529e-05 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518280+0000", 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0831000000000003e-05 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238266 9.9 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 478.70496668499999, 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55589687899999995, 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238266 2024-09-17T08:31:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962495+0000", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962499+0000", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4819999999999996e-06 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962532+0000", 2024-09-17T08:31:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2979999999999999e-05 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962548+0000", 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6359e-05 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965118+0000", 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025690779999999998 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965221+0000", 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103297 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.965327+0000", 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106302 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.968153+0000", 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028257759999999999 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.040235+0000", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072081920999999993 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518320+0000", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47808557299999999 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518360+0000", 2024-09-17T08:31:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9138999999999999e-05 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518392+0000", 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2768000000000002e-05 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238290 9.5 9:ac6fd467:::benchmark_data_smithi079_143589_object107091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 478.62059670899998, 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48563189400000001, 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238290 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046865+0000", 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046863+0000", 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046872+0000", 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9220000000000003e-06 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046890+0000", 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8059000000000001e-05 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046900+0000", 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8549999999999997e-06 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046953+0000", 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3090000000000002e-05 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.047110+0000", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156219 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,7", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.047224+0000", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114768 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.049865+0000", 2024-09-17T08:31:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002640725 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.049952+0000", 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.666e-05 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532402+0000", 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48245032700000001 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532445+0000", 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3058e-05 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.532497+0000", 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203999999999997e-05 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:119017 10.8 10:1bd4cd59:::benchmark_data_smithi079_172150_object119016:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.475200664, 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23653085400000001, 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119017 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192261+0000", 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192260+0000", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192286+0000", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6441e-05 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192313+0000", 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6330999999999999e-05 2024-09-17T08:31:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192321+0000", 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8010000000000004e-06 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192361+0000", 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0958999999999999e-05 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192488+0000", 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126334 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,7", 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.192611+0000", 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123399 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198243+0000", 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056316229999999997 2024-09-17T08:31:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198979+0000", 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073625899999999996 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428687+0000", 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.229707467 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428737+0000", 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0133999999999998e-05 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428792+0000", 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5630000000000001e-05 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:119020 10.8 10:1af502c1:::benchmark_data_smithi079_172150_object119019:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.473873475, 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23606200299999999, 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119020 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193589+0000", 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193587+0000", 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193609+0000", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2028999999999999e-05 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193620+0000", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0216e-05 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.193626+0000", 2024-09-17T08:31:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8040000000000004e-06 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195292+0000", 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016656710000000001 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195409+0000", 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011663999999999999 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,7", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.195540+0000", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131409 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.198351+0000", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028104279999999998 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199234+0000", 2024-09-17T08:31:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088331200000000003 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429570+0000", 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23033583699999999 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429618+0000", 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7882e-05 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429651+0000", 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3118999999999999e-05 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119023 10.1 e315/287 10:87de14dd:::benchmark_data_smithi079_172150_object119022:head v 315'7535, mlcod=315'7531)", 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46820858699999, 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22938773000000001, 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199254+0000", 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.199253+0000", 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200796+0000", 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543083 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200806+0000", 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3700000000000001e-06 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200813+0000", 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0450000000000003e-06 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200874+0000", 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1484999999999998e-05 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200893+0000", 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8522e-05 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428596+0000", 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22770304 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428641+0000", 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5673000000000001e-05 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:119017 10.8 e315/287 ondisk, result = 0)", 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46689531200002, 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.228234619, 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200567+0000", 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200566+0000", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200571+0000", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2630000000000003e-06 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200582+0000", 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0951999999999999e-05 2024-09-17T08:31:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200588+0000", 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1800000000000001e-06 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428672+0000", 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.228083016 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428680+0000", 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428801+0000", 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121805 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119026 10.c e315/287 10:306dec2d:::benchmark_data_smithi079_172150_object119025:head v 315'7477, mlcod=315'7475)", 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:31:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46662670199999, 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23368836000000001, 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200835+0000", 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200868+0000", 2024-09-17T08:31:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2904000000000002e-05 2024-09-17T08:31:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200875+0000", 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9450000000000004e-06 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.200879+0000", 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6310000000000002e-06 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201055+0000", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000176236 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201072+0000", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6368e-05 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.434273+0000", 2024-09-17T08:31:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23320106600000001 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.434524+0000", 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000250967 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119027 10.2 e315/287 10:42c7ce22:::benchmark_data_smithi079_172150_object119026:head v 315'7297, mlcod=315'7295)", 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:31:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46630481400001, 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23186778399999999, 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:31:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201157+0000", 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.201180+0000", 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3482e-05 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428534+0000", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22735417699999999 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428547+0000", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.275e-05 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428618+0000", 2024-09-17T08:31:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1295000000000001e-05 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428645+0000", 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6401e-05 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.432945+0000", 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043006119999999997 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.433025+0000", 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9678999999999994e-05 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.15576.0:119020 10.8 e315/287 ondisk, result = 0)", 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46505431100002, 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22724935700000001, 2024-09-17T08:31:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202408+0000", 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202407+0000", 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.202413+0000", 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428521+0000", 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.226107843 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428541+0000", 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9791000000000001e-05 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429557+0000", 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010156620000000001 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429564+0000", 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3980000000000002e-06 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429657+0000", 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2985000000000005e-05 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119030 10.c e315/287 10:3b44ad5c:::benchmark_data_smithi079_172150_object119029:head v 315'7478, mlcod=315'7475)", 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46360316900001, 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.2340527, 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203859+0000", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203856+0000", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999967 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203960+0000", 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001042 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428837+0000", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22487770800000001 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428845+0000", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1629999999999999e-06 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.431334+0000", 2024-09-17T08:31:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024897270000000002 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.431353+0000", 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8521000000000001e-05 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.437883+0000", 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065302750000000003 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.437912+0000", 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8523000000000001e-05 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.15576.0:119029 10.1 e315/287 10:898b06f3:::benchmark_data_smithi079_172150_object119028:head v 315'7536, mlcod=315'7533)", 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 341.46357331000002, 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.23247437100000001, 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203889+0000", 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.203888+0000", 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428932+0000", 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22504348800000001 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428940+0000", 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2770000000000005e-06 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.428945+0000", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.305e-06 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429160+0000", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000214477 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.429179+0000", 2024-09-17T08:31:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9454999999999999e-05 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.436327+0000", 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071472150000000002 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:36.436363+0000", 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6649000000000001e-05 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15576.0:121829 10.4 10:2418dda7:::benchmark_data_smithi079_172150_object121828:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 338.64990170200002, 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17637900000000001, 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15576", 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 121829 2024-09-17T08:31:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017560+0000", 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017559+0000", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017581+0000", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1721000000000001e-05 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017596+0000", 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5184999999999999e-05 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017602+0000", 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017651+0000", 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8971000000000003e-05 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017808+0000", 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015683900000000001 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3,4", 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.017965+0000", 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156625 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.020680+0000", 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027150059999999998 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.020812+0000", 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013275400000000001 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193767+0000", 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.17295470399999999 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193898+0000", 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013092600000000001 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:25:39.193939+0000", 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1458000000000002e-05 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:33540 11.b e368/359 11:d4d41e1f:::benchmark_data_smithi079_199625_object33539:head v 368'2106, mlcod=368'2104)", 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:28:29.120104+0000", 2024-09-17T08:31:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.54735834499999, 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.19324593200000001, 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120104+0000", 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120104+0000", 2024-09-17T08:31:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120103+0000", 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120158+0000", 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5460000000000001e-05 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120171+0000", 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.120178+0000", 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0829999999999998e-06 2024-09-17T08:31:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.121218+0000", 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001040647 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.308739+0000", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.187520408 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.313297+0000", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045581270000000004 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.313350+0000", 2024-09-17T08:31:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2797999999999998e-05 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:33546 11.9 e368/359 11:9e40829d:::benchmark_data_smithi079_199625_object33545:head v 368'2079, mlcod=368'2075)", 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:28:29.121896+0000", 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.545566278, 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.19156258100000001, 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.121896+0000", 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.121896+0000", 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.121895+0000", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.121922+0000", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7158999999999999e-05 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.308621+0000", 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18669859499999999 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.308630+0000", 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2909999999999996e-06 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.308950+0000", 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032017299999999999 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.308972+0000", 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2393000000000001e-05 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.313305+0000", 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043329450000000004 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:29.313458+0000", 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153067 2024-09-17T08:31:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:36796 11.6 11:67ff04f7:::benchmark_data_smithi079_199625_object36795:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:28:31.098329+0000", 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.56913290099999, 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16876354900000001, 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:31:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36796 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098329+0000", 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098329+0000", 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098327+0000", 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098351+0000", 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3218999999999999e-05 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098387+0000", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6664e-05 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098400+0000", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3098999999999999e-05 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098450+0000", 2024-09-17T08:31:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9279e-05 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098575+0000", 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125069 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.098704+0000", 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012885199999999999 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.103478+0000", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047745469999999996 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.104066+0000", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00058802699999999997 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267002+0000", 2024-09-17T08:31:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16293549600000001 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267054+0000", 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1842000000000003e-05 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267093+0000", 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9239000000000002e-05 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:36798 11.6 11:6c61dac3:::benchmark_data_smithi079_199625_object36797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-17T08:31:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:28:31.100045+0000", 2024-09-17T08:31:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.56741679699999, 2024-09-17T08:31:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16718087100000001, 2024-09-17T08:31:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36798 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100045+0000", 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100045+0000", 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100044+0000", 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100067+0000", 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3496e-05 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100081+0000", 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3684e-05 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100090+0000", 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999999e-06 2024-09-17T08:31:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100170+0000", 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9946999999999999e-05 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100282+0000", 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111568 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100426+0000", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014388399999999999 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.103486+0000", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030603259999999999 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.104097+0000", 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061083399999999999 2024-09-17T08:31:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267137+0000", 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16304058399999999 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267194+0000", 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6561999999999998e-05 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267226+0000", 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2296999999999999e-05 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:36804 11.6 11:6f6fef12:::benchmark_data_smithi079_199625_object36803:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:28:31.100968+0000", 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.56649459900001, 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.166988735, 2024-09-17T08:31:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36804 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100968+0000", 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100968+0000", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100967+0000", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.100990+0000", 2024-09-17T08:31:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3742000000000001e-05 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.101007+0000", 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6816999999999998e-05 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.101015+0000", 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6769999999999995e-06 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.101705+0000", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069058100000000001 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.101853+0000", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000147281 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.101997+0000", 2024-09-17T08:31:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014437 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.103638+0000", 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016406890000000001 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.104201+0000", 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00056285600000000003 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267665+0000", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.16346406799999999 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267847+0000", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000182635 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:28:31.267956+0000", 2024-09-17T08:31:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109027 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158228 11.6 11:62fafd48:::benchmark_data_smithi079_199625_object29350:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.576124+0000", 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.091338225000001, 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.34088306699999998, 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:31:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158228 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576124+0000", 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576124+0000", 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576122+0000", 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576129+0000", 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1080000000000004e-06 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576144+0000", 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4768e-05 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576152+0000", 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6680000000000001e-06 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576201+0000", 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8742999999999997e-05 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576290+0000", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9289000000000005e-05 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.576359+0000", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9006999999999998e-05 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.578760+0000", 2024-09-17T08:31:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024008010000000002 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.579413+0000", 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065332300000000003 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.916911+0000", 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33749810099999999 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.916961+0000", 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0322999999999999e-05 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917007+0000", 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5621000000000002e-05 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158232 11.6 11:6343935b:::benchmark_data_smithi079_199625_object29354:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.582382+0000", 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.085080395000006, 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33502390900000001, 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158232 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582382+0000", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582382+0000", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582380+0000", 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:31:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582388+0000", 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3710000000000002e-06 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582399+0000", 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1139e-05 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:31:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582404+0000", 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9309999999999999e-06 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582433+0000", 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9748e-05 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582512+0000", 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8135999999999999e-05 2024-09-17T08:31:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.582578+0000", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6125999999999997e-05 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.584805+0000", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022268980000000002 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.585028+0000", 2024-09-17T08:31:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000223563 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917166+0000", 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33213819999999999 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917364+0000", 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000197529 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:31:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917406+0000", 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1795999999999998e-05 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:17.792 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:31:17.792 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 4.19 scrub starts 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 4.19 scrub ok 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:31:17.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 4.19 scrub starts 2024-09-17T08:31:17.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 4.19 scrub ok 2024-09-17T08:31:17.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:31:17.794 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:17 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:31:17.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.795 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:17.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:17.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:17.932 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:17.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:17.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:17.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 4.19 scrub starts 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 4.19 scrub ok 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:31:17.952 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:17 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:31:18.034 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:18.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.080 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:18.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:18.110 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:31:18.110 INFO:teuthology.orchestra.run.smithi173.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T08:31:18.110 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:31:18.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:18.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.218 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:18.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:18.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:18.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:18.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:18.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.498 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:18.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:18.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:18.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:31:18.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.636 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:18.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:18.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-17T08:31:18.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:18.802 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:18.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:18.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:18.934 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:18.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:18.935 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: pgmap v963: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 289 B/s rd, 0 B/s wr, 631 op/s; 6688/104220 objects misplaced (6.417%); 8.1 MiB/s, 127 objects/s recovering 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: 11.4 scrub starts 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:31:18.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:18.989 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:19.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:19.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:19.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:31:19.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: pgmap v963: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 289 B/s rd, 0 B/s wr, 631 op/s; 6688/104220 objects misplaced (6.417%); 8.1 MiB/s, 127 objects/s recovering 2024-09-17T08:31:19.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: 11.4 scrub starts 2024-09-17T08:31:19.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:31:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:31:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:31:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:31:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:19.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: pgmap v963: 158 pgs: 4 active+remapped+backfill_toofull, 154 active+clean; 2.1 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 289 B/s rd, 0 B/s wr, 631 op/s; 6688/104220 objects misplaced (6.417%); 8.1 MiB/s, 127 objects/s recovering 2024-09-17T08:31:19.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: 11.4 scrub starts 2024-09-17T08:31:19.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:31:19.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:31:19.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:31:19.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:31:19.123 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:19.185 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:19.212 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:19.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:19.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:19.367 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:19.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:19.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:19.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.521 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:19.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:19.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:19.664 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:19.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:19.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:19.798 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:19.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[40649]: 11.4 scrub ok 2024-09-17T08:31:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:31:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:31:19.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:31:19.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:31:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[29565]: 11.4 scrub ok 2024-09-17T08:31:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:31:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:31:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:31:19.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:19 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:31:19.882 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:19.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:19.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:19.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:19.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:19 smithi173 ceph-mon[33187]: 11.4 scrub ok 2024-09-17T08:31:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:19 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:31:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:19 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:31:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:19 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:31:19.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:19 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:31:20.052 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:20.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:20.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:20.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:20.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.225 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:20.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:20.326 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:20.357 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:20.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:20.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:20.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.512 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:20.512 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:20.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:20.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:20.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:20.793 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:20.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:20.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:20.913 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:20 smithi079 ceph-mon[40649]: pgmap v964: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 895 B/s rd, 0 B/s wr, 1.41k op/s; 5335/83658 objects misplaced (6.377%); 7.4 MiB/s, 117 objects/s recovering 2024-09-17T08:31:20.914 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:20 smithi079 ceph-mon[29565]: pgmap v964: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 895 B/s rd, 0 B/s wr, 1.41k op/s; 5335/83658 objects misplaced (6.377%); 7.4 MiB/s, 117 objects/s recovering 2024-09-17T08:31:20.934 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:20.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:20.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:20.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:20 smithi173 ceph-mon[33187]: pgmap v964: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 895 B/s rd, 0 B/s wr, 1.41k op/s; 5335/83658 objects misplaced (6.377%); 7.4 MiB/s, 117 objects/s recovering 2024-09-17T08:31:21.036 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:21.072 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:21.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:21.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:21.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:21.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:21.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:21.320 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:31:21.320 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:31:21.349 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:21.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:21.356 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:31:21.356 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:31:21.391 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:31:21.392 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:31:21.422 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:31:21.423 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:31:21.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:21.459 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:31:21.459 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:31:21.487 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:31:21.487 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:31:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.489 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:21.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:21.525 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:31:21.525 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:31:21.560 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:31:21.561 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:31:21.580 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:21.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:21.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.631 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:21.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:21.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:21.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:21.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:21.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[29565]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:21.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[29565]: 1.9 scrub starts 2024-09-17T08:31:21.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[29565]: 1.9 scrub ok 2024-09-17T08:31:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2296495155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[40649]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[40649]: 1.9 scrub starts 2024-09-17T08:31:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[40649]: 1.9 scrub ok 2024-09-17T08:31:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:21 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2296495155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:21.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:21.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:21.911 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:21.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:21 smithi173 ceph-mon[33187]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:21 smithi173 ceph-mon[33187]: 1.9 scrub starts 2024-09-17T08:31:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:21 smithi173 ceph-mon[33187]: 1.9 scrub ok 2024-09-17T08:31:21.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:21 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2296495155' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:22.011 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:22.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:22.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:22.152 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:22.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.194 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:22.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:22.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:22.332 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:22.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:22.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:22.475 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:22.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:22.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:22.611 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:22.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:22.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:22.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.759 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:22.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:22.860 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:22.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:22.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:22.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:22.991 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:23.022 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:23.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:23.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:23.168 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:23.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:23.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:23.278 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:23.296 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:23.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:23.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:23.432 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:23.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:23.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:23.565 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:23.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:23.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:23.695 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:23.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.696 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:23.797 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:23.822 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:23.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[40649]: pgmap v965: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 792 B/s rd, 0 B/s wr, 1.25k op/s; 5335/83658 objects misplaced (6.377%); 6.6 MiB/s, 103 objects/s recovering 2024-09-17T08:31:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[40649]: osdmap e413: 8 total, 8 up, 7 in 2024-09-17T08:31:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[29565]: pgmap v965: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 792 B/s rd, 0 B/s wr, 1.25k op/s; 5335/83658 objects misplaced (6.377%); 6.6 MiB/s, 103 objects/s recovering 2024-09-17T08:31:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:23 smithi079 ceph-mon[29565]: osdmap e413: 8 total, 8 up, 7 in 2024-09-17T08:31:23.924 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:23.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:23.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:23.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:23.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:23 smithi173 ceph-mon[33187]: pgmap v965: 158 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.7 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 792 B/s rd, 0 B/s wr, 1.25k op/s; 5335/83658 objects misplaced (6.377%); 6.6 MiB/s, 103 objects/s recovering 2024-09-17T08:31:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:23 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:23 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:23 smithi173 ceph-mon[33187]: osdmap e413: 8 total, 8 up, 7 in 2024-09-17T08:31:24.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:24.089 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:24.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:24.191 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:24.201 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:31:24.201 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:31:24.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.224 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:24.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:24.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:24.363 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 1] out_osds: [3] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:31:24.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:31:24.363 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T08:31:24.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-17T08:31:24.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:24.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:24.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.520 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:24.520 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:24.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:24.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:24.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:24.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:24.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.801 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:24.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:24.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:24 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:31:24.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:24.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:24 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:31:24.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:24.942 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:24.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:24.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:24.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:24 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:31:25.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:25.075 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:25.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:25.211 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:25.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:25.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.351 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.351 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:25.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:25.485 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:25.587 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:25.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:25 smithi079 ceph-mon[40649]: pgmap v967: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 8.6 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 4125/71421 objects misplaced (5.776%); 3.2 MiB/s, 51 objects/s recovering 2024-09-17T08:31:25.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:25 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:31:25.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:25 smithi079 ceph-mon[29565]: pgmap v967: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 8.6 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 4125/71421 objects misplaced (5.776%); 3.2 MiB/s, 51 objects/s recovering 2024-09-17T08:31:25.620 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:25 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:31:25.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.624 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:25.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-17T08:31:25.725 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:25.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.761 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:25.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:25.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:31:25.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:25.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:25.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:25.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:25 smithi173 ceph-mon[33187]: pgmap v967: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 8.6 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 4125/71421 objects misplaced (5.776%); 3.2 MiB/s, 51 objects/s recovering 2024-09-17T08:31:25.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:25 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:31:26.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:26.086 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:26.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:26.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:26.218 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:26.320 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:26.354 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:26.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:26.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:26.487 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:26.490 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:26.490 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:26.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.511 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:26.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:26.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:26.625 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:26 smithi079 ceph-mon[29565]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:26.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.657 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:26.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:26.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:26.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:26.793 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:26.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:26.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:26.933 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:26.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:26.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:26 smithi173 ceph-mon[33187]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:26.983 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:26 smithi079 ceph-mon[40649]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:27.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:27.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.069 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:27.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:27.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:27.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.208 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:27.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:27.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:27.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.346 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:27.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:27.447 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:27.476 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:27.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:27.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:27.610 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:27.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:27.662 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:31:27.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:27.747 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:27.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:27.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:27.888 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:27.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:27.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:27 smithi173 ceph-mon[33187]: pgmap v968: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 3314/65613 objects misplaced (5.051%); 6.2 MiB/s, 98 objects/s recovering 2024-09-17T08:31:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:27 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:31:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:27 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2486434351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:27 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:27.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:28.029 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:28.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:28.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[40649]: pgmap v968: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 3314/65613 objects misplaced (5.051%); 6.2 MiB/s, 98 objects/s recovering 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2486434351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[29565]: pgmap v968: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 3314/65613 objects misplaced (5.051%); 6.2 MiB/s, 98 objects/s recovering 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2486434351' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:28.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:27 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:31:28.131 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:28.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.154 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:28.154 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:28.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:28.279 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:28.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:28.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:28.401 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:28.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:28.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:28.535 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:28.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:28.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:28.675 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:28.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:28.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:28.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.806 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:28.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:28.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:28.941 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:28.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:28.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:28 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:31:28.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:28 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:31:28.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:28 smithi173 ceph-mon[33187]: osdmap e414: 8 total, 8 up, 7 in 2024-09-17T08:31:29.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:29.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.077 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:29.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:31:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:31:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[40649]: osdmap e414: 8 total, 8 up, 7 in 2024-09-17T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:28 smithi079 ceph-mon[29565]: osdmap e414: 8 total, 8 up, 7 in 2024-09-17T08:31:29.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:29.203 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:29.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:29.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.343 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:29.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:29.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:29.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:29.602 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:29.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:29 smithi173 ceph-mon[33187]: pgmap v970: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 752 op/s; 3314/65613 objects misplaced (5.051%); 7.4 MiB/s, 118 objects/s recovering 2024-09-17T08:31:29.724 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:31:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a[66404]: logger=cleanup t=2024-09-17T08:31:29.437054547Z level=info msg="Completed cleanup jobs" duration=20.548687ms 2024-09-17T08:31:29.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.738 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:29.839 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:29.862 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:29.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:29.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:29.987 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:29.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:29.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:30.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:30.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.111 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:30.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:29 smithi079 ceph-mon[40649]: pgmap v970: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 752 op/s; 3314/65613 objects misplaced (5.051%); 7.4 MiB/s, 118 objects/s recovering 2024-09-17T08:31:30.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:29 smithi079 ceph-mon[29565]: pgmap v970: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 1.3 GiB data, 8.2 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 752 op/s; 3314/65613 objects misplaced (5.051%); 7.4 MiB/s, 118 objects/s recovering 2024-09-17T08:31:30.212 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:30.234 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:30.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:30.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:30.357 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:30.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:30.459 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:30.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:30.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:30.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.607 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:30.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:30.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:30.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:30.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:30.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:30.871 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:30.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:30.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:30.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:30 smithi173 ceph-mon[33187]: osdmap e415: 8 total, 8 up, 7 in 2024-09-17T08:31:30.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:30 smithi173 ceph-mon[33187]: pgmap v972: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.79k op/s; 1402/46101 objects misplaced (3.041%); 19 MiB/s, 301 objects/s recovering 2024-09-17T08:31:30.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:30 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:31:30.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:30 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:31:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:31.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:31.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[29565]: osdmap e415: 8 total, 8 up, 7 in 2024-09-17T08:31:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[29565]: pgmap v972: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.79k op/s; 1402/46101 objects misplaced (3.041%); 19 MiB/s, 301 objects/s recovering 2024-09-17T08:31:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:31:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:31:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[40649]: osdmap e415: 8 total, 8 up, 7 in 2024-09-17T08:31:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[40649]: pgmap v972: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.79k op/s; 1402/46101 objects misplaced (3.041%); 19 MiB/s, 301 objects/s recovering 2024-09-17T08:31:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:31:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:30 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:31:31.124 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:31.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:31.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:31.248 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:31.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:31.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:31.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.371 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:31.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:31.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:31.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.495 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:31.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:31.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:31.617 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:31.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:31.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:31.742 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:31.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:31.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:31.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:31.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:31.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:31 smithi173 ceph-mon[33187]: osdmap e416: 8 total, 8 up, 7 in 2024-09-17T08:31:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:31 smithi173 ceph-mon[33187]: 5.1c deep-scrub starts 2024-09-17T08:31:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:31 smithi173 ceph-mon[33187]: 5.1c deep-scrub ok 2024-09-17T08:31:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:31 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:31:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:31 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:31:31.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:32.014 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:32.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:32.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:32.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[29565]: osdmap e416: 8 total, 8 up, 7 in 2024-09-17T08:31:32.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[29565]: 5.1c deep-scrub starts 2024-09-17T08:31:32.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[29565]: 5.1c deep-scrub ok 2024-09-17T08:31:32.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:31:32.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:31:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[40649]: osdmap e416: 8 total, 8 up, 7 in 2024-09-17T08:31:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[40649]: 5.1c deep-scrub starts 2024-09-17T08:31:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[40649]: 5.1c deep-scrub ok 2024-09-17T08:31:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:31:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:31 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:31:32.149 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:32.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:32.251 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:32.287 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:32.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:32.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:32.424 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:32.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:32.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:32.567 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:32.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.568 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:32.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:32.702 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:32.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:32.804 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:32.839 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:32.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:32.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:32.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:32.973 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:32.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:32 smithi173 ceph-mon[33187]: osdmap e417: 8 total, 8 up, 7 in 2024-09-17T08:31:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:32 smithi173 ceph-mon[33187]: pgmap v975: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.61k op/s; 1402/46101 objects misplaced (3.041%); 18 MiB/s, 286 objects/s recovering 2024-09-17T08:31:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]: dispatch 2024-09-17T08:31:32.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]: dispatch 2024-09-17T08:31:33.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[40649]: osdmap e417: 8 total, 8 up, 7 in 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[40649]: pgmap v975: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.61k op/s; 1402/46101 objects misplaced (3.041%); 18 MiB/s, 286 objects/s recovering 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]: dispatch 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]: dispatch 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[29565]: osdmap e417: 8 total, 8 up, 7 in 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[29565]: pgmap v975: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 949 MiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.61k op/s; 1402/46101 objects misplaced (3.041%); 18 MiB/s, 286 objects/s recovering 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]: dispatch 2024-09-17T08:31:33.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]: dispatch 2024-09-17T08:31:33.074 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:33.095 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:33.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:33.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:33.234 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:33.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:33.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:33.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.367 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:33.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:33.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:33.503 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:33.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:33.605 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:33.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.626 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:33.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:33.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:33.764 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:33.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:33.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:33.899 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:33.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:33.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:33.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]': finished 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]': finished 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: osdmap e418: 8 total, 8 up, 7 in 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:34.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:34.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.041 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:34.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]': finished 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]': finished 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: osdmap e418: 8 total, 8 up, 7 in 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:31:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 7]}]': finished 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 0]}]': finished 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: osdmap e418: 8 total, 8 up, 7 in 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:34.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:34.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:34.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:34.274 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:34.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:34.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:34.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:34.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.446 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:34.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:34.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:34.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.583 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:34.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:34.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:34.715 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:34.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:34.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:34.854 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:34.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:34.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:34.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: 1.6 deep-scrub starts 2024-09-17T08:31:34.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: 1.6 deep-scrub ok 2024-09-17T08:31:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: pgmap v977: 158 pgs: 2 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 153 active+clean; 760 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 749 op/s; 965/37056 objects misplaced (2.604%); 2.1 MiB/s, 32 objects/s recovering 2024-09-17T08:31:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: osdmap e419: 8 total, 8 up, 7 in 2024-09-17T08:31:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:34.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:34 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:31:34.992 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:34.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:34.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: 1.6 deep-scrub starts 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: 1.6 deep-scrub ok 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: pgmap v977: 158 pgs: 2 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 153 active+clean; 760 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 749 op/s; 965/37056 objects misplaced (2.604%); 2.1 MiB/s, 32 objects/s recovering 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: osdmap e419: 8 total, 8 up, 7 in 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:31:35.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: 1.6 deep-scrub starts 2024-09-17T08:31:35.026 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: 1.6 deep-scrub ok 2024-09-17T08:31:35.026 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: pgmap v977: 158 pgs: 2 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 153 active+clean; 760 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 749 op/s; 965/37056 objects misplaced (2.604%); 2.1 MiB/s, 32 objects/s recovering 2024-09-17T08:31:35.026 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: osdmap e419: 8 total, 8 up, 7 in 2024-09-17T08:31:35.026 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-17T08:31:35.026 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:34 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:31:35.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:35.114 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:35.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:35.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:35.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.250 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:35.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:35.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:35.385 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:35.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:35.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:35.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.518 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:35.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:35.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:35.640 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:35.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:35.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:35.780 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:35.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:35.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:35.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:35.927 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:35.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:36.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[29565]: 1.3 scrub starts 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[29565]: 1.3 scrub ok 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[40649]: 1.3 scrub starts 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[40649]: 1.3 scrub ok 2024-09-17T08:31:36.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:35 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:31:36.064 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:36.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:36.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:36.204 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:36.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:35 smithi173 ceph-mon[33187]: 1.3 scrub starts 2024-09-17T08:31:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:35 smithi173 ceph-mon[33187]: 1.3 scrub ok 2024-09-17T08:31:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:35 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:31:36.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:36.341 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:36.443 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:36.473 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:36.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:36.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:36.604 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:36.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:36.706 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:36.736 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:36.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:36.838 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:36.869 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:36.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:36.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:36.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:37.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.003 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:37.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:37.105 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[40649]: pgmap v979: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 816 B/s rd, 0 B/s wr, 884 op/s; 806/32796 objects misplaced (2.458%); 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:31:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[40649]: 4.c deep-scrub starts 2024-09-17T08:31:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[40649]: 4.c deep-scrub ok 2024-09-17T08:31:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[29565]: pgmap v979: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 816 B/s rd, 0 B/s wr, 884 op/s; 806/32796 objects misplaced (2.458%); 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:31:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[29565]: 4.c deep-scrub starts 2024-09-17T08:31:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:36 smithi079 ceph-mon[29565]: 4.c deep-scrub ok 2024-09-17T08:31:37.138 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:37.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:36 smithi173 ceph-mon[33187]: pgmap v979: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 816 B/s rd, 0 B/s wr, 884 op/s; 806/32796 objects misplaced (2.458%); 3.6 MiB/s, 57 objects/s recovering 2024-09-17T08:31:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:36 smithi173 ceph-mon[33187]: 4.c deep-scrub starts 2024-09-17T08:31:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:36 smithi173 ceph-mon[33187]: 4.c deep-scrub ok 2024-09-17T08:31:37.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:37.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:37.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:37.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:37.408 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:37.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:37.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:37.545 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:37.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:37.647 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:37.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:37.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:37.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:37.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.830 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:37.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:37.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:37.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:37.970 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:37.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:38.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:38.102 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:38.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:38.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:38.237 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:38.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:38.316 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:31:38.316 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:31:38.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:38.348 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:31:38.348 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:31:38.377 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:38.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:38.385 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:31:38.385 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:31:38.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:31:38.415 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:31:38.455 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:31:38.455 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:31:38.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:38.487 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:31:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:31:38.513 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:38.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:38.524 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:31:38.524 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:31:38.556 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:31:38.557 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:31:38.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:38.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.653 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:38.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:38.754 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:38.788 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:38.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.789 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:38.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[29565]: 3.2 deep-scrub starts 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[29565]: 3.2 deep-scrub ok 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[29565]: pgmap v980: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 680 B/s rd, 0 B/s wr, 737 op/s; 806/32796 objects misplaced (2.458%); 3.0 MiB/s, 47 objects/s recovering 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1566802614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[40649]: 3.2 deep-scrub starts 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[40649]: 3.2 deep-scrub ok 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[40649]: pgmap v980: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 680 B/s rd, 0 B/s wr, 737 op/s; 806/32796 objects misplaced (2.458%); 3.0 MiB/s, 47 objects/s recovering 2024-09-17T08:31:38.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:38 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1566802614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:38.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:38.923 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:38.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:38.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:39.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:39.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:39.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:39.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:39.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.191 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:39.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:39.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:38 smithi173 ceph-mon[33187]: 3.2 deep-scrub starts 2024-09-17T08:31:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:38 smithi173 ceph-mon[33187]: 3.2 deep-scrub ok 2024-09-17T08:31:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:38 smithi173 ceph-mon[33187]: pgmap v980: 158 pgs: 2 peering, 2 active+remapped+backfill_toofull, 154 active+clean; 672 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 680 B/s rd, 0 B/s wr, 737 op/s; 806/32796 objects misplaced (2.458%); 3.0 MiB/s, 47 objects/s recovering 2024-09-17T08:31:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:38 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1566802614' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:39.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:39.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-17T08:31:39.326 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:39.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:39.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:39.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.492 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:39.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:39.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:39.624 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:39.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:39.728 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:39.765 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:39.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:39.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:39.901 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:39.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:39.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:40.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.040 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:40.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:40.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:39 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:31:40.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:39 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:31:40.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:39 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:31:40.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:39 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:31:40.141 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:40.180 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:40.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:39 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:31:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:39 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:31:40.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:40.318 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:40.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:40.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:40.452 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:40.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:40.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:40.585 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:40.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:40.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:40.726 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:40.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:40.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:40.859 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:40.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:40.868 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Removed 128875 objects 2024-09-17T08:31:40.868 INFO:tasks.radosbench.radosbench.0.smithi079.stdout:Clean up completed and total clean up time :115.398 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:31:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: pgmap v981: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s; 312/11421 objects misplaced (2.732%); 2.2 MiB/s, 37 objects/s recovering 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: osdmap e420: 8 total, 8 up, 7 in 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 4.17 deep-scrub starts 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 4.17 deep-scrub ok 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:40.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: pgmap v981: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s; 312/11421 objects misplaced (2.732%); 2.2 MiB/s, 37 objects/s recovering 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: osdmap e420: 8 total, 8 up, 7 in 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 4.17 deep-scrub starts 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 4.17 deep-scrub ok 2024-09-17T08:31:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:40 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:31:40.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:40.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:40.985 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:40.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:41.086 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:41.110 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:41.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:41.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:31:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:31:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: pgmap v981: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.45k op/s; 312/11421 objects misplaced (2.732%); 2.2 MiB/s, 37 objects/s recovering 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: osdmap e420: 8 total, 8 up, 7 in 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 4.17 deep-scrub starts 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 4.17 deep-scrub ok 2024-09-17T08:31:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:40 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:31:41.234 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:41.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:41.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:41.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:41.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:41.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:41.486 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:41.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:41.566 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:31:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:31:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:31:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:31:41.566 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:31:41.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:41.612 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:41.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:41.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:41.739 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:41.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:31:41.836 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:31:41.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:31:41.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:31:41.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:31:41.837 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:41 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:31:41.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:41.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.865 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:41.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:41.991 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:41.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:41.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:42.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 420, 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66e8930a-74cb-11ef-bceb-c7b262605968", 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-17T08:04:41.066270+0000", 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-17T08:31:39.733069+0000", 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-17T08:20:22.465407+0000", 2024-09-17T08:31:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-17T08:30:46.603459+0000", 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 19, 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T08:31:42.106 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 11, 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:07:44.689342+0000", 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 11, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 11, 2024-09-17T08:31:42.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 11, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 11, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 11, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "334", 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "332", 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.110 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.111 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-17T08:31:42.112 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 3.1800000667572021, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 3.1800000667572021, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3.1800000667572021, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3.1800000667572021, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.113 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".rgw.root", 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.075171+0000", 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.114 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.115 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "66", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.116 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.118 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 1.9700000286102295, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 1.9700000286102295, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-17T08:31:42.119 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "datapool", 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:53.309442+0000", 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.120 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 3, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 3, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 3, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 3, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 3, 2024-09-17T08:31:42.121 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.123 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "69", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 69, 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.3299999237060547, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.3299999237060547, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.86000001430511475, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 4, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.log", 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:54.771473+0000", 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.128 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.129 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "68", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.131 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.132 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.4100000858306885, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.4100000858306885, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.4100000858306885, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.4100000858306885, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.133 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 5, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.control", 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:56.745903+0000", 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.134 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.135 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "70", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.136 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.137 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:31:42.138 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.190000057220459, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.190000057220459, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.190000057220459, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.190000057220459, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.139 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 6, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "default.rgw.meta", 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:09:58.808366+0000", 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.140 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.142 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_bias": 4 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.144 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw": {} 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.190000057220459, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.190000057220459, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.190000057220459, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.190000057220459, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 11, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "unique_pool_4", 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-17T08:27:59.045325+0000", 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 3, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 2, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-17T08:31:42.146 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 16, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 16, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 16, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 16, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 16, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-17T08:31:42.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "364", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-17T08:31:42.148 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-17T08:31:42.149 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "rados": {} 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-17T08:31:42.150 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.630000114440918, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.630000114440918, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.630000114440918, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.630000114440918, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.68974494934082031, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.68999999761581421 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.151 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "793e335a-ea62-499c-af46-e48c3190408a", 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 8, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 418, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.152 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-17T08:31:42.155 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2553823997 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/2553823997", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/2553823997", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/2553823997", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/2553823997", 2024-09-17T08:31:42.156 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "dda3e35f-7364-47fa-af06-610c50ac856d", 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 13, 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 416, 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-17T08:31:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 221772614 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/221772614", 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/221772614", 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/221772614", 2024-09-17T08:31:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/221772614", 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e121b6c9-f67b-4802-a92b-9c709ff433c2", 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.051025390625, 2024-09-17T08:31:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 18, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 204, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 227, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 408, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 205, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-17T08:31:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 359954782 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/359954782", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/359954782", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/359954782", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/359954782", 2024-09-17T08:31:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "69af955a-9dbe-4d86-8278-27df098a3886", 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 0, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 27, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 403, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3442203570 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/3442203570", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/3442203570", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/3442203570", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/3442203570", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2ab07ff0-d7f3-4fca-a97b-843f7e69fc35", 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 32, 2024-09-17T08:31:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 408, 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6800", 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6801", 2024-09-17T08:31:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6802", 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6803", 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6806", 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6807", 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6804", 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6805", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2625332597 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6801/2625332597", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6803/2625332597", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6807/2625332597", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6805/2625332597", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2fc412d9-7b28-43b0-b7d3-80fff14855ad", 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 41, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 205, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6808", 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6809", 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6810", 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6811", 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6814", 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6815", 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6812", 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6813", 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 329100432 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6809/329100432", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6811/329100432", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6815/329100432", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6813/329100432", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "19197da7-927a-4422-8d18-ca9918136919", 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.9889984130859375, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 50, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 418, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6816", 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6817", 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6818", 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6819", 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6822", 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6823", 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6820", 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6821", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1332566206 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6817/1332566206", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6819/1332566206", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6823/1332566206", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6821/1332566206", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "b32fec83-5977-488d-b36c-bba34b9f3dc0", 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.477935791015625, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 59, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 408, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-17T08:31:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-17T08:31:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6824", 2024-09-17T08:31:42.193 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6825", 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6826", 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6827", 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6830", 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6831", 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6828", 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.173:6829", 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 777262325 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.173:6825/777262325", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.173:6827/777262325", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.173:6831/777262325", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.173:6829/777262325", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:12.201846+0000", 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:04.326959+0000", 2024-09-17T08:31:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:22.580874+0000", 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 204 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-17T08:31:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 65536, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:30:43.473459+0000", 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:39.420251+0000", 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:03.764792+0000", 2024-09-17T08:31:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:31:08.403180+0000", 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-17T08:31:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-17T08:30:29.452410+0000", 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: 4, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:31:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: 6 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "5.2", 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: 6, 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-17T08:31:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.f", 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 4, 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 0 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.12", 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 7, 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.7", 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 7 2024-09-17T08:31:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.a", 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 0 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.f", 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.10", 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 5 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "4.18", 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "6.b", 2024-09-17T08:31:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 1, 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 6 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [ 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "11.1", 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:31:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: 6, 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: 0, 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: 3 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "11.2", 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: 0, 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: 2, 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: 3 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-17T08:31:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": { 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3019822778": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3759482241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2023211636": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/128416645": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2554615564": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3326230156": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/673928727": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1830656502": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/1850619208": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/610659042": "2024-09-18T08:05:11.956861+0000", 2024-09-17T08:31:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/2454308049": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/107735338": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:0/3951775509": "2024-09-18T08:05:34.382508+0000", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6800/3253737241": "2024-09-18T08:10:31.196134+0000", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "172.21.15.79:6801/3253737241": "2024-09-18T08:10:31.196134+0000" 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:31:42.215 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:31:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:41 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:31:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:41 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:31:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:41 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:31:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:41 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:31:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:41 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:31:42.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:42.320 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:42.424 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:42.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.451 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:42.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:42.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.578 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:42.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T08:31:42.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:42.704 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:42.706 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-17T08:31:42.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-17T08:31:42.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:42.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-17T08:31:42.829 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:42.931 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:42.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:42.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:42.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[29565]: pgmap v983: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 312/11421 objects misplaced (2.732%); 1.2 MiB/s, 21 objects/s recovering 2024-09-17T08:31:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2802210539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:31:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:31:43.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:31:43.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[40649]: pgmap v983: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 312/11421 objects misplaced (2.732%); 1.2 MiB/s, 21 objects/s recovering 2024-09-17T08:31:43.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2802210539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:31:43.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:31:43.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:42 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:31:43.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:43.072 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:43.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:43.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.205 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:43.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:42 smithi173 ceph-mon[33187]: pgmap v983: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 226 MiB data, 5.4 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 312/11421 objects misplaced (2.732%); 1.2 MiB/s, 21 objects/s recovering 2024-09-17T08:31:43.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2802210539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-17T08:31:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:42 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:31:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:42 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:31:43.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:43.327 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:43.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:43.451 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:43.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:43.575 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:43.600 INFO:tasks.thrashosds.thrasher:j is {'epoch': 420, 'fsid': '66e8930a-74cb-11ef-bceb-c7b262605968', 'created': '2024-09-17T08:04:41.066270+0000', 'modified': '2024-09-17T08:31:39.733069+0000', 'last_up_change': '2024-09-17T08:20:22.465407+0000', 'last_in_change': '2024-09-17T08:30:46.603459+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T08:07:44.689342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '334', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '332', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 3.180000066757202, 'score_stable': 3.180000066757202, 'optimal_score': 1, 'raw_score_acting': 3.180000066757202, 'raw_score_stable': 3.180000066757202, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-17T08:09:53.075171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 1.9700000286102295, 'score_stable': 1.9700000286102295, 'optimal_score': 1, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-17T08:09:53.309442+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-17T08:09:54.771473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.4100000858306885, 'score_stable': 2.4100000858306885, 'optimal_score': 1, 'raw_score_acting': 2.4100000858306885, 'raw_score_stable': 2.4100000858306885, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-17T08:09:56.745903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1, 'raw_score_acting': 2.190000057220459, 'raw_score_stable': 2.190000057220459, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-17T08:09:58.808366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1, 'raw_score_acting': 2.190000057220459, 'raw_score_stable': 2.190000057220459, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-17T08:27:59.045325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '364', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.6897449493408203, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.6899999976158142}}], 'osds': [{'osd': 0, 'uuid': '793e335a-ea62-499c-af46-e48c3190408a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 418, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2553823997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2553823997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 2553823997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2553823997}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2553823997}]}, 'public_addr': '172.21.15.79:6803/2553823997', 'cluster_addr': '172.21.15.79:6805/2553823997', 'heartbeat_back_addr': '172.21.15.79:6809/2553823997', 'heartbeat_front_addr': '172.21.15.79:6807/2553823997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'dda3e35f-7364-47fa-af06-610c50ac856d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 416, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 221772614}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 221772614}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 221772614}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 221772614}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 221772614}]}, 'public_addr': '172.21.15.79:6811/221772614', 'cluster_addr': '172.21.15.79:6813/221772614', 'heartbeat_back_addr': '172.21.15.79:6817/221772614', 'heartbeat_front_addr': '172.21.15.79:6815/221772614', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e121b6c9-f67b-4802-a92b-9c709ff433c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.051025390625, 'last_clean_begin': 18, 'last_clean_end': 204, 'up_from': 227, 'up_thru': 408, 'down_at': 205, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 359954782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 359954782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 359954782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 359954782}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 359954782}]}, 'public_addr': '172.21.15.79:6819/359954782', 'cluster_addr': '172.21.15.79:6821/359954782', 'heartbeat_back_addr': '172.21.15.79:6825/359954782', 'heartbeat_front_addr': '172.21.15.79:6823/359954782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '69af955a-9dbe-4d86-8278-27df098a3886', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 403, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 3442203570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 3442203570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 3442203570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 3442203570}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 3442203570}]}, 'public_addr': '172.21.15.79:6827/3442203570', 'cluster_addr': '172.21.15.79:6829/3442203570', 'heartbeat_back_addr': '172.21.15.79:6833/3442203570', 'heartbeat_front_addr': '172.21.15.79:6831/3442203570', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2ab07ff0-d7f3-4fca-a97b-843f7e69fc35', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 408, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 2625332597}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 2625332597}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 2625332597}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 2625332597}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 2625332597}]}, 'public_addr': '172.21.15.173:6801/2625332597', 'cluster_addr': '172.21.15.173:6803/2625332597', 'heartbeat_back_addr': '172.21.15.173:6807/2625332597', 'heartbeat_front_addr': '172.21.15.173:6805/2625332597', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2fc412d9-7b28-43b0-b7d3-80fff14855ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 41, 'up_thru': 205, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 329100432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 329100432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 329100432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 329100432}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 329100432}]}, 'public_addr': '172.21.15.173:6809/329100432', 'cluster_addr': '172.21.15.173:6811/329100432', 'heartbeat_back_addr': '172.21.15.173:6815/329100432', 'heartbeat_front_addr': '172.21.15.173:6813/329100432', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '19197da7-927a-4422-8d18-ca9918136919', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9889984130859375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 50, 'up_thru': 418, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 1332566206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 1332566206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 1332566206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 1332566206}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 1332566206}]}, 'public_addr': '172.21.15.173:6817/1332566206', 'cluster_addr': '172.21.15.173:6819/1332566206', 'heartbeat_back_addr': '172.21.15.173:6823/1332566206', 'heartbeat_front_addr': '172.21.15.173:6821/1332566206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b32fec83-5977-488d-b36c-bba34b9f3dc0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.477935791015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 59, 'up_thru': 408, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 777262325}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 777262325}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 777262325}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 777262325}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 777262325}]}, 'public_addr': '172.21.15.173:6825/777262325', 'cluster_addr': '172.21.15.173:6827/777262325', 'heartbeat_back_addr': '172.21.15.173:6831/777262325', 'heartbeat_front_addr': '172.21.15.173:6829/777262325', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:12.201846+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:04.326959+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:22.580874+0000', 'dead_epoch': 204}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T08:30:43.473459+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:39.420251+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:03.764792+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:31:08.403180+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T08:30:29.452410+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '5.2', 'osds': [5, 6, 2]}], 'pg_upmap_items': [{'pgid': '2.f', 'mappings': [{'from': 7, 'to': 2}, {'from': 4, 'to': 0}]}, {'pgid': '2.12', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 7}]}, {'pgid': '4.a', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.10', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '6.b', 'mappings': [{'from': 1, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '11.1', 'osds': [6, 0, 3]}, {'pgid': '11.2', 'osds': [0, 2, 3]}], 'primary_temp': [], 'blocklist': {'172.21.15.79:0/3019822778': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3759482241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/2023211636': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/128416645': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2554615564': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/3326230156': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/673928727': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1830656502': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/1850619208': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6800/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:6801/610659042': '2024-09-18T08:05:11.956861+0000', '172.21.15.79:0/2454308049': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:0/107735338': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:0/3951775509': '2024-09-18T08:05:34.382508+0000', '172.21.15.79:6800/3253737241': '2024-09-18T08:10:31.196134+0000', '172.21.15.79:6801/3253737241': '2024-09-18T08:10:31.196134+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T08:31:43.600 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.7 2024-09-17T08:31:43.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd rm-pg-upmap-items 4.7 2024-09-17T08:31:43.637 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:43.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:43.702 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:43.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:43.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:43.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:43.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:31:43.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:31:43.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:31:43.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:31:43.865 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:43.866 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:31:43.866 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:31:43.866 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:31:43.866 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:31:43.866 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:43.927 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:43.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:43.954 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:43.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:43.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:31:43.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:31:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:31:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:31:43.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:44.056 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:44.082 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:44.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:44.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:44.210 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:44.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:44.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:44.333 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:44.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:44.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:44.461 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:44.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:44.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:44.587 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:44.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:44.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:44.714 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:44.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:44.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:44.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:44.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:44.959 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:44.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:44.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:45.006 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:45.061 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:45.087 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:45.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:45.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[29565]: pgmap v984: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 40 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1 KiB/s rd, 0 B/s wr, 1.15k op/s; 993 KiB/s, 16 objects/s recovering 2024-09-17T08:31:45.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:31:45.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:31:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[40649]: pgmap v984: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 40 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1 KiB/s rd, 0 B/s wr, 1.15k op/s; 993 KiB/s, 16 objects/s recovering 2024-09-17T08:31:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:31:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:44 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:31:45.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:45.215 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:45.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:45.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:44 smithi173 ceph-mon[33187]: pgmap v984: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 40 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1 KiB/s rd, 0 B/s wr, 1.15k op/s; 993 KiB/s, 16 objects/s recovering 2024-09-17T08:31:45.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:44 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:31:45.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:44 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:31:45.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:45.338 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:45.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:45.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:45.462 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:45.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:45.564 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:45.590 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:45.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:45.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:45.716 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:45.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:45.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[40649]: 2.1c deep-scrub starts 2024-09-17T08:31:45.815 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[40649]: 2.1c deep-scrub ok 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[29565]: 2.1c deep-scrub starts 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[29565]: 2.1c deep-scrub ok 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:31:45.816 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:45 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:31:45.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:45.839 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:45.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:45.941 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:45.966 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:45.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:45.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:46.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:46.089 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:46.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:46.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:46.213 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:46.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:46.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:45 smithi173 ceph-mon[33187]: 2.1c deep-scrub starts 2024-09-17T08:31:46.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:45 smithi173 ceph-mon[33187]: 2.1c deep-scrub ok 2024-09-17T08:31:46.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:45 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:31:46.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:45 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:31:46.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:46.334 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:46.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:46.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:46.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:46.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:46.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:46.585 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:46.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:46.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:46.709 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:46.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:46.811 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:46.836 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:46.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:46.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:46.964 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:46.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:46.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:47.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:47.087 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:47.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:47.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:46 smithi079 ceph-mon[29565]: pgmap v985: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 43 B/s, 1 objects/s recovering 2024-09-17T08:31:47.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:46 smithi079 ceph-mon[40649]: pgmap v985: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 43 B/s, 1 objects/s recovering 2024-09-17T08:31:47.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:47.210 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:47.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:47.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:46 smithi173 ceph-mon[33187]: pgmap v985: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 43 B/s, 1 objects/s recovering 2024-09-17T08:31:47.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:47.339 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:47.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:47.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:47.466 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:47.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:47.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:47.592 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:47.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:47.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:47.717 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:47.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-17T08:31:47.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:47.843 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:47.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:47.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:47.967 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:47.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:47.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:48.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:48.090 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:48.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: osdmap e421: 8 total, 8 up, 7 in 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: 1.5 deep-scrub starts 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: 1.5 deep-scrub ok 2024-09-17T08:31:48.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: osdmap e421: 8 total, 8 up, 7 in 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: 1.5 deep-scrub starts 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: 1.5 deep-scrub ok 2024-09-17T08:31:48.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:47 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: osdmap e421: 8 total, 8 up, 7 in 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: 1.5 deep-scrub starts 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: 1.5 deep-scrub ok 2024-09-17T08:31:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:47 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-17T08:31:48.192 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:48.217 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:48.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:48.318 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:48.343 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:48.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:48.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:48.467 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:48.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:48.569 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:48.592 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:48.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:48.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:48.718 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:48.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:48.842 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:48.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:48.849 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:48.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:48.965 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:48.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:48.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:48.990 INFO:teuthology.orchestra.run.smithi079.stderr:clear 4.7 pg_upmap_items mapping 2024-09-17T08:31:49.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:49.088 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:49.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: pgmap v987: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:31:49.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:49.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:31:49.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: osdmap e422: 8 total, 8 up, 7 in 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]: dispatch 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: pgmap v987: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: osdmap e422: 8 total, 8 up, 7 in 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:49.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:48 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]: dispatch 2024-09-17T08:31:49.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:49.204 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: pgmap v987: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 451 op/s 2024-09-17T08:31:49.205 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:31:49.205 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2141662182' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-17T08:31:49.205 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: osdmap e422: 8 total, 8 up, 7 in 2024-09-17T08:31:49.205 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:49.206 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:31:49.206 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:48 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]: dispatch 2024-09-17T08:31:49.212 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:49.314 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:49.335 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:49.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:49.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:49.459 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:49.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:49.582 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:49.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:49.704 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:49.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:49.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:49.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:49.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:49.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:49.952 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:49.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:50.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:50.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.076 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:50.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:50.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:50.198 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:50.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]': finished 2024-09-17T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:49 smithi173 ceph-mon[33187]: osdmap e423: 8 total, 8 up, 7 in 2024-09-17T08:31:50.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:50.301 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]': finished 2024-09-17T08:31:50.301 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:49 smithi079 ceph-mon[29565]: osdmap e423: 8 total, 8 up, 7 in 2024-09-17T08:31:50.302 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2145944818' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.7"}]': finished 2024-09-17T08:31:50.302 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:49 smithi079 ceph-mon[40649]: osdmap e423: 8 total, 8 up, 7 in 2024-09-17T08:31:50.308 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:50.318 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:31:50.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:31:50.321 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:50.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:50.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:31:50.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:31:50.352 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:31:50.353 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:31:50.367 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:31:50.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:31:50.384 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:31:50.384 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:31:50.400 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:31:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:31:50.416 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-17T08:31:50.417 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T08:31:50.423 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:50.432 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:31:50.432 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:31:50.444 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:50.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:50.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:50.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:50.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:50.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:50.692 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:50.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:50.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:50.816 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:50.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:50.918 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:50.939 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:50.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:51.041 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:51.063 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:51.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:51.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.186 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:51.267 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[29565]: pgmap v990: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-17T08:31:51.268 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[29565]: osdmap e424: 8 total, 8 up, 7 in 2024-09-17T08:31:51.268 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3144964335' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:51.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[40649]: pgmap v990: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-17T08:31:51.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[40649]: osdmap e424: 8 total, 8 up, 7 in 2024-09-17T08:31:51.268 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3144964335' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:51.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:51.309 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.310 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:51.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-17T08:31:51.411 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:51.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.437 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:51.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:50 smithi173 ceph-mon[33187]: pgmap v990: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-17T08:31:51.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:50 smithi173 ceph-mon[33187]: osdmap e424: 8 total, 8 up, 7 in 2024-09-17T08:31:51.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3144964335' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:31:51.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:51.561 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:51.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:51.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:51.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.685 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:51.786 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:51.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.808 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:51.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:51.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:51.931 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:51.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:51.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:52.032 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6 2024-09-17T08:31:52.054 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:52.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.6' 2024-09-17T08:31:52.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:52.177 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:52.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:52.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:52.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:52.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:52.425 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:52.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:52.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:52.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.549 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:52.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:52.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:52.671 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:52.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:52.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:52.797 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:52.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:52.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:52.922 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:52.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:52.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:53.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:53.046 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:53.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:53.148 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:53.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:53.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:53.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:53.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T08:31:53.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.297 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:53.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:53.317 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:31:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:31:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:31:53.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:53.420 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:53.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:53.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:53.550 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:53.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.551 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:53.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:53.673 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:53.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:53.802 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:53.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:53.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:53 smithi079 ceph-mon[40649]: pgmap v992: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T08:31:53.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:53 smithi079 ceph-mon[29565]: pgmap v992: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T08:31:53.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:53.929 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:53.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:53.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:53 smithi173 ceph-mon[33187]: pgmap v992: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T08:31:53.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:31:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:31:54.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:54.048 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:54.055 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:54.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:54.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:54.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.179 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:54.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:54.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.302 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:54.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:54.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.425 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:54.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:54.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:54.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.670 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:54.771 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:54.794 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:54.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:54.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:54.894 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-17T08:31:54.895 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T08:31:54.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:54.917 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:54.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:54.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:54 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:31:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:31:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:31:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:31:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:31:55.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:55.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.040 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:55.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:55.047 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 1] out_osds: [3] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:31:55.047 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T08:31:55.047 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-17T08:31:55.047 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-17T08:31:55.048 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6 2024-09-17T08:31:55.140 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:55.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.163 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:55.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:31:55.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:55.287 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:55.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:55.366 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:55 smithi173 systemd[1]: Stopping Ceph osd.6 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:31:55.388 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:55.409 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:55.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:55.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:55.518 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'unique_pool_4' removed 2024-09-17T08:31:55.569 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:55.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:55.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:55.693 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:55.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:55.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[50870]: 2024-09-17T08:31:55.366+0000 7f26cec69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:31:55.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[50870]: 2024-09-17T08:31:55.366+0000 7f26cec69640 -1 osd.6 424 *** Got signal Terminated *** 2024-09-17T08:31:55.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[50870]: 2024-09-17T08:31:55.366+0000 7f26cec69640 -1 osd.6 424 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: pgmap v993: 158 pgs: 1 peering, 1 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2915167956' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: osd.6 marked itself down and dead 2024-09-17T08:31:55.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:31:55.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:31:55.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:55 smithi173 ceph-mon[33187]: osdmap e425: 8 total, 7 up, 7 in 2024-09-17T08:31:55.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:55.815 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:55.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:55.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: pgmap v993: 158 pgs: 1 peering, 1 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2915167956' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: osd.6 marked itself down and dead 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[29565]: osdmap e425: 8 total, 7 up, 7 in 2024-09-17T08:31:55.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: pgmap v993: 158 pgs: 1 peering, 1 active+remapped+backfill_toofull, 156 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2915167956' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: osd.6 marked itself down and dead 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T08:31:55.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:55 smithi079 ceph-mon[40649]: osdmap e425: 8 total, 7 up, 7 in 2024-09-17T08:31:55.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:55.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:55.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:56.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:56.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-17T08:31:56.061 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:56.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:56.185 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:56.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:56.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:56.308 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:56.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:56.431 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:56.473 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:56 smithi173 podman[162172]: 2024-09-17 08:31:56.206098068 +0000 UTC m=+0.961807349 container died 0283b6651c03c2c99ff275e455fcc5c249579d9a736cbfae615596372c767f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-17T08:31:56.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:56.554 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:56.656 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:56.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:56.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:56.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2 2024-09-17T08:31:56.801 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.2' 2024-09-17T08:31:56.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:56.925 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:56.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:56.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:56.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:56 smithi173 podman[162172]: 2024-09-17 08:31:56.765949954 +0000 UTC m=+1.521659185 container cleanup 0283b6651c03c2c99ff275e455fcc5c249579d9a736cbfae615596372c767f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-17T08:31:56.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:56 smithi173 bash[162172]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6 2024-09-17T08:31:57.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:57.071 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:57.172 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:57.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.195 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:57.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1 2024-09-17T08:31:57.319 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:57.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.1' 2024-09-17T08:31:57.356 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162262]: 2024-09-17 08:31:57.034999619 +0000 UTC m=+0.763748875 container remove 0283b6651c03c2c99ff275e455fcc5c249579d9a736cbfae615596372c767f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-17T08:31:57.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:57.442 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:57.544 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:57.565 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:57.638 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:57 smithi173 ceph-mon[33187]: pgmap v995: 142 pgs: 30 stale+active+clean, 112 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:31:57.638 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:57 smithi173 ceph-mon[33187]: osdmap e426: 8 total, 7 up, 7 in 2024-09-17T08:31:57.638 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:57 smithi173 ceph-mon[33187]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:31:57.638 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.259287896 +0000 UTC m=+0.015761774 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:31:57.638 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.393806385 +0000 UTC m=+0.150280262 container create cf1d79109622e695c4263bd7a4f2773cb678a92c891a346c3c3e51241adfa472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-17T08:31:57.638 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.638243761 +0000 UTC m=+0.394717645 container init cf1d79109622e695c4263bd7a4f2773cb678a92c891a346c3c3e51241adfa472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[40649]: pgmap v995: 142 pgs: 30 stale+active+clean, 112 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[40649]: osdmap e426: 8 total, 7 up, 7 in 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[40649]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[29565]: pgmap v995: 142 pgs: 30 stale+active+clean, 112 active+clean; 740 KiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[29565]: osdmap e426: 8 total, 7 up, 7 in 2024-09-17T08:31:57.655 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:57 smithi079 ceph-mon[29565]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-17T08:31:57.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:57.689 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:57.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:57.812 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:57.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:57.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0 2024-09-17T08:31:57.930 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.643406706 +0000 UTC m=+0.399880576 container start cf1d79109622e695c4263bd7a4f2773cb678a92c891a346c3c3e51241adfa472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-17T08:31:57.930 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.763314401 +0000 UTC m=+0.519788293 container attach cf1d79109622e695c4263bd7a4f2773cb678a92c891a346c3c3e51241adfa472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=reef-19e2a2b, ceph=True) 2024-09-17T08:31:57.936 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:57.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:57.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.0' 2024-09-17T08:31:58.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:58.060 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:58.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:58.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:58.223 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:31:57 smithi173 podman[162403]: 2024-09-17 08:31:57.931214414 +0000 UTC m=+0.687688299 container died cf1d79109622e695c4263bd7a4f2773cb678a92c891a346c3c3e51241adfa472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T08:31:58.285 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:58.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:58.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:58.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.430 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:58.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:58.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:58.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service' 2024-09-17T08:31:58.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:31:58.648 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-17T08:31:58.648 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-17T08:31:58.648 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T08:31:58.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.6 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-17T08:31:58.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.679 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:58.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:58.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:58.789 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.6/config 2024-09-17T08:31:58.803 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:58.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:58.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3 2024-09-17T08:31:58.928 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: docker: command not found 2024-09-17T08:31:58.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:58.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi079 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.3' 2024-09-17T08:31:59.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7 2024-09-17T08:31:59.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:59.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:59.053 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.7' 2024-09-17T08:31:59.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:59.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:59.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:59.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:59.278 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5 2024-09-17T08:31:59.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:59.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:59.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.5' 2024-09-17T08:31:59.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4 2024-09-17T08:31:59.424 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-17T08:31:59.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:31:59.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd.4' 2024-09-17T08:31:59.527 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T08:31:59.539 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T08:31:59.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:31:59 smithi173 ceph-mon[33187]: pgmap v997: 142 pgs: 1 active+undersized, 29 stale+active+clean, 112 active+clean; 740 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 531 B/s rd, 0 op/s 2024-09-17T08:31:59.726 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-17T08:31:59.726 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T08:31:59.743 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-17T08:31:59.743 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T08:31:59.761 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-17T08:31:59.762 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T08:31:59.762 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:31:59 smithi079 ceph-mon[40649]: pgmap v997: 142 pgs: 1 active+undersized, 29 stale+active+clean, 112 active+clean; 740 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 531 B/s rd, 0 op/s 2024-09-17T08:31:59.762 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:31:59 smithi079 ceph-mon[29565]: pgmap v997: 142 pgs: 1 active+undersized, 29 stale+active+clean, 112 active+clean; 740 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 531 B/s rd, 0 op/s 2024-09-17T08:31:59.777 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-17T08:31:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T08:31:59.793 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-17T08:31:59.793 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T08:31:59.809 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-17T08:31:59.810 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T08:31:59.810 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-17T08:31:59.825 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-17T08:31:59.826 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T08:32:00.170 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:00.515 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-17T08:32:00.732 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T08:32:00.732 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T08:32:00.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4261186251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:32:00.822 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:00.822 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4261186251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:32:00.822 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4261186251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-17T08:32:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[29565]: pgmap v998: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 14 KiB/s rd, 9 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[29565]: Health check failed: Degraded data redundancy: 95/585 objects degraded (16.239%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[29565]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[29565]: osdmap e427: 8 total, 7 up, 7 in 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[40649]: pgmap v998: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 14 KiB/s rd, 9 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[40649]: Health check failed: Degraded data redundancy: 95/585 objects degraded (16.239%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[40649]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:01.771 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:01.772 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:01 smithi079 ceph-mon[40649]: osdmap e427: 8 total, 7 up, 7 in 2024-09-17T08:32:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:01 smithi173 ceph-mon[33187]: pgmap v998: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 14 KiB/s rd, 9 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:01 smithi173 ceph-mon[33187]: Health check failed: Degraded data redundancy: 95/585 objects degraded (16.239%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:01 smithi173 ceph-mon[33187]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:01 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/258383480' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:01 smithi173 ceph-mon[33187]: osdmap e427: 8 total, 7 up, 7 in 2024-09-17T08:32:03.262 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:03.263 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:32:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-17T08:32:03.263 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:03.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[29565]: pgmap v1000: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 17 KiB/s rd, 142 KiB/s wr, 21 op/s; 95/585 objects degraded (16.239%); 50 KiB/s, 0 objects/s recovering 2024-09-17T08:32:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:03.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[40649]: pgmap v1000: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 17 KiB/s rd, 142 KiB/s wr, 21 op/s; 95/585 objects degraded (16.239%); 50 KiB/s, 0 objects/s recovering 2024-09-17T08:32:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:03.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:03.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:03 smithi173 ceph-mon[33187]: pgmap v1000: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 17 KiB/s rd, 142 KiB/s wr, 21 op/s; 95/585 objects degraded (16.239%); 50 KiB/s, 0 objects/s recovering 2024-09-17T08:32:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:03.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-17T08:32:04.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-17T08:32:04.179 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:04.939 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:04.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:05.602 INFO:teuthology.orchestra.run.smithi173.stdout:6.b 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.a 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.6 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.3 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.2 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.1f 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.1b 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:2.2 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:4.e 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:6.1e 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:2.19 2024-09-17T08:32:05.603 INFO:teuthology.orchestra.run.smithi173.stdout:2.11 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:2.e 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:11.a 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:6.f 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:4.a 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:2.16 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:6.4 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:11.9 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:11.1 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:1.3 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:4.c 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:11.5 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:1.5 2024-09-17T08:32:05.604 INFO:teuthology.orchestra.run.smithi173.stdout:2.17 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:4.12 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:1.6 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:2.7 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:11.d 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:1.8 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:4.13 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:6.5 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:2.1b 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:1.a 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:2.1a 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:1.7 2024-09-17T08:32:05.605 INFO:teuthology.orchestra.run.smithi173.stdout:4.9 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:4.15 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:5.16 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:11.0 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:2.6 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:6.8 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:11.3 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:3.0 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:11.8 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:3.2 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:4.10 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:4.18 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:1.4 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:4.1 2024-09-17T08:32:05.606 INFO:teuthology.orchestra.run.smithi173.stdout:2.1d 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:4.11 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:4.1d 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:4.5 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:4.1f 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:2.14 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:4.4 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.f 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.10 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.2 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:1.0 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.18 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.1f 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:6.13 2024-09-17T08:32:05.607 INFO:teuthology.orchestra.run.smithi173.stdout:5.3 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:5.9 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:4.8 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:6.15 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:6.c 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:11.4 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:5.11 2024-09-17T08:32:05.608 INFO:teuthology.orchestra.run.smithi173.stdout:6.1 2024-09-17T08:32:05.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:05 smithi079 ceph-mon[40649]: pgmap v1001: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s rd, 110 KiB/s wr, 16 op/s; 95/585 objects degraded (16.239%); 39 KiB/s, 0 objects/s recovering 2024-09-17T08:32:05.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:05 smithi079 ceph-mon[29565]: pgmap v1001: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s rd, 110 KiB/s wr, 16 op/s; 95/585 objects degraded (16.239%); 39 KiB/s, 0 objects/s recovering 2024-09-17T08:32:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:05 smithi173 ceph-mon[33187]: pgmap v1001: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s rd, 110 KiB/s wr, 16 op/s; 95/585 objects degraded (16.239%); 39 KiB/s, 0 objects/s recovering 2024-09-17T08:32:06.651 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:06.652 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:32:06.652 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:32:06.652 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:32:06.652 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:06.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.6 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 2.19 --file /var/log/ceph/exp.2.19.6 2024-09-17T08:32:06.957 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.6/config 2024-09-17T08:32:07.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-17T08:32:07.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:07 smithi079 ceph-mon[29565]: pgmap v1002: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 12 KiB/s rd, 96 KiB/s wr, 15 op/s; 95/585 objects degraded (16.239%); 34 KiB/s, 0 objects/s recovering 2024-09-17T08:32:07.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:07 smithi079 ceph-mon[40649]: pgmap v1002: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 12 KiB/s rd, 96 KiB/s wr, 15 op/s; 95/585 objects degraded (16.239%); 34 KiB/s, 0 objects/s recovering 2024-09-17T08:32:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:07 smithi173 ceph-mon[33187]: pgmap v1002: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 12 KiB/s rd, 96 KiB/s wr, 15 op/s; 95/585 objects degraded (16.239%); 34 KiB/s, 0 objects/s recovering 2024-09-17T08:32:08.342 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-17T08:32:08.758 INFO:teuthology.orchestra.run.smithi173.stderr:Exporting 2.19 info 2.19( empty local-lis/les=401/402 n=0 ec=64/64 lis/c=401/401 les/c/f=402/402/0 sis=401) 2024-09-17T08:32:08.760 INFO:teuthology.orchestra.run.smithi173.stderr:Export successful 2024-09-17T08:32:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:08 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1387367110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:08 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:08 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1387367110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:08 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:08 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1387367110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:08 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-17T08:32:09.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-17T08:32:09.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[40649]: pgmap v1003: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 11 KiB/s rd, 88 KiB/s wr, 13 op/s; 95/585 objects degraded (16.239%); 31 KiB/s, 0 objects/s recovering 2024-09-17T08:32:09.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[40649]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:09.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:32:09.786 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[40649]: osdmap e428: 8 total, 7 up, 7 in 2024-09-17T08:32:09.786 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[29565]: pgmap v1003: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 11 KiB/s rd, 88 KiB/s wr, 13 op/s; 95/585 objects degraded (16.239%); 31 KiB/s, 0 objects/s recovering 2024-09-17T08:32:09.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[29565]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:09.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:32:09.787 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:09 smithi079 ceph-mon[29565]: osdmap e428: 8 total, 7 up, 7 in 2024-09-17T08:32:09.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:09 smithi173 ceph-mon[33187]: pgmap v1003: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 11 KiB/s rd, 88 KiB/s wr, 13 op/s; 95/585 objects degraded (16.239%); 31 KiB/s, 0 objects/s recovering 2024-09-17T08:32:09.942 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:09 smithi173 ceph-mon[33187]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-17T08:32:09.942 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:09 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-17T08:32:09.942 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:09 smithi173 ceph-mon[33187]: osdmap e428: 8 total, 7 up, 7 in 2024-09-17T08:32:09.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.6 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 2.19 2024-09-17T08:32:10.102 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.6/config 2024-09-17T08:32:10.126 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:10.699 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:10 smithi079 ceph-mon[40649]: 1.1 scrub starts 2024-09-17T08:32:10.699 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:10 smithi079 ceph-mon[40649]: 1.1 scrub ok 2024-09-17T08:32:10.699 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:10 smithi079 ceph-mon[29565]: 1.1 scrub starts 2024-09-17T08:32:10.699 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:10 smithi079 ceph-mon[29565]: 1.1 scrub ok 2024-09-17T08:32:10.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:10 smithi173 ceph-mon[33187]: 1.1 scrub starts 2024-09-17T08:32:10.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:10 smithi173 ceph-mon[33187]: 1.1 scrub ok 2024-09-17T08:32:11.829 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:11 smithi079 ceph-mon[40649]: pgmap v1005: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:11.830 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:11 smithi079 ceph-mon[29565]: pgmap v1005: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:11.965 INFO:teuthology.orchestra.run.smithi173.stdout: marking collection for removal 2024-09-17T08:32:11.965 INFO:teuthology.orchestra.run.smithi173.stdout:setting '_remove' omap key 2024-09-17T08:32:11.965 INFO:teuthology.orchestra.run.smithi173.stdout:finish_remove_pgs 2.19_head removing 2.19 2024-09-17T08:32:11.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:11 smithi173 ceph-mon[33187]: pgmap v1005: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:11.967 INFO:teuthology.orchestra.run.smithi173.stdout:Remove successful 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:210614 9.2 9:448427a4:::benchmark_data_smithi079_143589_object79415:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:32:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 556.54799352700002, 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36495978699999998, 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 210614 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:32:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945769+0000", 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945768+0000", 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945772+0000", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7929999999999997e-06 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945790+0000", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7125999999999998e-05 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945796+0000", 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1890000000000004e-06 2024-09-17T08:32:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945837+0000", 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1242000000000003e-05 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.945946+0000", 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010887600000000001 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.946056+0000", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110115 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.948721+0000", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026654339999999999 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:55.949343+0000", 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062103199999999999 2024-09-17T08:32:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310604+0000", 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36126145799999998 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310682+0000", 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8459000000000002e-05 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:56.310729+0000", 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6282000000000002e-05 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238260 9.c 9:3ad2fbde:::benchmark_data_smithi079_143589_object107061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.53505310800006, 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55530729599999995, 2024-09-17T08:32:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238260 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958709+0000", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958708+0000", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958715+0000", 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.007e-06 2024-09-17T08:32:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958728+0000", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2736e-05 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958736+0000", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3329999999999997e-06 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958824+0000", 2024-09-17T08:32:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8157000000000003e-05 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.958940+0000", 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011576199999999999 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959047+0000", 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010683699999999999 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964643+0000", 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055964830000000002 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967521+0000", 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028780609999999999 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.513927+0000", 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54640532900000005 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.513970+0000", 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3497e-05 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514017+0000", 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6619000000000002e-05 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238262 9.c 9:3e3399a0:::benchmark_data_smithi079_143589_object107063:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.53381512999999, 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55418753300000001, 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238262 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959947+0000", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959946+0000", 2024-09-17T08:32:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959952+0000", 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4500000000000003e-06 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959967+0000", 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5302999999999999e-05 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.959975+0000", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4059999999999999e-06 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961568+0000", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001593265 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961685+0000", 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117594 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:32:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.961815+0000", 2024-09-17T08:32:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012927899999999999 2024-09-17T08:32:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967563+0000", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057487989999999997 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967572+0000", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4810000000000006e-06 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514062+0000", 2024-09-17T08:32:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54648985500000002 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514101+0000", 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9530000000000003e-05 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514135+0000", 2024-09-17T08:32:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3488999999999997e-05 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238269 9.c 9:300a75d2:::benchmark_data_smithi079_143589_object107070:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.53095671100004, 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.551483222, 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238269 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962806+0000", 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962805+0000", 2024-09-17T08:32:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962808+0000", 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8140000000000001e-06 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962814+0000", 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0260000000000004e-06 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.962820+0000", 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5450000000000003e-06 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964667+0000", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001847392 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964772+0000", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010490899999999999 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.964884+0000", 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112475 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967590+0000", 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027056939999999998 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970332+0000", 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027415999999999999 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514219+0000", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.543886961 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514254+0000", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5525000000000003e-05 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514289+0000", 2024-09-17T08:32:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4709000000000003e-05 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238270 9.c 9:3be7327c:::benchmark_data_smithi079_143589_object107071:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.529963952, 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55059389999999997, 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238270 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963798+0000", 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963803+0000", 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-17T08:32:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963817+0000", 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3968000000000001e-05 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.963824+0000", 2024-09-17T08:32:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8909999999999998e-06 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967612+0000", 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037879839999999999 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967728+0000", 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116365 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,7", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.967860+0000", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131507 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970361+0000", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025006899999999999 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:18.970432+0000", 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1725999999999997e-05 2024-09-17T08:32:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514325+0000", 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.54389226899999998 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514359+0000", 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570999999999998e-05 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.514392+0000", 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3257999999999999e-05 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238273 9.1 9:8dab1ad1:::benchmark_data_smithi079_143589_object107074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.460526964, 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48495675599999999, 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238273 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033235+0000", 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033234+0000", 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033240+0000", 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2550000000000003e-06 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033252+0000", 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1790000000000001e-05 2024-09-17T08:32:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033258+0000", 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8449999999999999e-06 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033293+0000", 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5088e-05 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033384+0000", 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1261999999999995e-05 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033498+0000", 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113414 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.038146+0000", 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046477230000000003 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.042772+0000", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046263770000000001 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518104+0000", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.475331644 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518151+0000", 2024-09-17T08:32:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7188999999999998e-05 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518192+0000", 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1462999999999999e-05 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238276 9.1 9:8a6a764e:::benchmark_data_smithi079_143589_object107077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.45986502699998, 2024-09-17T08:32:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.48449216099999998, 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238276 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033897+0000", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033901+0000", 2024-09-17T08:32:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9870000000000001e-06 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033911+0000", 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0243e-05 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.033918+0000", 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9349999999999997e-06 2024-09-17T08:32:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035704+0000", 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001785998 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035838+0000", 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134497 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.035960+0000", 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122063 2024-09-17T08:32:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.038305+0000", 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023450060000000002 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.042809+0000", 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045036399999999997 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518313+0000", 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.475503499 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518356+0000", 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2954000000000002e-05 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518390+0000", 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4028e-05 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:238286 9.1 9:88a162d9:::benchmark_data_smithi079_143589_object107087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 533.44989606299998, 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47487866200000001, 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 238286 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043866+0000", 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043865+0000", 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043870+0000", 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7450000000000003e-06 2024-09-17T08:32:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043881+0000", 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1239e-05 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043887+0000", 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9859999999999999e-06 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043915+0000", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8085e-05 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.043989+0000", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3700000000000002e-05 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6,7", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.044053+0000", 2024-09-17T08:32:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4226999999999993e-05 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046200+0000", 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002146448 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.046890+0000", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069050399999999999 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518492+0000", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.47160132500000002 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518708+0000", 2024-09-17T08:32:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000216216 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:23:19.518745+0000", 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7187000000000002e-05 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:106703 11.a 11:5180121c:::benchmark_data_smithi079_199625_object106702:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:24.441796+0000", 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.05196675400001, 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.297678579, 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106703 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441796+0000", 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441796+0000", 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441795+0000", 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441812+0000", 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7394999999999999e-05 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441820+0000", 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.306e-06 2024-09-17T08:32:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.441825+0000", 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8849999999999998e-06 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.443117+0000", 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012925860000000001 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.443235+0000", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011747 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.443365+0000", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013035299999999999 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.448806+0000", 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054407559999999997 2024-09-17T08:32:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.450034+0000", 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012277449999999999 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.739393+0000", 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28935895099999998 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.739443+0000", 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9993000000000001e-05 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.739474+0000", 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1726000000000001e-05 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:106708 11.8 11:12156efd:::benchmark_data_smithi079_199625_object106707:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:24.449291+0000", 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.04447124800001, 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29383870899999998, 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106708 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449291+0000", 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449291+0000", 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449290+0000", 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449318+0000", 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7815000000000001e-05 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449330+0000", 2024-09-17T08:32:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2165e-05 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449337+0000", 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7780000000000002e-06 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.451767+0000", 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00243063 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.452710+0000", 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000943252 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.452865+0000", 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015504700000000001 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.458136+0000", 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052704559999999998 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.458144+0000", 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8140000000000001e-06 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.743076+0000", 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28493278999999999 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.743108+0000", 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1096000000000002e-05 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.743130+0000", 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2316000000000001e-05 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:106713 11.8 11:1d041a12:::benchmark_data_smithi079_199625_object106712:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:24.449450+0000", 2024-09-17T08:32:12.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.04431289199999, 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29862526099999998, 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106713 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449450+0000", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449450+0000", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449449+0000", 2024-09-17T08:32:12.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449493+0000", 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4227000000000002e-05 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449501+0000", 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6559999999999992e-06 2024-09-17T08:32:12.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.449506+0000", 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.848e-06 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.455350+0000", 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005844391 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.455476+0000", 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012626599999999999 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.455607+0000", 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000130949 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.458248+0000", 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002640351 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459487+0000", 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001239668 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.748015+0000", 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28852770700000002 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.748051+0000", 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5451000000000003e-05 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.748075+0000", 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4249000000000001e-05 2024-09-17T08:32:12.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:106719 11.a 11:58634daa:::benchmark_data_smithi079_199625_object106718:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:24.457052+0000", 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.03671052000001, 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29778907100000002, 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106719 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457052+0000", 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457052+0000", 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457051+0000", 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457099+0000", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7799000000000001e-05 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457107+0000", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2339999999999994e-06 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457112+0000", 2024-09-17T08:32:12.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2809999999999999e-06 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.457521+0000", 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040893199999999998 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.458173+0000", 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065246200000000003 2024-09-17T08:32:12.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.458327+0000", 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015335199999999999 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.467072+0000", 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0087452350000000005 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.468106+0000", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001033497 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754753+0000", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.28664758099999998 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754803+0000", 2024-09-17T08:32:12.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9699e-05 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754841+0000", 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8152999999999999e-05 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.587 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:106721 11.a 11:57501e6a:::benchmark_data_smithi079_199625_object106720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:29:24.459322+0000", 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.034440612, 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29565149699999999, 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106721 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459322+0000", 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459322+0000", 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459320+0000", 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459382+0000", 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1631000000000007e-05 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459396+0000", 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4087000000000001e-05 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.459404+0000", 2024-09-17T08:32:12.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9289999999999998e-06 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.461877+0000", 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024724629999999998 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.462194+0000", 2024-09-17T08:32:12.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031787800000000001 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.462665+0000", 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047087600000000001 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.467101+0000", 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044351959999999998 2024-09-17T08:32:12.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.468124+0000", 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001023695 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754887+0000", 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.286762348 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754940+0000", 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2998000000000003e-05 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:29:24.754973+0000", 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3742999999999999e-05 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:191217 11.1 e399/359 11:84330dd1:::benchmark_data_smithi079_199625_object62339:head v 399'11822, mlcod=399'11819)", 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:40.226924+0000", 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 92.266838687000003, 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.039612969999999997, 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226924+0000", 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226924+0000", 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226923+0000", 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226929+0000", 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4269999999999999e-06 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226940+0000", 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1426999999999999e-05 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.226954+0000", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3696000000000001e-05 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.239704+0000", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012750008 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.239722+0000", 2024-09-17T08:32:12.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7762e-05 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.266503+0000", 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026781968 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.266537+0000", 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3256000000000001e-05 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:191224 11.9 11:9f6e9334:::benchmark_data_smithi079_199625_object62346:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:40.231402+0000", 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 92.262360071000003, 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.039199746000000001, 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 191224 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231402+0000", 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231402+0000", 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231402+0000", 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231407+0000", 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231418+0000", 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1207e-05 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.231425+0000", 2024-09-17T08:32:12.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7279999999999998e-06 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.248937+0000", 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017512122000000001 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.249037+0000", 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9541000000000001e-05 2024-09-17T08:32:12.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.249136+0000", 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8974000000000005e-05 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.256491+0000", 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007355416 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.266659+0000", 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010168259000000001 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.270522+0000", 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003862851 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.270565+0000", 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2845000000000001e-05 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:40.270602+0000", 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7168000000000001e-05 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194498 11.b 11:df434530:::benchmark_data_smithi079_199625_object65620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:42.883704+0000", 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.610058514000002, 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.038934071000000001, 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194498 2024-09-17T08:32:12.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883704+0000", 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883704+0000", 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883703+0000", 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883708+0000", 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1669999999999998e-06 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883719+0000", 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0281e-05 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883726+0000", 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9670000000000004e-06 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883828+0000", 2024-09-17T08:32:12.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102698 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.883948+0000", 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011961 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.884063+0000", 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011497600000000001 2024-09-17T08:32:12.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.887753+0000", 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036897369999999998 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.889570+0000", 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181694 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.922551+0000", 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.032981819000000002 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.922592+0000", 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0883999999999997e-05 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.922638+0000", 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5711000000000003e-05 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194871 11.a 11:5a131336:::benchmark_data_smithi079_199625_object65993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:43.204021+0000", 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.289741527000004, 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041084753000000002, 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194871 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204021+0000", 2024-09-17T08:32:12.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204021+0000", 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204019+0000", 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204027+0000", 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7060000000000004e-06 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204040+0000", 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2909e-05 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204047+0000", 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6329999999999996e-06 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.215063+0000", 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011015917 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.215182+0000", 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118681 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.215287+0000", 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104634 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.227078+0000", 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011791149000000001 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.227108+0000", 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0429999999999998e-05 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.244961+0000", 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017852733999999999 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.245073+0000", 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112513 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.245106+0000", 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2184999999999998e-05 2024-09-17T08:32:12.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194875 11.a 11:56a01131:::benchmark_data_smithi079_199625_object65997:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:43.204454+0000", 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.289308801999994, 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040772709999999997, 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194875 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204454+0000", 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204454+0000", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204453+0000", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204458+0000", 2024-09-17T08:32:12.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5090000000000001e-06 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204468+0000", 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4280000000000002e-06 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.204474+0000", 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5610000000000004e-06 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.217768+0000", 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013294257 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.218014+0000", 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024528700000000001 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.218123+0000", 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109361 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.227137+0000", 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090139699999999996 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.227164+0000", 2024-09-17T08:32:12.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.245113+0000", 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017948391000000001 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.245191+0000", 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8213000000000005e-05 2024-09-17T08:32:12.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.245226+0000", 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5431e-05 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:197462 11.b 11:d7d2904b:::benchmark_data_smithi079_199625_object68584:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e400)", 2024-09-17T08:32:12.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:45.267184+0000", 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.226578248999999, 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045324353999999997, 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 197462 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267184+0000", 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267184+0000", 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267182+0000", 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267192+0000", 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9529999999999994e-06 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267214+0000", 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1562000000000001e-05 2024-09-17T08:32:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267224+0000", 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.008e-05 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267282+0000", 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8193999999999999e-05 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.267425+0000", 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000143101 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,1", 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.268310+0000", 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088550300000000005 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.274616+0000", 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063053859999999996 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.282590+0000", 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079738629999999994 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.308943+0000", 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02635301 2024-09-17T08:32:12.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.312457+0000", 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035143029999999999 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:45.312509+0000", 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1495000000000001e-05 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:199088 11.8 11:12963423:::benchmark_data_smithi079_199625_object70210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e400)", 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:46.610398+0000", 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.883364150000006, 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042293029000000003, 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 199088 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610398+0000", 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610398+0000", 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610397+0000", 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610403+0000", 2024-09-17T08:32:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2220000000000001e-06 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610412+0000", 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5279999999999992e-06 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.610418+0000", 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3090000000000001e-06 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.615273+0000", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048545189999999998 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.615390+0000", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117398 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.615505+0000", 2024-09-17T08:32:12.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011464900000000001 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.635557+0000", 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020051626999999999 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.652691+0000", 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017134730000000001 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:12.645 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:12.645 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:12.645 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-17T08:32:12.774 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2734269227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:12.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:12 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:12.811 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2734269227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:12.811 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:12 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:12.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2734269227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:12.811 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:12 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-17T08:32:13.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -n osd.6 --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.19.6 2024-09-17T08:32:13.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:13.250 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/osd.6/config 2024-09-17T08:32:13.710 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:13.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:13 smithi173 ceph-mon[33187]: pgmap v1006: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:13.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:13 smithi173 ceph-mon[33187]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:32:13.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:13 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:13.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:13 smithi173 ceph-mon[33187]: osdmap e429: 8 total, 7 up, 7 in 2024-09-17T08:32:13.827 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-17T08:32:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[29565]: pgmap v1006: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[29565]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[29565]: osdmap e429: 8 total, 7 up, 7 in 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[40649]: pgmap v1006: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[40649]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-17T08:32:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:13 smithi079 ceph-mon[40649]: osdmap e429: 8 total, 7 up, 7 in 2024-09-17T08:32:14.844 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:14 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:32:14.844 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:14 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:32:14.844 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:14.844 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:14 smithi173 ceph-mon[33187]: 2.1f scrub starts 2024-09-17T08:32:14.844 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:14 smithi173 ceph-mon[33187]: 2.1f scrub ok 2024-09-17T08:32:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:32:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:32:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[40649]: 2.1f scrub starts 2024-09-17T08:32:14.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[40649]: 2.1f scrub ok 2024-09-17T08:32:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:32:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:32:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[29565]: 2.1f scrub starts 2024-09-17T08:32:14.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:14 smithi079 ceph-mon[29565]: 2.1f scrub ok 2024-09-17T08:32:14.957 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:15.157 INFO:teuthology.orchestra.run.smithi173.stdout:get_pg_num_history pg_num_history pg_num_history(e424 pg_nums {1={332=11},2={64=32},3={64=3},4={66=32},5={68=32},6={70=32},11={359=16}} deleted_pools ) 2024-09-17T08:32:15.157 INFO:teuthology.orchestra.run.smithi173.stdout:Importing pgid 2.19 2024-09-17T08:32:15.157 INFO:teuthology.orchestra.run.smithi173.stdout:write_pg epoch 419 info 2.19( empty local-lis/les=401/402 n=0 ec=64/64 lis/c=401/401 les/c/f=402/402/0 sis=401) 2024-09-17T08:32:15.157 INFO:teuthology.orchestra.run.smithi173.stdout:Import successful 2024-09-17T08:32:15.677 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:15 smithi173 ceph-mon[33187]: pgmap v1008: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:15.677 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:15 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:32:15.677 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:15 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:32:15.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[40649]: pgmap v1008: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:32:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:32:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[29565]: pgmap v1008: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:32:15.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:15 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:32:16.333 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /var/log/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/exp.2.19.6 2024-09-17T08:32:16.923 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:16 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:16.923 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:16 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:16.923 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:16 smithi173 ceph-mon[33187]: osdmap e430: 8 total, 7 up, 7 in 2024-09-17T08:32:16.923 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:16 smithi173 ceph-mon[33187]: 1.2 scrub starts 2024-09-17T08:32:16.923 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:16 smithi173 ceph-mon[33187]: 1.2 scrub ok 2024-09-17T08:32:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:32:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:32:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[40649]: osdmap e430: 8 total, 7 up, 7 in 2024-09-17T08:32:17.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[40649]: 1.2 scrub starts 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[40649]: 1.2 scrub ok 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[29565]: osdmap e430: 8 total, 7 up, 7 in 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[29565]: 1.2 scrub starts 2024-09-17T08:32:17.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:16 smithi079 ceph-mon[29565]: 1.2 scrub ok 2024-09-17T08:32:17.259 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:17.321 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:17.321 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:32:17.321 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:17.914 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:17 smithi173 ceph-mon[33187]: pgmap v1010: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:17.937 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-17T08:32:18.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:17 smithi079 ceph-mon[40649]: pgmap v1010: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:18.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:17 smithi079 ceph-mon[29565]: pgmap v1010: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.4 KiB/s rd, 1 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:18.137 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: pgmap v1011: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:32:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:19 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:32:19.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:32:19.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:32:19.289 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: pgmap v1011: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: pgmap v1011: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 373 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 95/585 objects degraded (16.239%) 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:19.290 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:19.291 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:32:19.291 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:19 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:32:19.900 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:19.900 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:19.900 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:32:19.900 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:32:19.900 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:20.138 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:20 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:20.138 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:20 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:20.138 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:20 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:32:20.138 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:20 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:32:20.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:20 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:32:20.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-17T08:32:20.640 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:21.362 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-17T08:32:21.362 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-17T08:32:21.362 INFO:tasks.cephadm.osd.6:Restarting osd.6 (starting--it wasn't running)... 2024-09-17T08:32:21.362 DEBUG:teuthology.orchestra.run.smithi173:osd.6> sudo journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service 2024-09-17T08:32:21.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl start ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6 2024-09-17T08:32:21.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:32:21.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:32:21.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: pgmap v1012: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:21.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:32:21.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 95/582 objects degraded (16.323%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: osdmap e431: 8 total, 7 up, 7 in 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: pgmap v1012: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 95/582 objects degraded (16.323%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:21.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: osdmap e431: 8 total, 7 up, 7 in 2024-09-17T08:32:21.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:32:21.371 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:21 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: pgmap v1012: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 95/582 objects degraded (16.323%), 22 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: osdmap e431: 8 total, 7 up, 7 in 2024-09-17T08:32:21.387 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:32:21.388 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:21 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:32:21.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:21 smithi173 systemd[1]: Starting Ceph osd.6 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:115398 10.1 10:83fea32d:::benchmark_data_smithi079_172150_object115397:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e314)", 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:32.799961+0000", 2024-09-17T08:32:22.418 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 409.61576932399998, 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.12195975100000001, 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 115398 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.799961+0000", 2024-09-17T08:32:22.419 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.799961+0000", 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.799960+0000", 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.420 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.799992+0000", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1866999999999998e-05 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.800002+0000", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7729999999999994e-06 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.800008+0000", 2024-09-17T08:32:22.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4330000000000001e-06 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.803088+0000", 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030797480000000002 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.803208+0000", 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012039700000000001 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,7", 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.804589+0000", 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0013802409999999999 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.808643+0000", 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004054289 2024-09-17T08:32:22.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.809971+0000", 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001328075 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.921824+0000", 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.111852805 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.921882+0000", 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7893999999999998e-05 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:32.921921+0000", 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9073999999999999e-05 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.425 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:117200 10.d 10:beba7cdf:::benchmark_data_smithi079_172150_object117199:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.410144+0000", 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 408.00558627999999, 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10620903199999999, 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 117200 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410144+0000", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410144+0000", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410142+0000", 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410221+0000", 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8887999999999994e-05 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410232+0000", 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.045e-05 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410238+0000", 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.562e-06 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410409+0000", 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017088800000000001 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410529+0000", 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119799 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.410668+0000", 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013870000000000001 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.416068+0000", 2024-09-17T08:32:22.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0054000740000000004 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.420549+0000", 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0044809910000000001 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.516274+0000", 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.095725560000000001 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.516312+0000", 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7478999999999998e-05 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.516353+0000", 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1164000000000001e-05 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.432 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:117202 10.d 10:b87cc6e2:::benchmark_data_smithi079_172150_object117201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.412098+0000", 2024-09-17T08:32:22.433 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 408.003632547, 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.108738932, 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 117202 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412098+0000", 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412098+0000", 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412096+0000", 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412160+0000", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3393999999999996e-05 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412177+0000", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7674000000000001e-05 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.412185+0000", 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1930000000000004e-06 2024-09-17T08:32:22.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.413446+0000", 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012616529999999999 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.413562+0000", 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011606 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.413692+0000", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012973600000000001 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.416437+0000", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027454089999999999 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.420595+0000", 2024-09-17T08:32:22.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0041571139999999999 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.520726+0000", 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.100131626 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.520798+0000", 2024-09-17T08:32:22.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1431000000000001e-05 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.520837+0000", 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9038999999999997e-05 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119023 10.1 10:87de14dd:::benchmark_data_smithi079_172150_object119022:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:36.198479+0000", 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 406.217251315, 2024-09-17T08:32:22.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.23069165899999999, 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119023 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198479+0000", 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198479+0000", 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198477+0000", 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198515+0000", 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7407999999999998e-05 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198527+0000", 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2405999999999999e-05 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198534+0000", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3309999999999998e-06 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.198805+0000", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00027022599999999998 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.199011+0000", 2024-09-17T08:32:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000206455 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,7", 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.199188+0000", 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017712500000000001 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202380+0000", 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031921190000000002 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202738+0000", 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035788399999999999 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.429092+0000", 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.226353999 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.429138+0000", 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5302000000000001e-05 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.429171+0000", 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3061000000000001e-05 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.447 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119029 10.1 10:898b06f3:::benchmark_data_smithi079_172150_object119028:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:36.202695+0000", 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 406.21303496100001, 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.23505372399999999, 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119029 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202695+0000", 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202695+0000", 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202694+0000", 2024-09-17T08:32:22.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202727+0000", 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2793000000000003e-05 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202740+0000", 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2896e-05 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202746+0000", 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6819999999999997e-06 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202796+0000", 2024-09-17T08:32:22.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9197999999999999e-05 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.202946+0000", 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015070900000000001 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,7", 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.203108+0000", 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016195099999999999 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.207073+0000", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0039650739999999999 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.207785+0000", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00071131799999999995 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.437656+0000", 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.22987149800000001 2024-09-17T08:32:22.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.437699+0000", 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3124999999999998e-05 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.437749+0000", 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9697000000000002e-05 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119039 10.d 10:b5473fe6:::benchmark_data_smithi079_172150_object119038:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e315)", 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:36.216094+0000", 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 406.19963585900001, 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.229717809, 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119039 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.456 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216094+0000", 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216094+0000", 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216093+0000", 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216141+0000", 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7364000000000003e-05 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216153+0000", 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2439999999999999e-05 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216159+0000", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7320000000000003e-06 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216202+0000", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3618999999999997e-05 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216338+0000", 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013570699999999999 2024-09-17T08:32:22.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.216501+0000", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000163029 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.218919+0000", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024179340000000001 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.218980+0000", 2024-09-17T08:32:22.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1136999999999994e-05 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.445715+0000", 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.226734512 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.445773+0000", 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8650000000000003e-05 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:36.445812+0000", 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8915000000000003e-05 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119670 10.1 10:849b8499:::benchmark_data_smithi079_172150_object119669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:37.016371+0000", 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 405.399358812, 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11438048100000001, 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.462 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119670 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016371+0000", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016371+0000", 2024-09-17T08:32:22.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016370+0000", 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016404+0000", 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3735000000000001e-05 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016414+0000", 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9110000000000005e-06 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016420+0000", 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3810000000000004e-06 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016463+0000", 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3322000000000003e-05 2024-09-17T08:32:22.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016594+0000", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000130958 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,7", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.016740+0000", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014599700000000001 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.022420+0000", 2024-09-17T08:32:22.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056800870000000003 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.025250+0000", 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028299879999999999 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.130651+0000", 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.105400524 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.130709+0000", 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8016999999999997e-05 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.130752+0000", 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2951000000000003e-05 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.468 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119678 10.d 10:b887aa48:::benchmark_data_smithi079_172150_object119677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:37.026229+0000", 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 405.38950123900003, 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.118846072, 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119678 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026229+0000", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026229+0000", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026227+0000", 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:22.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026260+0000", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2758e-05 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026273+0000", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3332e-05 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026280+0000", 2024-09-17T08:32:22.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9179999999999997e-06 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026379+0000", 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8857000000000006e-05 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026511+0000", 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013213 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:22.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026653+0000", 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014144900000000001 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.032071+0000", 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0054179529999999997 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.032077+0000", 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6590000000000001e-06 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.144977+0000", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.112900111 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.145029+0000", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1783999999999998e-05 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.145075+0000", 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5840999999999997e-05 2024-09-17T08:32:22.474 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15576.0:119679 10.d 10:b4bf0e71:::benchmark_data_smithi079_172150_object119678:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e316)", 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:37.026294+0000", 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 405.38943661899998, 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.120465588, 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15576", 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/1525075198", 2024-09-17T08:32:22.475 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 119679 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026294+0000", 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026294+0000", 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026293+0000", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026318+0000", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5303000000000002e-05 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026325+0000", 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5300000000000002e-06 2024-09-17T08:32:22.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.026330+0000", 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5560000000000001e-06 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.029481+0000", 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0031515359999999999 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.029621+0000", 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000140259 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 0,6", 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.029778+0000", 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000156627 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.032190+0000", 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024125000000000001 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.032239+0000", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8551000000000003e-05 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.146681+0000", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.11444164599999999 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.146724+0000", 2024-09-17T08:32:22.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3652999999999997e-05 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:37.146759+0000", 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4944999999999998e-05 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:36177 11.e 11:7b22d3b0:::benchmark_data_smithi079_199625_object36176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:28:30.706462+0000", 2024-09-17T08:32:22.481 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 231.709268568, 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.10911483900000001, 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 36177 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706462+0000", 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706462+0000", 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706460+0000", 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706503+0000", 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2484999999999997e-05 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706514+0000", 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1202e-05 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706519+0000", 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4689999999999996e-06 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706547+0000", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7276000000000001e-05 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706636+0000", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9640000000000002e-05 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.706726+0000", 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.0161999999999995e-05 2024-09-17T08:32:22.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.707871+0000", 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011442679999999999 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.708123+0000", 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000252441 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.815483+0000", 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.107360022 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.815522+0000", 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9137000000000002e-05 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:28:30.815577+0000", 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4245e-05 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.487 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106856 11.e 11:7557c0fe:::benchmark_data_smithi079_199625_object106855:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.816557+0000", 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 177.59917281899999, 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.100236984, 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106856 2024-09-17T08:32:22.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816557+0000", 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816557+0000", 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816556+0000", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816619+0000", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2556000000000002e-05 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816631+0000", 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2391999999999999e-05 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816639+0000", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4669999999999998e-06 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.817435+0000", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00079679299999999996 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.817562+0000", 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012671500000000001 2024-09-17T08:32:22.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.817658+0000", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5568000000000004e-05 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.823029+0000", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0053716049999999998 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.823396+0000", 2024-09-17T08:32:22.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000367091 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916687+0000", 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093291186999999998 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916738+0000", 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0430999999999999e-05 2024-09-17T08:32:22.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916794+0000", 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6538e-05 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.494 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106857 11.e 11:78dc6711:::benchmark_data_smithi079_199625_object106856:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.816695+0000", 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 177.59903520500001, 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.100250463, 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106857 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816695+0000", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816695+0000", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816694+0000", 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:22.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816774+0000", 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0028e-05 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816782+0000", 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6020000000000002e-06 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.816787+0000", 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1030000000000001e-06 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.820115+0000", 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.003327736 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.820244+0000", 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012934700000000001 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.820378+0000", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000133626 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.823154+0000", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027760189999999998 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.823523+0000", 2024-09-17T08:32:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000369601 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916853+0000", 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093329700000000002 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916910+0000", 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7117999999999998e-05 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.916946+0000", 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5240999999999997e-05 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106868 11.e 11:7cf228ac:::benchmark_data_smithi079_199625_object106867:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.831157+0000", 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 177.58457355199999, 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.092858048999999998, 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.501 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106868 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831157+0000", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831157+0000", 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831155+0000", 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831220+0000", 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4499999999999996e-05 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831233+0000", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2785e-05 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831240+0000", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3479999999999998e-06 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831273+0000", 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2934e-05 2024-09-17T08:32:22.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.831361+0000", 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8263999999999994e-05 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.914080+0000", 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.082718353999999994 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.917065+0000", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029854109999999999 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.917116+0000", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1335999999999998e-05 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.923747+0000", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0066310190000000001 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.923837+0000", 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9746000000000004e-05 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.924015+0000", 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017760700000000001 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:133333 11.e 11:7cebea56:::benchmark_data_smithi079_199625_object4455:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:32:22.507 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.540805+0000", 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.874924988, 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.18182327300000001, 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 133333 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540805+0000", 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540805+0000", 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540804+0000", 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540811+0000", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0369999999999997e-06 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540824+0000", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3448000000000001e-05 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540832+0000", 2024-09-17T08:32:22.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7719999999999994e-06 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.540894+0000", 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2187999999999995e-05 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.541000+0000", 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010573099999999999 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.511 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.541103+0000", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000102827 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.543803+0000", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027002340000000001 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545646+0000", 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001843005 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722533+0000", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.17688707400000001 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722586+0000", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2608999999999997e-05 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722629+0000", 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2935999999999997e-05 2024-09-17T08:32:22.513 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:133334 11.c e387/359 11:30970702:::benchmark_data_smithi079_199625_object4456:head v 387'8361, mlcod=387'8359)", 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.542718+0000", 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.87301211400001, 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.184513072, 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542718+0000", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542718+0000", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542717+0000", 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542729+0000", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1432e-05 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542741+0000", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2836999999999999e-05 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542748+0000", 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9110000000000004e-06 2024-09-17T08:32:22.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542921+0000", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017226599999999999 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.542941+0000", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0469999999999999e-05 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.727166+0000", 2024-09-17T08:32:22.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.184224634 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.727231+0000", 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5556999999999996e-05 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:133336 11.c e387/359 11:37b0f737:::benchmark_data_smithi079_199625_object4458:head v 387'8362, mlcod=387'8359)", 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.545511+0000", 2024-09-17T08:32:22.518 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.87021967199999, 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.18553583000000001, 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545511+0000", 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545511+0000", 2024-09-17T08:32:22.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545510+0000", 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545520+0000", 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0708e-05 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545533+0000", 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2414e-05 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545539+0000", 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6189999999999996e-06 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545583+0000", 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4045000000000003e-05 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.545602+0000", 2024-09-17T08:32:22.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8660000000000001e-05 2024-09-17T08:32:22.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.730941+0000", 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.18533854699999999 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.731046+0000", 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000105868 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.523 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.25757.0:133333 11.e e387/359 ondisk, result = 0)", 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.546246+0000", 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.86948464100001, 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.17638832300000001, 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.546246+0000", 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.546246+0000", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.546245+0000", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.546250+0000", 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3419999999999998e-06 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722461+0000", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.17621041200000001 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722473+0000", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2455e-05 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722516+0000", 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2812999999999998e-05 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722525+0000", 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3030000000000005e-06 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722634+0000", 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000108848 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.527 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:133339 11.d e387/359 11:be96758d:::benchmark_data_smithi079_199625_object4461:head v 387'8234, mlcod=387'8232)", 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.547401+0000", 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.86832881399999, 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.18094911699999999, 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.547401+0000", 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.547401+0000", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.547401+0000", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.547409+0000", 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3930000000000008e-06 2024-09-17T08:32:22.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722451+0000", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.175041471 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722466+0000", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5556000000000001e-05 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722666+0000", 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000200175 2024-09-17T08:32:22.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722685+0000", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.908e-05 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.728290+0000", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0056044409999999999 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.728351+0000", 2024-09-17T08:32:22.531 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0815e-05 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:133364 11.e 11:795d248a:::benchmark_data_smithi079_199625_object4486:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e387)", 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:49.571361+0000", 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 152.84436943, 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.20294066999999999, 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.532 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 133364 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.571361+0000", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.571361+0000", 2024-09-17T08:32:22.533 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.571359+0000", 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.571369+0000", 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7680000000000003e-06 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.534 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722710+0000", 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.15134172700000001 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722720+0000", 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0148e-05 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722752+0000", 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1377000000000001e-05 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.535 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722869+0000", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116822 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.722992+0000", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000123779 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.731367+0000", 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0083750000000000005 2024-09-17T08:32:22.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.731379+0000", 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.133e-05 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.774219+0000", 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042840784 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.537 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.774266+0000", 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6062e-05 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:49.774302+0000", 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5957999999999997e-05 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.538 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:256482 11.e 11:7709f0b1:::benchmark_data_smithi079_199625_object127604:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e420)", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:31:39.917789+0000", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 42.497940890000002, 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093585408999999994, 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 256482 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:22.539 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917789+0000", 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917789+0000", 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917788+0000", 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:22.540 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917793+0000", 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.417e-06 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917803+0000", 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8849999999999994e-06 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:22.541 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917809+0000", 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0839999999999997e-06 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.917981+0000", 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017214199999999999 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.918086+0000", 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000104942 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.542 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,7", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.918184+0000", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7962999999999997e-05 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.923420+0000", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0052358539999999999 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:39.923427+0000", 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2749999999999998e-06 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.543 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:40.011168+0000", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.087740769999999996 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:40.011336+0000", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016789600000000001 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:31:40.011375+0000", 2024-09-17T08:32:22.544 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8649999999999998e-05 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:22.545 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:23.159 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-17T08:32:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:23.408 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.163509199 +0000 UTC m=+0.346636330 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:32:23.408 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.297724876 +0000 UTC m=+0.480852008 container create 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T08:32:23.657 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:23.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-mon[33187]: pgmap v1014: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-mon[33187]: 5.5 deep-scrub starts 2024-09-17T08:32:23.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-mon[33187]: 5.5 deep-scrub ok 2024-09-17T08:32:23.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.500449438 +0000 UTC m=+0.683576574 container init 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-17T08:32:23.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.506219359 +0000 UTC m=+0.689346491 container start 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-17T08:32:23.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.600387471 +0000 UTC m=+0.783514616 container attach 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T08:32:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[40649]: pgmap v1014: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:23.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[40649]: 5.5 deep-scrub starts 2024-09-17T08:32:23.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[40649]: 5.5 deep-scrub ok 2024-09-17T08:32:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[29565]: pgmap v1014: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[29565]: 5.5 deep-scrub starts 2024-09-17T08:32:23.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:23 smithi079 ceph-mon[29565]: 5.5 deep-scrub ok 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-09-17T08:32:24.224 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-09-17T08:32:24.225 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T08:32:24.225 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T08:32:24.225 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate[165595]: --> ceph-volume raw activate successful for osd ID: 6 2024-09-17T08:32:24.225 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 bash[165538]: --> ceph-volume raw activate successful for osd ID: 6 2024-09-17T08:32:24.225 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:23 smithi173 podman[165538]: 2024-09-17 08:32:23.923405666 +0000 UTC m=+1.106532843 container died 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b) 2024-09-17T08:32:24.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:24.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:24.926 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:25.223 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:24 smithi173 podman[165777]: 2024-09-17 08:32:24.926133048 +0000 UTC m=+1.046380244 container remove 5e58ff2d918fea2a5cdc9a7dac58a6619264ea2347656526de4fc8742f308e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T08:32:25.710 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:25 smithi173 ceph-mon[33187]: pgmap v1015: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:25.710 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:25 smithi173 ceph-mon[33187]: 2.18 scrub starts 2024-09-17T08:32:25.711 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:25 smithi173 ceph-mon[33187]: 2.18 scrub ok 2024-09-17T08:32:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[40649]: pgmap v1015: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[40649]: 2.18 scrub starts 2024-09-17T08:32:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[40649]: 2.18 scrub ok 2024-09-17T08:32:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[29565]: pgmap v1015: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[29565]: 2.18 scrub starts 2024-09-17T08:32:25.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:25 smithi079 ceph-mon[29565]: 2.18 scrub ok 2024-09-17T08:32:25.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:25 smithi173 podman[165907]: 2024-09-17 08:32:25.712970123 +0000 UTC m=+0.247027870 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:32:25.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:25 smithi173 podman[165907]: 2024-09-17 08:32:25.841168481 +0000 UTC m=+0.375226206 container create 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b) 2024-09-17T08:32:26.171 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:26.171 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:26.171 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:32:26.171 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:26.395 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 podman[165907]: 2024-09-17 08:32:26.286963023 +0000 UTC m=+0.821020767 container init 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:32:26.395 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 podman[165907]: 2024-09-17 08:32:26.2922129 +0000 UTC m=+0.826270625 container start 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-17T08:32:26.395 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 ceph-osd[165987]: -- 172.21.15.173:0/3354100016 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556332b651e0 con 0x5563338e4400 2024-09-17T08:32:26.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:32:26.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:26 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:32:26.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:26 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:32:26.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 bash[165907]: 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 2024-09-17T08:32:26.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 systemd[1]: Started Ceph osd.6 for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:32:26.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:26 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:32:26.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:26 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:32:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:26 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:32:26.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:26 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:32:27.008 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:32:26.829+0000 7f73e7c4e640 -1 Falling back to public interface 2024-09-17T08:32:27.032 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-17T08:32:27.139 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[40649]: pgmap v1016: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[40649]: 5.e deep-scrub starts 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[40649]: 5.e deep-scrub ok 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[29565]: pgmap v1016: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[29565]: 5.e deep-scrub starts 2024-09-17T08:32:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:27 smithi079 ceph-mon[29565]: 5.e deep-scrub ok 2024-09-17T08:32:27.972 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:27.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:27 smithi173 ceph-mon[33187]: pgmap v1016: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:27 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:32:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:27 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:32:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:27 smithi173 ceph-mon[33187]: 5.e deep-scrub starts 2024-09-17T08:32:27.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:27 smithi173 ceph-mon[33187]: 5.e deep-scrub ok 2024-09-17T08:32:28.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:28 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:32:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:28 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:32:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:28 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:32:28.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:28 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:32:29.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:32:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:32:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:32:29.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:32:29.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:32:29.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:32:29.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:32:29.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:28 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:32:29.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: pgmap v1017: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:29.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:29 smithi173 ceph-mon[33187]: osdmap e432: 8 total, 7 up, 7 in 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:no valid command found; 10 closest matches: 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:0 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:1 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:2 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:abort 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:assert 2024-09-17T08:32:30.069 INFO:teuthology.orchestra.run.smithi173.stderr:bluefs debug_inject_read_zeros 2024-09-17T08:32:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:bluefs files list 2024-09-17T08:32:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:bluefs stats 2024-09-17T08:32:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:bluestore allocator dump block 2024-09-17T08:32:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:bluestore allocator fragmentation block 2024-09-17T08:32:30.070 INFO:teuthology.orchestra.run.smithi173.stderr:admin_socket: invalid command 2024-09-17T08:32:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: pgmap v1017: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:32:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[40649]: osdmap e432: 8 total, 7 up, 7 in 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: pgmap v1017: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 95/582 objects degraded (16.323%) 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:32:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:29 smithi079 ceph-mon[29565]: osdmap e432: 8 total, 7 up, 7 in 2024-09-17T08:32:30.473 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:32:30.147+0000 7f73e7c4e640 -1 osd.6 424 log_to_monitors true 2024-09-17T08:32:30.881 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:32:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:32:30.660+0000 7f73db9f0640 -1 osd.6 424 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T08:32:30.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:30 smithi173 ceph-mon[33187]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:30.881 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:30 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:30.949 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T08:32:30.950 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-17T08:32:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:30 smithi079 ceph-mon[29565]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:31.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:30 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:30 smithi079 ceph-mon[40649]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:31.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:30 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T08:32:31.204 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:31.204 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:31.204 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:32:31.204 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:32:31.204 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: pgmap v1019: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: 5.0 deep-scrub starts 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: 5.0 deep-scrub ok 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: osdmap e433: 8 total, 7 up, 7 in 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:31.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:31 smithi173 ceph-mon[33187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:32.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: pgmap v1019: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:32.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:32:32.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:32:32.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: 5.0 deep-scrub starts 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: 5.0 deep-scrub ok 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: osdmap e433: 8 total, 7 up, 7 in 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[40649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: pgmap v1019: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: 5.0 deep-scrub starts 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: 5.0 deep-scrub ok 2024-09-17T08:32:32.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T08:32:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: osdmap e433: 8 total, 7 up, 7 in 2024-09-17T08:32:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: from='osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:32.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:31 smithi079 ceph-mon[29565]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-17T08:32:32.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-17T08:32:32.375 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:32.769 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:32 smithi173 ceph-mon[33187]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:32:32.769 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:32 smithi173 ceph-mon[33187]: osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064] boot 2024-09-17T08:32:32.769 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:32 smithi173 ceph-mon[33187]: osdmap e434: 8 total, 8 up, 7 in 2024-09-17T08:32:32.769 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:32 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:32.769 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:32 smithi173 ceph-mon[33187]: osdmap e435: 8 total, 8 up, 7 in 2024-09-17T08:32:33.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[40649]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[40649]: osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064] boot 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[40649]: osdmap e434: 8 total, 8 up, 7 in 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[40649]: osdmap e435: 8 total, 8 up, 7 in 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[29565]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[29565]: osd.6 [v2:172.21.15.173:6816/2093386064,v1:172.21.15.173:6817/2093386064] boot 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[29565]: osdmap e434: 8 total, 8 up, 7 in 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T08:32:33.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:32 smithi079 ceph-mon[29565]: osdmap e435: 8 total, 8 up, 7 in 2024-09-17T08:32:33.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:33.788 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: pgmap v1022: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:33.788 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:33.789 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:34.045 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:32:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:32:34.045 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238266 9.9 e279/221 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head v 279'14961, mlcod=279'14957)", 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 555.07823247700003, 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.074654730000000002, 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965478+0000", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965477+0000", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.965486+0000", 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4800000000000001e-06 2024-09-17T08:32:34.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033392+0000", 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.067906525999999995 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033402+0000", 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0030999999999999e-05 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033429+0000", 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6608e-05 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033446+0000", 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6991e-05 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040094+0000", 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0066476039999999997 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040133+0000", 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9164e-05 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:116954 10.9 e315/287 10:95a59d3e:::benchmark_data_smithi079_172150_object116953:head v 315'7258, mlcod=315'7255)", 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 419.83432069700001, 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.027391598, 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.050 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209390+0000", 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.051 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209389+0000", 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209455+0000", 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6304999999999996e-05 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209467+0000", 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1497e-05 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209472+0000", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.733e-06 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209511+0000", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8862999999999997e-05 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.209524+0000", 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.294e-05 2024-09-17T08:32:34.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236634+0000", 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.027110026999999998 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236781+0000", 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000147019 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:116958 10.9 e315/287 10:925c62cc:::benchmark_data_smithi079_172150_object116957:head v 315'7259, mlcod=315'7255)", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 419.83011499200001, 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023238808, 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213595+0000", 2024-09-17T08:32:34.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213594+0000", 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213696+0000", 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000101399 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213710+0000", 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4599e-05 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.213717+0000", 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9349999999999997e-06 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.216068+0000", 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0023509519999999999 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.216118+0000", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9301000000000001e-05 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236785+0000", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020667723999999998 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:25:34.236834+0000", 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8998000000000001e-05 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:159938 10.3 e325/287 10:cc1b825c:::benchmark_data_smithi079_172150_object27406:head v 325'10006, mlcod=325'10004)", 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 382.79559700800002, 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020320609, 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.059 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248113+0000", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248119+0000", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6669999999999999e-06 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248130+0000", 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1043000000000001e-05 2024-09-17T08:32:34.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248136+0000", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0839999999999997e-06 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248255+0000", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000118835 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.248275+0000", 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9669000000000001e-05 2024-09-17T08:32:34.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.268393+0000", 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020117939000000001 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:11.268434+0000", 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1230000000000003e-05 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:182969 10.9 e328/287 10:9fd709d5:::benchmark_data_smithi079_172150_object50437:head v 328'11344, mlcod=328'11342)", 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 362.83481583700001, 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020290156, 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208895+0000", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208893+0000", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208902+0000", 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4929999999999998e-06 2024-09-17T08:32:34.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208921+0000", 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9609000000000001e-05 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.208929+0000", 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5039999999999996e-06 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.209027+0000", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8041999999999994e-05 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.209047+0000", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9690999999999998e-05 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.229119+0000", 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020072494999999999 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.229185+0000", 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5562999999999996e-05 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:182971 10.9 e328/287 10:9d474694:::benchmark_data_smithi079_172150_object50439:head v 328'11345, mlcod=328'11342)", 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 362.83205643899998, 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022397592000000001, 2024-09-17T08:32:34.068 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211654+0000", 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211653+0000", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211661+0000", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.619e-06 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211678+0000", 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6467e-05 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.211685+0000", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4309999999999997e-06 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.224431+0000", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012746061 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.225375+0000", 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00094351000000000005 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.233968+0000", 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0085933899999999994 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:31.234052+0000", 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3244000000000005e-05 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:187094 10.7 e329/287 10:ea312feb:::benchmark_data_smithi079_172150_object54562:head v 329'11879, mlcod=328'11877)", 2024-09-17T08:32:34.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 359.26163878900002, 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.022168162000000002, 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.073 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782072+0000", 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782071+0000", 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782079+0000", 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1540000000000002e-06 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782095+0000", 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6490999999999998e-05 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782103+0000", 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2910000000000002e-06 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782256+0000", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015338400000000001 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.782276+0000", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9857999999999999e-05 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.804124+0000", 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.021847833000000001 2024-09-17T08:32:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:34.804240+0000", 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000116125 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189324 10.9 e330/287 10:9d7ba2fc:::benchmark_data_smithi079_172150_object56792:head v 330'11742, mlcod=329'11739)", 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 357.229554538, 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037981067, 2024-09-17T08:32:34.077 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814156+0000", 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814155+0000", 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814162+0000", 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.233e-06 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814172+0000", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0161e-05 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814178+0000", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0419999999999999e-06 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814210+0000", 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1576999999999999e-05 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814222+0000", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2439999999999999e-05 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.852112+0000", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037889277999999998 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.852137+0000", 2024-09-17T08:32:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5248e-05 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189320 10.a e330/287 10:510c5709:::benchmark_data_smithi079_172150_object56788:head v 330'11641, mlcod=329'11638)", 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 357.22882241999997, 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.040285575999999997, 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814888+0000", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814887+0000", 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.814893+0000", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2989999999999996e-06 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844966+0000", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.030073150999999999 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844982+0000", 2024-09-17T08:32:34.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5826000000000001e-05 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845125+0000", 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014297100000000001 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845146+0000", 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1331999999999998e-05 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.855122+0000", 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0099756859999999992 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.855174+0000", 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2009999999999998e-05 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189327 10.7 e330/287 10:eda6383a:::benchmark_data_smithi079_172150_object56795:head v 330'12005, mlcod=329'12003)", 2024-09-17T08:32:34.086 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 357.22863010600003, 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.044942533999999999, 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:32:34.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815080+0000", 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.815084+0000", 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6940000000000001e-06 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844976+0000", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.029892245000000001 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.844988+0000", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.241e-05 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845185+0000", 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019688299999999999 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845205+0000", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9544000000000001e-05 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.859975+0000", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.014770471 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.860023+0000", 2024-09-17T08:32:34.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7463000000000003e-05 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189339 10.3 e330/287 10:c4086931:::benchmark_data_smithi079_172150_object56807:head v 330'11923, mlcod=329'11921)", 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 357.218556412, 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.025840539999999999, 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825154+0000", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825153+0000", 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:34.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.825163+0000", 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9520000000000006e-06 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845010+0000", 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.019847265999999999 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845018+0000", 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2099999999999993e-06 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845124+0000", 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106198 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.845146+0000", 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1807e-05 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.850941+0000", 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0057952309999999996 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:36.850995+0000", 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3202999999999999e-05 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189616 10.6 e330/287 10:6e001a48:::benchmark_data_smithi079_172150_object57084:head v 330'11764, mlcod=330'11762)", 2024-09-17T08:32:34.095 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 356.96317713299999, 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.056560214999999997, 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080533+0000", 2024-09-17T08:32:34.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080532+0000", 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080540+0000", 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6850000000000001e-06 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.097 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080551+0000", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1029000000000001e-05 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080557+0000", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6080000000000001e-06 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080598+0000", 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1151e-05 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.098 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.080616+0000", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7897000000000002e-05 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.136110+0000", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.055493847999999998 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.137094+0000", 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098344300000000008 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.099 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:189620 10.a e330/287 10:537f11e0:::benchmark_data_smithi079_172150_object57088:head v 330'11661, mlcod=330'11659)", 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 356.962514559, 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.048894895000000001, 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:32:34.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081196+0000", 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081195+0000", 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.081201+0000", 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0260000000000004e-06 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118855+0000", 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.037654253999999998 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118871+0000", 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5211000000000001e-05 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118929+0000", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7923000000000001e-05 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.118951+0000", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2662000000000001e-05 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.129820+0000", 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.010868274000000001 2024-09-17T08:32:34.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:37.130091+0000", 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00027126199999999998 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:195080 10.6 e330/287 10:6651d373:::benchmark_data_smithi079_172150_object62548:head v 330'12097, mlcod=330'12094)", 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 352.44479930300002, 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.023128816999999999, 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.105 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598911+0000", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598910+0000", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598919+0000", 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4910000000000005e-06 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598929+0000", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8109999999999997e-06 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.598934+0000", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6699999999999999e-06 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.605407+0000", 2024-09-17T08:32:34.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0064728019999999997 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.605427+0000", 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.963e-05 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.621926+0000", 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.016499533 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:41.622040+0000", 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000113596 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15576.0:195597 10.9 e330/287 10:96153e48:::benchmark_data_smithi079_172150_object63065:head v 330'12114, mlcod=330'12112)", 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 352.01780597300001, 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.021097376000000001, 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025904+0000", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025903+0000", 2024-09-17T08:32:34.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025912+0000", 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8230000000000001e-06 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025931+0000", 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9462000000000001e-05 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.025939+0000", 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7190000000000001e-06 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.026286+0000", 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034661099999999998 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.026357+0000", 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1161999999999994e-05 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.046853+0000", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.020496209000000001 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:26:42.047002+0000", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000148651 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106703 11.a e382/359 11:5180121c:::benchmark_data_smithi079_199625_object106702:head v 382'6641, mlcod=382'6638)", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:32:34.113 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 189.60021238300001, 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29516395099999998, 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443498+0000", 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443497+0000", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443545+0000", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8253e-05 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443557+0000", 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2259000000000001e-05 2024-09-17T08:32:34.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443564+0000", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0740000000000004e-06 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443636+0000", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1633999999999999e-05 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.443654+0000", 2024-09-17T08:32:34.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7776000000000001e-05 2024-09-17T08:32:34.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.117 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.117 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446434+0000", 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027802529999999999 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738662+0000", 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29222788500000002 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106701 11.7 e382/359 11:e771ded9:::benchmark_data_smithi079_199625_object106700:head v 382'6643, mlcod=382'6641)", 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:32:34.118 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 189.599575896, 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29689527900000001, 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444135+0000", 2024-09-17T08:32:34.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444134+0000", 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444176+0000", 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2512000000000002e-05 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444187+0000", 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1239e-05 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444193+0000", 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0270000000000001e-06 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444240+0000", 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7003999999999999e-05 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.444258+0000", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7932999999999999e-05 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.740993+0000", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29673467399999998 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741030+0000", 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6742999999999997e-05 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.122 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106709 11.5 e382/359 11:aac0c29c:::benchmark_data_smithi079_199625_object106708:head v 382'6653, mlcod=382'6651)", 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 189.596036371, 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.300296497, 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:32:34.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447674+0000", 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447673+0000", 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.124 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447697+0000", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4358e-05 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738547+0000", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.29084915 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738570+0000", 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3396000000000001e-05 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738689+0000", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000119388 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738711+0000", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1302e-05 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.747842+0000", 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.009131676 2024-09-17T08:32:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.747971+0000", 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012826299999999999 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106708 11.8 e382/359 11:12156efd:::benchmark_data_smithi079_199625_object106707:head v 382'6620, mlcod=382'6618)", 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 189.59071656399999, 2024-09-17T08:32:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28997071800000002, 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452994+0000", 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.452993+0000", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.453044+0000", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1183000000000001e-05 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738567+0000", 2024-09-17T08:32:34.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28552359399999999 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738574+0000", 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6900000000000003e-06 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738619+0000", 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4552000000000003e-05 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738649+0000", 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0037000000000001e-05 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.742942+0000", 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004293056 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.742965+0000", 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2759e-05 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.25757.0:106751 11.7 e382/359 11:e1a8d295:::benchmark_data_smithi079_199625_object106750:head v 382'6644, mlcod=382'6641)", 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 189.548967551, 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.24684335199999999, 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:32:34.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494743+0000", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494741+0000", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.494797+0000", 2024-09-17T08:32:34.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6169999999999999e-05 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738553+0000", 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.243755797 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738566+0000", 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2909e-05 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:34.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738738+0000", 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000172225 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.738757+0000", 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8749999999999998e-05 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741555+0000", 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027980050000000001 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:34.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.741586+0000", 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1399000000000002e-05 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:34.136 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: pgmap v1022: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:34.137 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: pgmap v1022: 142 pgs: 41 active+undersized, 22 active+undersized+degraded, 79 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 95/582 objects degraded (16.323%) 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:34.138 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:34.915 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-17T08:32:34.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:34 smithi173 ceph-mon[33187]: pgmap v1024: 142 pgs: 28 active+undersized, 13 active+undersized+degraded, 101 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s; 60/582 objects degraded (10.309%) 2024-09-17T08:32:34.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:34 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:32:34.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:34 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:32:34.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:34 smithi173 ceph-mon[33187]: 6.18 scrub starts 2024-09-17T08:32:34.938 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:34 smithi173 ceph-mon[33187]: 6.18 scrub ok 2024-09-17T08:32:35.114 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[29565]: pgmap v1024: 142 pgs: 28 active+undersized, 13 active+undersized+degraded, 101 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s; 60/582 objects degraded (10.309%) 2024-09-17T08:32:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:32:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:32:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[29565]: 6.18 scrub starts 2024-09-17T08:32:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[29565]: 6.18 scrub ok 2024-09-17T08:32:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[40649]: pgmap v1024: 142 pgs: 28 active+undersized, 13 active+undersized+degraded, 101 active+clean; 320 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 1006 B/s rd, 0 op/s; 60/582 objects degraded (10.309%) 2024-09-17T08:32:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:32:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:32:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[40649]: 6.18 scrub starts 2024-09-17T08:32:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:34 smithi079 ceph-mon[40649]: 6.18 scrub ok 2024-09-17T08:32:35.951 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:32:35.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:35 smithi173 ceph-mon[33187]: 2.14 deep-scrub starts 2024-09-17T08:32:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:35 smithi173 ceph-mon[33187]: 2.14 deep-scrub ok 2024-09-17T08:32:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:35 smithi173 ceph-mon[33187]: Health check update: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:35 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:32:35.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:35 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:32:36.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[29565]: 2.14 deep-scrub starts 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[29565]: 2.14 deep-scrub ok 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[29565]: Health check update: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[40649]: 2.14 deep-scrub starts 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[40649]: 2.14 deep-scrub ok 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[40649]: Health check update: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded (PG_DEGRADED) 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:32:36.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:35 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:32:36.862 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:36 smithi173 ceph-mon[33187]: pgmap v1025: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:36.862 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:36 smithi173 ceph-mon[33187]: 4.9 scrub starts 2024-09-17T08:32:36.862 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:36 smithi173 ceph-mon[33187]: 4.9 scrub ok 2024-09-17T08:32:36.916 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:36.916 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:32:36.916 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:37.092 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[40649]: pgmap v1025: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[40649]: 4.9 scrub starts 2024-09-17T08:32:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[40649]: 4.9 scrub ok 2024-09-17T08:32:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[29565]: pgmap v1025: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[29565]: 4.9 scrub starts 2024-09-17T08:32:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:36 smithi079 ceph-mon[29565]: 4.9 scrub ok 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded) 2024-09-17T08:32:37.784 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:37 smithi173 ceph-mon[33187]: Cluster is now healthy 2024-09-17T08:32:37.805 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-17T08:32:38.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:32:38.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded) 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[29565]: Cluster is now healthy 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 60/582 objects degraded (10.309%), 13 pgs degraded) 2024-09-17T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:37 smithi079 ceph-mon[40649]: Cluster is now healthy 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: pgmap v1026: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:32:38.814 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:38.815 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:38 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:39.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: pgmap v1026: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:32:39.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: pgmap v1026: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:32:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:38 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:32:39.344 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:32:39.344 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:39.557 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:39.947 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:39 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:32:39.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:39 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:32:39.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:39 smithi173 ceph-mon[33187]: osdmap e436: 8 total, 8 up, 7 in 2024-09-17T08:32:40.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:32:40.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:32:40.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[29565]: osdmap e436: 8 total, 8 up, 7 in 2024-09-17T08:32:40.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:32:40.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:32:40.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:39 smithi079 ceph-mon[40649]: osdmap e436: 8 total, 8 up, 7 in 2024-09-17T08:32:40.269 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T08:32:40.269 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-17T08:32:40.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd in 3 2024-09-17T08:32:40.482 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:40.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[40649]: pgmap v1028: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:40 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:40 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:40 smithi173 ceph-mon[33187]: pgmap v1028: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:40 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:32:40.870 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:40 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:32:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:32:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:32:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[29565]: pgmap v1028: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:32:40.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:40 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:32:41.763 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:41 smithi173 ceph-mon[33187]: 2.8 deep-scrub starts 2024-09-17T08:32:41.764 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:41 smithi173 ceph-mon[33187]: 2.8 deep-scrub ok 2024-09-17T08:32:41.772 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:32:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:32:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:32:41.773 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:41.956 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:41 smithi079 ceph-mon[40649]: 2.8 deep-scrub starts 2024-09-17T08:32:41.956 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:41 smithi079 ceph-mon[40649]: 2.8 deep-scrub ok 2024-09-17T08:32:41.956 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:41 smithi079 ceph-mon[29565]: 2.8 deep-scrub starts 2024-09-17T08:32:41.957 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:41 smithi079 ceph-mon[29565]: 2.8 deep-scrub ok 2024-09-17T08:32:42.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-17T08:32:42.646 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:42.707 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.3. 2024-09-17T08:32:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:32:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:32:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: 1.a scrub starts 2024-09-17T08:32:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: 1.a scrub ok 2024-09-17T08:32:42.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: pgmap v1029: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:42.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3787630849' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:42.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:42.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:32:42.546+0000 7f02ec2c3640 -1 mon.a@0(leader).osd e436 definitely_dead 0 2024-09-17T08:32:42.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: 1.a scrub starts 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: 1.a scrub ok 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: pgmap v1029: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3787630849' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:42.872 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:42 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: 1.a scrub starts 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: 1.a scrub ok 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: pgmap v1029: 142 pgs: 142 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3787630849' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:42.958 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:42 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["3"]}]: dispatch 2024-09-17T08:32:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:43.505 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-17T08:32:43.505 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-17T08:32:43.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pgp_num 11 2024-09-17T08:32:43.717 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:43.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[29565]: 6.3 scrub starts 2024-09-17T08:32:43.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[29565]: 6.3 scrub ok 2024-09-17T08:32:43.852 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["3"]}]': finished 2024-09-17T08:32:43.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[29565]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T08:32:43.853 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:43.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[40649]: 6.3 scrub starts 2024-09-17T08:32:43.853 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[40649]: 6.3 scrub ok 2024-09-17T08:32:43.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["3"]}]': finished 2024-09-17T08:32:43.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[40649]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T08:32:43.854 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:43.858 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-mon[33187]: 6.3 scrub starts 2024-09-17T08:32:43.858 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-mon[33187]: 6.3 scrub ok 2024-09-17T08:32:43.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["3"]}]': finished 2024-09-17T08:32:43.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-mon[33187]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T08:32:43.859 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:43.859 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238260 9.c e279/221 9:3ad2fbde:::benchmark_data_smithi079_143589_object107061:head v 279'15028, mlcod=279'15025)", 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.45922851299997, 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.55165437399999995, 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.423 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961997+0000", 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.961996+0000", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:18.962003+0000", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6880000000000002e-06 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030074+0000", 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.068071330999999999 2024-09-17T08:32:44.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030089+0000", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4934e-05 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030144+0000", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4675000000000001e-05 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030167+0000", 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.304e-05 2024-09-17T08:32:44.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513620+0000", 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48345298399999997 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513651+0000", 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1421000000000003e-05 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238262 9.c e279/221 9:3e3399a0:::benchmark_data_smithi079_143589_object107063:head v 279'15029, mlcod=279'15025)", 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.39110610399996, 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48366091900000002, 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030119+0000", 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030118+0000", 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030126+0000", 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7919999999999993e-06 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030136+0000", 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6800000000000005e-06 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030141+0000", 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9599999999999999e-06 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032835+0000", 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.002693955 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032854+0000", 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9167000000000001e-05 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513749+0000", 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48089472900000002 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513780+0000", 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1254000000000003e-05 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.433 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238269 9.c e279/221 9:300a75d2:::benchmark_data_smithi079_143589_object107070:head v 279'15030, mlcod=279'15026)", 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.39107930900002, 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48365520499999998, 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030146+0000", 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030150+0000", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.014e-06 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030155+0000", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2700000000000004e-06 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030159+0000", 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9009999999999996e-06 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.035254+0000", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005095558 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.035272+0000", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7187999999999999e-05 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513783+0000", 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47851158300000002 2024-09-17T08:32:44.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513801+0000", 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7895999999999999e-05 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238270 9.c e279/221 9:3be7327c:::benchmark_data_smithi079_143589_object107071:head v 279'15031, mlcod=279'15026)", 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:32:44.438 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.39106245899995, 2024-09-17T08:32:44.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48365735300000001, 2024-09-17T08:32:44.439 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.439 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.439 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030163+0000", 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:32:44.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030170+0000", 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.17e-06 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030175+0000", 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4439999999999999e-06 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030179+0000", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.031e-06 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.042670+0000", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.012490846 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.042689+0000", 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9309000000000001e-05 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.442 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513803+0000", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47111345199999999 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513820+0000", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7261e-05 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238263 9.9 e279/221 9:970a2f6b:::benchmark_data_smithi079_143589_object107064:head v 279'14960, mlcod=279'14957)", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.39078608099999, 2024-09-17T08:32:44.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48596704299999999, 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030439+0000", 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030444+0000", 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.062e-06 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030451+0000", 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3339999999999996e-06 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.030455+0000", 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0890000000000002e-06 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033010+0000", 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025551879999999999 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033030+0000", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9850000000000001e-05 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516378+0000", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48334795200000003 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516406+0000", 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8008e-05 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.447 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238266 9.9 e279/221 9:9bbe865d:::benchmark_data_smithi079_143589_object107067:head v 279'14961, mlcod=279'14957)", 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.38831704799998, 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48354262399999998, 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032908+0000", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032918+0000", 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0468999999999999e-05 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032924+0000", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6760000000000004e-06 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.032929+0000", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9010000000000002e-06 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040159+0000", 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0072303089999999999 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.040178+0000", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.836e-05 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516433+0000", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47625525400000002 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516451+0000", 2024-09-17T08:32:44.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7932999999999999e-05 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238273 9.1 e279/221 9:8dab1ad1:::benchmark_data_smithi079_143589_object107074:head v 279'14726, mlcod=279'14724)", 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.38746007700001, 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.4826628, 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033765+0000", 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033764+0000", 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033771+0000", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3969999999999999e-06 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033781+0000", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0417999999999999e-05 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.033787+0000", 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6200000000000004e-06 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.037324+0000", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0035372799999999999 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.037343+0000", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9313999999999998e-05 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516409+0000", 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47906535 2024-09-17T08:32:44.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516428+0000", 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9205e-05 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238276 9.1 e279/221 9:8a6a764e:::benchmark_data_smithi079_143589_object107077:head v 279'14727, mlcod=279'14724)", 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.38501925000003, 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48026569699999999, 2024-09-17T08:32:44.456 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036206+0000", 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036205+0000", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036212+0000", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.55e-06 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036222+0000", 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8549999999999997e-06 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.036227+0000", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7640000000000002e-06 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.039923+0000", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0036956379999999998 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.039940+0000", 2024-09-17T08:32:44.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7447000000000001e-05 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516453+0000", 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47651224599999997 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516472+0000", 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8930999999999999e-05 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238283 9.e 9:7b3bdc8a:::benchmark_data_smithi079_143589_object107084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.37738656099998, 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47686604199999999, 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238283 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043839+0000", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043837+0000", 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:44.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043843+0000", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9900000000000002e-06 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043853+0000", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5910000000000002e-06 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043859+0000", 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1680000000000001e-06 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043909+0000", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0373e-05 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044031+0000", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000121384 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,4", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513503+0000", 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46947200700000002 2024-09-17T08:32:44.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516350+0000", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028466569999999998 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.518549+0000", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0021995579999999999 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520469+0000", 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001919511 2024-09-17T08:32:44.465 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520515+0000", 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6544e-05 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520705+0000", 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018956499999999999 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.466 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238285 9.d 9:b4139a65:::benchmark_data_smithi079_143589_object107086:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.37733039700004, 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48801911199999998, 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238285 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:32:44.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043895+0000", 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043894+0000", 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043898+0000", 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8469999999999998e-06 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043908+0000", 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9709999999999999e-06 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043913+0000", 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4070000000000001e-06 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.043951+0000", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7672000000000001e-05 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044051+0000", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000100319 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513514+0000", 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46946270299999998 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522917+0000", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0094026289999999992 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531719+0000", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0088019359999999998 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531752+0000", 2024-09-17T08:32:44.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3612000000000003e-05 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531869+0000", 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011707800000000001 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.531914+0000", 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4614999999999999e-05 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238282 9.a e279/221 9:5c132a73:::benchmark_data_smithi079_143589_object107083:head v 279'14822, mlcod=279'14817)", 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.37721751699996, 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47837759200000002, 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044008+0000", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044007+0000", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044014+0000", 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6279999999999999e-06 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044022+0000", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2519999999999999e-06 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044027+0000", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5799999999999999e-06 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044064+0000", 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6637000000000002e-05 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044084+0000", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.969e-05 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522318+0000", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.478234569 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522385+0000", 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6926000000000003e-05 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.476 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238286 9.1 e279/221 9:88a162d9:::benchmark_data_smithi079_143589_object107087:head v 279'14728, mlcod=279'14724)", 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.37697874399998, 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.47235775800000002, 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:32:44.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044246+0000", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.044252+0000", 2024-09-17T08:32:44.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0109999999999997e-06 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513439+0000", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46918737900000002 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513457+0000", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8286000000000001e-05 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513649+0000", 2024-09-17T08:32:44.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00019104500000000001 2024-09-17T08:32:44.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513678+0000", 2024-09-17T08:32:44.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9445e-05 2024-09-17T08:32:44.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516589+0000", 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029111139999999998 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.516604+0000", 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5124e-05 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.15327.0:238289 9.7 9:e13aa397:::benchmark_data_smithi079_143589_object107090:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:32:44.482 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.37437661000001, 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.48274150100000002, 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.15327", 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 238289 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046849+0000", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046848+0000", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.046853+0000", 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5659999999999997e-06 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513663+0000", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.466810365 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513676+0000", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2177e-05 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514277+0000", 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00060180900000000002 2024-09-17T08:32:44.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514428+0000", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015038000000000001 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3,5", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.514561+0000", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013314300000000001 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517577+0000", 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030164469999999998 2024-09-17T08:32:44.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.520600+0000", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0030229660000000002 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529514+0000", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0089139070000000004 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529552+0000", 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7676000000000003e-05 2024-09-17T08:32:44.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.529590+0000", 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8146999999999999e-05 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.15327.0:238291 9.a e279/221 9:5e6ad04a:::benchmark_data_smithi079_143589_object107092:head v 279'14823, mlcod=279'14820)", 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 565.36854969000001, 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.46976095600000001, 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.488 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052676+0000", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052675+0000", 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:44.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.052683+0000", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1140000000000006e-06 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513440+0000", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.460757156 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.513458+0000", 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7580000000000001e-05 2024-09-17T08:32:44.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517250+0000", 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037925939999999998 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.517269+0000", 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8785999999999999e-05 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522391+0000", 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0051220800000000002 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:23:19.522436+0000", 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5509000000000001e-05 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106709 11.5 11:aac0c29c:::benchmark_data_smithi079_199625_object106708:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:32:44.492 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.974375103, 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.30129817399999997, 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106709 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446850+0000", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446848+0000", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446923+0000", 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4624999999999998e-05 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446938+0000", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5270000000000001e-05 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.446946+0000", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9440000000000005e-06 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447254+0000", 2024-09-17T08:32:44.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000307552 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447414+0000", 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000159918 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.447576+0000", 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016261800000000001 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.449833+0000", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022572059999999999 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.449866+0000", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2344000000000001e-05 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748051+0000", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.298184966 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748102+0000", 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1711e-05 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748148+0000", 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5850000000000003e-05 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:106724 11.5 11:aca01d77:::benchmark_data_smithi079_199625_object106723:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e382)", 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 199.95797369499999, 2024-09-17T08:32:44.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28498498500000002, 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 106724 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463252+0000", 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463250+0000", 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463304+0000", 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4355999999999999e-05 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463318+0000", 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3997e-05 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463325+0000", 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8929999999999999e-06 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463377+0000", 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1746000000000002e-05 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463540+0000", 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016334100000000001 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 5,6", 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.463704+0000", 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000163974 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.466429+0000", 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0027248369999999999 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.466584+0000", 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015447700000000001 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748185+0000", 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28160126699999999 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748216+0000", 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0923999999999997e-05 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:29:24.748236+0000", 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0757000000000001e-05 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.504 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158194 11.f 11:f9a88cbf:::benchmark_data_smithi079_199625_object29316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.871673442, 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36185254900000002, 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158194 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.505 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549552+0000", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549550+0000", 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:44.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549560+0000", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0039e-05 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549574+0000", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4100000000000001e-05 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549581+0000", 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9720000000000003e-06 2024-09-17T08:32:44.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549628+0000", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6598999999999999e-05 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549787+0000", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015902200000000001 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 1,2", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.549918+0000", 2024-09-17T08:32:44.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013126199999999999 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552370+0000", 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0024517179999999999 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.561759+0000", 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0093890620000000001 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911298+0000", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.34953903800000002 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911357+0000", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9194000000000003e-05 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.911404+0000", 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7009999999999999e-05 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.510 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158196 11.4 11:2df499b9:::benchmark_data_smithi079_199625_object29318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.86902628499999, 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36381823200000002, 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.511 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158196 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552199+0000", 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552198+0000", 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552203+0000", 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.31e-06 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.513 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552214+0000", 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1260000000000001e-05 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552221+0000", 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7410000000000003e-06 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552271+0000", 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9874999999999999e-05 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.514 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552393+0000", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000122748 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.552521+0000", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000127143 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557360+0000", 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.004838869 2024-09-17T08:32:44.515 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557497+0000", 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000137899 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.915929+0000", 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.358431267 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.516 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.915977+0000", 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8420000000000001e-05 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.916017+0000", 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0068000000000003e-05 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158197 11.4 11:2749ddc7:::benchmark_data_smithi079_199625_object29319:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:44.517 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.866787773, 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.36599574699999998, 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158197 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:32:44.518 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554437+0000", 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554436+0000", 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.519 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554442+0000", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6949999999999997e-06 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554455+0000", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3114e-05 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554462+0000", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.764e-06 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.520 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.554856+0000", 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000394239 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.555013+0000", 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015731600000000001 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.555124+0000", 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011102799999999999 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.521 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.557638+0000", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025140509999999998 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.558070+0000", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00043175 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920349+0000", 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.362278875 2024-09-17T08:32:44.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920400+0000", 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0889e-05 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920433+0000", 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3420999999999997e-05 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.523 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.25757.0:158230 11.4 11:2ed1e3f1:::benchmark_data_smithi079_199625_object29352:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 153.84197076000001, 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.341273365, 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.25757", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 158230 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:32:44.524 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579254+0000", 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:32:44.525 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579257+0000", 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.799e-06 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579268+0000", 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0529e-05 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579273+0000", 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1290000000000004e-06 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.526 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579316+0000", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3248999999999999e-05 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579426+0000", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000109676 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 2,6", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.579502+0000", 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6363000000000001e-05 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.527 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.581538+0000", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020359100000000002 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.581655+0000", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011646600000000001 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920463+0000", 2024-09-17T08:32:44.528 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.33880848200000002 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920493+0000", 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0063e-05 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:30:10.920528+0000", 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4613999999999998e-05 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.529 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.530 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.530 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:32:44.530 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:32:44.530 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:32:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:32:44.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: pgmap v1032: 142 pgs: 14 peering, 128 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: 5.1b scrub starts 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[29565]: 5.1b scrub ok 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: pgmap v1032: 142 pgs: 14 peering, 128 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: 5.1b scrub starts 2024-09-17T08:32:44.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:44 smithi079 ceph-mon[40649]: 5.1b scrub ok 2024-09-17T08:32:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:32:44.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:32:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T08:32:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: pgmap v1032: 142 pgs: 14 peering, 128 active+clean; 420 KiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:32:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: 5.1b scrub starts 2024-09-17T08:32:44.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:44 smithi173 ceph-mon[33187]: 5.1b scrub ok 2024-09-17T08:32:45.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-17T08:32:45.658 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:32:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:32:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[40649]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T08:32:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:32:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:32:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:45 smithi079 ceph-mon[29565]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T08:32:45.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:45 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:32:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:45 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:32:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:45 smithi173 ceph-mon[33187]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T08:32:46.730 INFO:teuthology.orchestra.run.smithi079.stderr:set pool 1 pgp_num to 11 2024-09-17T08:32:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:32:46.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: pgmap v1034: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 209 B/s, 10 objects/s recovering 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1277875453' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:32:46.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: pgmap v1034: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 209 B/s, 10 objects/s recovering 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1277875453' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:32:46.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:46 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: pgmap v1034: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 209 B/s, 10 objects/s recovering 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1277875453' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-17T08:32:47.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:32:47.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:46 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:32:47.655 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:47.656 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:32:47.656 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-17T08:32:47.656 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: 6.1c scrub starts 2024-09-17T08:32:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: 6.1c scrub ok 2024-09-17T08:32:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:32:47.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[29565]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: 6.1c scrub starts 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: 6.1c scrub ok 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-17T08:32:47.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:47 smithi079 ceph-mon[40649]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T08:32:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: 6.1c scrub starts 2024-09-17T08:32:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: 6.1c scrub ok 2024-09-17T08:32:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:32:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:32:48.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-17T08:32:48.126 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:47 smithi173 ceph-mon[33187]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T08:32:48.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-17T08:32:48.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T08:32:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 4.4 scrub starts 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 4.4 scrub ok 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: pgmap v1037: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 312 B/s, 15 objects/s recovering 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:49.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 4.5 deep-scrub starts 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[40649]: 4.5 deep-scrub ok 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 4.4 scrub starts 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 4.4 scrub ok 2024-09-17T08:32:49.045 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: pgmap v1037: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 312 B/s, 15 objects/s recovering 2024-09-17T08:32:49.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:49.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:49.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 4.5 deep-scrub starts 2024-09-17T08:32:49.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:48 smithi079 ceph-mon[29565]: 4.5 deep-scrub ok 2024-09-17T08:32:49.144 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:32:49.144 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:32:49.144 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 4.4 scrub starts 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 4.4 scrub ok 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: pgmap v1037: 142 pgs: 2 active+undersized+remapped, 14 peering, 126 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 312 B/s, 15 objects/s recovering 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 4.5 deep-scrub starts 2024-09-17T08:32:49.145 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:48 smithi173 ceph-mon[33187]: 4.5 deep-scrub ok 2024-09-17T08:32:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 4.f deep-scrub starts 2024-09-17T08:32:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 4.f deep-scrub ok 2024-09-17T08:32:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 3.0 deep-scrub starts 2024-09-17T08:32:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 3.0 deep-scrub ok 2024-09-17T08:32:50.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 4.d deep-scrub starts 2024-09-17T08:32:50.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[29565]: 4.d deep-scrub ok 2024-09-17T08:32:50.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 4.f deep-scrub starts 2024-09-17T08:32:50.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 4.f deep-scrub ok 2024-09-17T08:32:50.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 3.0 deep-scrub starts 2024-09-17T08:32:50.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 3.0 deep-scrub ok 2024-09-17T08:32:50.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 4.d deep-scrub starts 2024-09-17T08:32:50.122 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:49 smithi079 ceph-mon[40649]: 4.d deep-scrub ok 2024-09-17T08:32:50.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 4.f deep-scrub starts 2024-09-17T08:32:50.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 4.f deep-scrub ok 2024-09-17T08:32:50.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 3.0 deep-scrub starts 2024-09-17T08:32:50.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 3.0 deep-scrub ok 2024-09-17T08:32:50.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 4.d deep-scrub starts 2024-09-17T08:32:50.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:49 smithi173 ceph-mon[33187]: 4.d deep-scrub ok 2024-09-17T08:32:50.281 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:50.763 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[29565]: pgmap v1038: 142 pgs: 2 active+undersized+remapped, 140 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 6.5 KiB/s rd, 36 KiB/s wr, 7 op/s; 123 KiB/s, 14 objects/s recovering 2024-09-17T08:32:50.763 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:32:50.763 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:32:50.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[40649]: pgmap v1038: 142 pgs: 2 active+undersized+remapped, 140 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 6.5 KiB/s rd, 36 KiB/s wr, 7 op/s; 123 KiB/s, 14 objects/s recovering 2024-09-17T08:32:50.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:32:50.764 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:50 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:32:50.769 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:50.769 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-17T08:32:50.769 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-17T08:32:50.769 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-17T08:32:50.769 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:50 smithi173 ceph-mon[33187]: pgmap v1038: 142 pgs: 2 active+undersized+remapped, 140 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 6.5 KiB/s rd, 36 KiB/s wr, 7 op/s; 123 KiB/s, 14 objects/s recovering 2024-09-17T08:32:51.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:50 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:32:51.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:50 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:32:52.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:32:52.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:32:52.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:32:52.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:32:52.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:51 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:32:52.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:32:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:32:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:32:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:32:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:32:52.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:51 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:32:52.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-17T08:32:53.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:32:53.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:32:53.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[29565]: pgmap v1039: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.7 KiB/s rd, 31 KiB/s wr, 5 op/s; 105 KiB/s, 3 objects/s recovering 2024-09-17T08:32:53.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:32:53.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:32:53.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:52 smithi079 ceph-mon[40649]: pgmap v1039: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.7 KiB/s rd, 31 KiB/s wr, 5 op/s; 105 KiB/s, 3 objects/s recovering 2024-09-17T08:32:53.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:52 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:32:53.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:52 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:32:53.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:52 smithi173 ceph-mon[33187]: pgmap v1039: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.7 KiB/s rd, 31 KiB/s wr, 5 op/s; 105 KiB/s, 3 objects/s recovering 2024-09-17T08:32:53.321 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:32:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:32:53.328 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-17T08:32:53.328 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:32:53.330 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-17T08:32:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:32:53.333 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:53.339 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-17T08:32:53.340 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T08:32:53.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 5.4 scrub starts 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 5.4 scrub ok 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/791131058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:32:53.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:53 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:54.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 5.4 scrub starts 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 5.4 scrub ok 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/791131058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 5.4 scrub starts 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 5.4 scrub ok 2024-09-17T08:32:54.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:32:54.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:32:54.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:32:54.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:32:54.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/791131058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T08:32:54.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:53 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:32:54.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-17T08:32:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:32:55.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 5.13 deep-scrub starts 2024-09-17T08:32:55.043 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 5.13 deep-scrub ok 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: pgmap v1040: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 4 op/s; 92 KiB/s, 3 objects/s recovering 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 5.13 deep-scrub starts 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 5.13 deep-scrub ok 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:32:55.044 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:32:55.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:32:55.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:32:55.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: pgmap v1040: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 4 op/s; 92 KiB/s, 3 objects/s recovering 2024-09-17T08:32:55.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:54 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:55.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 5.13 deep-scrub starts 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 5.13 deep-scrub ok 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: pgmap v1040: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 4 op/s; 92 KiB/s, 3 objects/s recovering 2024-09-17T08:32:55.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:54 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:56.008 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:55 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:56.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:55 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:32:56.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:55 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:32:56.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:55 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213094 9.6 9:6c0b11e2:::benchmark_data_smithi079_143589_object81895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:32:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 598.47709246500006, 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.31323531599999999, 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213094 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337365+0000", 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337363+0000", 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337371+0000", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6359999999999994e-06 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337385+0000", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4183e-05 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337392+0000", 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4719999999999998e-06 2024-09-17T08:32:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337442+0000", 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9378000000000001e-05 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337568+0000", 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012602300000000001 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:32:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.337689+0000", 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012135599999999999 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341229+0000", 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003539979 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.345283+0000", 2024-09-17T08:32:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040540510000000004 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650515+0000", 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30523122800000002 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650560+0000", 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5611000000000001e-05 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650600+0000", 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0299000000000002e-05 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213100 9.6 9:65f8bbfc:::benchmark_data_smithi079_143589_object81901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 598.47323868800004, 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30948176700000002, 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213100 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341219+0000", 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341217+0000", 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341225+0000", 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.328e-06 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341239+0000", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4051e-05 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341247+0000", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3319999999999992e-06 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341301+0000", 2024-09-17T08:32:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3996000000000003e-05 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341442+0000", 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141328 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.341562+0000", 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011956 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.346195+0000", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046330629999999998 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.347443+0000", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001248292 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650634+0000", 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.30319073099999999 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650668+0000", 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4360999999999998e-05 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650701+0000", 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2305999999999999e-05 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.15327.0:213109 9.6 9:690187cb:::benchmark_data_smithi079_143589_object81910:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 598.46363614400002, 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29998592099999999, 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.15327", 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3807762441", 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 213109 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350821+0000", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350820+0000", 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350826+0000", 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5889999999999999e-06 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350840+0000", 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3474e-05 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350848+0000", 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9829999999999996e-06 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.350896+0000", 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8618000000000001e-05 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351004+0000", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108012 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1,5", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.351110+0000", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105951 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355185+0000", 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040743740000000004 2024-09-17T08:32:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.355199+0000", 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-05 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650732+0000", 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.29553320199999999 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650766+0000", 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4260000000000001e-05 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:22:58.650807+0000", 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0850999999999999e-05 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:158194 11.f e390/359 11:f9a88cbf:::benchmark_data_smithi079_199625_object29316:head v 390'9938, mlcod=390'9934)", 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:32:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.26401375899999, 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36035172500000001, 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550444+0000", 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550443+0000", 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550451+0000", 2024-09-17T08:32:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4999999999999999e-06 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550463+0000", 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1854e-05 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550470+0000", 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5899999999999996e-06 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550523+0000", 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3628999999999998e-05 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.550542+0000", 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8199e-05 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910751+0000", 2024-09-17T08:32:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36020933599999999 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910796+0000", 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589999999999998e-05 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158198 11.2 11:452bec04:::benchmark_data_smithi079_199625_object29320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.261976877, 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.38156971499999998, 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158198 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552481+0000", 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552479+0000", 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552487+0000", 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8180000000000004e-06 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552500+0000", 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3468e-05 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552508+0000", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5490000000000001e-06 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552557+0000", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9351999999999998e-05 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552734+0000", 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177116 2024-09-17T08:32:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552848+0000", 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114298 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913036+0000", 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36018772500000001 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913068+0000", 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1971000000000003e-05 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.933979+0000", 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020910900999999999 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.934023+0000", 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3476000000000002e-05 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.934050+0000", 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7875e-05 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.25757.0:158196 11.4 e390/359 11:2df499b9:::benchmark_data_smithi079_199625_object29318:head v 390'10130, mlcod=390'10128)", 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.26184643900001, 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36284417299999999, 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552611+0000", 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552610+0000", 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552620+0000", 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4720000000000001e-06 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552628+0000", 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6510000000000001e-06 2024-09-17T08:32:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.552633+0000", 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0549999999999999e-06 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910783+0000", 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.358150621 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910809+0000", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6072999999999999e-05 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.915431+0000", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046219549999999996 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.915455+0000", 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4005000000000001e-05 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:158201 11.d 11:baf37dc6:::benchmark_data_smithi079_199625_object29323:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.26125333799999, 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.36441254099999998, 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158201 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553204+0000", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553203+0000", 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.553209+0000", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.356e-06 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910726+0000", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35751760900000001 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910743+0000", 2024-09-17T08:32:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6948000000000001e-05 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910798+0000", 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4549999999999998e-05 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910929+0000", 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013176099999999999 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4,6", 2024-09-17T08:32:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.911046+0000", 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116297 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917507+0000", 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064610680000000004 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917536+0000", 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8711e-05 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917542+0000", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.241e-06 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917581+0000", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9060000000000002e-05 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.917617+0000", 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5938e-05 2024-09-17T08:32:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:158198 11.2 e390/359 ondisk, result = 0)", 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.25700583099999, 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35561820999999999, 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:32:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557452+0000", 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557450+0000", 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557458+0000", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3479999999999998e-06 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910819+0000", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35336151999999998 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910827+0000", 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7500000000000003e-06 2024-09-17T08:32:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913061+0000", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022343430000000002 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913065+0000", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7079999999999999e-06 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913070+0000", 2024-09-17T08:32:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8459999999999999e-06 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.25757.0:158198 11.2 e390/359 ondisk, result = 0)", 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 166.256736762, 2024-09-17T08:32:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.355317826, 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557721+0000", 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557720+0000", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.557726+0000", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0460000000000002e-06 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910738+0000", 2024-09-17T08:32:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.35301196699999998 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.910749+0000", 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1367e-05 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913026+0000", 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022763359999999999 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913032+0000", 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:10.913039+0000", 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.545e-06 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:189643 11.2 11:48f748f6:::benchmark_data_smithi079_199625_object60765:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:32:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 137.871298653, 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.038635083000000001, 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 189643 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:32:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943159+0000", 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943158+0000", 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999995 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943161+0000", 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0850000000000001e-06 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943168+0000", 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7120000000000003e-06 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.943173+0000", 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.977e-06 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957497+0000", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014323989000000001 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957626+0000", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012899299999999999 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.957750+0000", 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124084 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.967557+0000", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098063520000000008 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.969091+0000", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015346940000000001 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981558+0000", 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012466761999999999 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981670+0000", 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011168 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:38.981794+0000", 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012425899999999999 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:190741 11.2 11:4543be5d:::benchmark_data_smithi079_199625_object61863:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 136.96201847399999, 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052378642000000003, 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 190741 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852439+0000", 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852431+0000", 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999919 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852444+0000", 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2499e-05 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852455+0000", 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0869999999999999e-05 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.852462+0000", 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1230000000000002e-06 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.862987+0000", 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010525223 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.863271+0000", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028371400000000001 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.863568+0000", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029748900000000001 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.866664+0000", 2024-09-17T08:32:56.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030955219999999999 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.867430+0000", 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076666199999999999 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904714+0000", 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.037283789999999997 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904766+0000", 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1990000000000002e-05 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:39.904818+0000", 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1706000000000003e-05 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.895 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194321 11.2 11:44f6beda:::benchmark_data_smithi079_199625_object65443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:42.740443+0000", 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 134.07401419000001, 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.039108465000000002, 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194321 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740443+0000", 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740443+0000", 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740442+0000", 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740448+0000", 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2659999999999998e-06 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740464+0000", 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.52e-05 2024-09-17T08:32:56.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.740470+0000", 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9500000000000004e-06 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.755104+0000", 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014633343 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.755276+0000", 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000172268 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.755379+0000", 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103168 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.764836+0000", 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094564760000000001 2024-09-17T08:32:56.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.765600+0000", 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076405500000000003 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.779465+0000", 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013865496999999999 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.779512+0000", 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6332999999999998e-05 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:42.779552+0000", 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0238000000000003e-05 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:194869 11.2 11:4144ac05:::benchmark_data_smithi079_199625_object65991:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e399)", 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:43.199484+0000", 2024-09-17T08:32:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 133.61497397599999, 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.036192706999999998, 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 194869 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199484+0000", 2024-09-17T08:32:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199484+0000", 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199482+0000", 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199488+0000", 2024-09-17T08:32:56.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.037e-06 2024-09-17T08:32:56.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199500+0000", 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.185e-05 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.199506+0000", 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3319999999999997e-06 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.201909+0000", 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002402965 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.202024+0000", 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114855 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.202129+0000", 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105282 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.205274+0000", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031450879999999999 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.216673+0000", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011398906 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.235584+0000", 2024-09-17T08:32:56.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018910218999999999 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.235634+0000", 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0219000000000001e-05 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:43.235676+0000", 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2536000000000001e-05 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:198762 11.2 11:40803471:::benchmark_data_smithi079_199625_object69884:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e400)", 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:46.351854+0000", 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.46260374600001, 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041102697000000001, 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 198762 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351854+0000", 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351854+0000", 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351853+0000", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351857+0000", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6040000000000003e-06 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351870+0000", 2024-09-17T08:32:56.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2462e-05 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.351876+0000", 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0750000000000003e-06 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.352302+0000", 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000426489 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.352407+0000", 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104722 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.352507+0000", 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9654999999999994e-05 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.364638+0000", 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012131404 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.379213+0000", 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014575065 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.392853+0000", 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013640172000000001 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.392914+0000", 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1121e-05 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.392957+0000", 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2039000000000002e-05 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:198770 11.2 11:461e8fd4:::benchmark_data_smithi079_199625_object69892:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e400)", 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:46.355367+0000", 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.459090366, 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.037719298999999998, 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 198770 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355367+0000", 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355367+0000", 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355366+0000", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999986 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355372+0000", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0680000000000002e-06 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355388+0000", 2024-09-17T08:32:56.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5551999999999999e-05 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.355395+0000", 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1770000000000001e-06 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.357222+0000", 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001827483 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.357513+0000", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029042299999999999 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.357858+0000", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034515300000000001 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.364646+0000", 2024-09-17T08:32:56.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067885580000000001 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.379737+0000", 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01509105 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.393007+0000", 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013269352 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.393048+0000", 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1652e-05 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.393087+0000", 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8074000000000002e-05 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:199091 11.2 11:4bdaea75:::benchmark_data_smithi079_199625_object70213:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e400)", 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:46.611473+0000", 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.20298464300001, 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1642691869999999, 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 199091 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611473+0000", 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611473+0000", 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611472+0000", 2024-09-17T08:32:56.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.999999 2024-09-17T08:32:56.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611477+0000", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6010000000000002e-06 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611489+0000", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1762e-05 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611496+0000", 2024-09-17T08:32:56.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0489999999999998e-06 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611577+0000", 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0955e-05 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611676+0000", 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8692999999999999e-05 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3", 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.611784+0000", 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010787500000000001 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.621148+0000", 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093646800000000002 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.639030+0000", 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017881844000000001 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.639043+0000", 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2381999999999999e-05 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:47.623732+0000", 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.984689178 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:47.623745+0000", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2748000000000001e-05 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.775676+0000", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.151931005 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.775742+0000", 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6507999999999994e-05 2024-09-17T08:32:56.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.25757.0:199119 11.2 11:4ae0d896:::benchmark_data_smithi079_199625_object70241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e401)", 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-17T08:30:46.673200+0000", 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.14125759300001, 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1192458740000002, 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.25757", 2024-09-17T08:32:56.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2738998101", 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 199119 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673200+0000", 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673200+0000", 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-17T08:32:56.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673198+0000", 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4294967295.9999981 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673206+0000", 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7130000000000005e-06 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673221+0000", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5274e-05 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673230+0000", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6230000000000003e-06 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673284+0000", 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4423999999999999e-05 2024-09-17T08:32:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:46.673295+0000", 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0869e-05 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:47.623769+0000", 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.95047405200000001 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-17T08:32:56.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:47.623777+0000", 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0409999999999998e-06 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.775972+0000", 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.152194961 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.776082+0000", 2024-09-17T08:32:56.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110547 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0,3,7", 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.776552+0000", 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047004099999999998 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.790356+0000", 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013803164999999999 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-17T08:32:56.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.791605+0000", 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012495430000000001 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.792219+0000", 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061403 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.792253+0000", 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.341e-05 2024-09-17T08:32:56.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.792335+0000", 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2293000000000006e-05 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-17T08:30:48.792446+0000", 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110968 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-17T08:32:56.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-17T08:32:56.941 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-17T08:32:57.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[29565]: pgmap v1041: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 4 op/s; 74 KiB/s, 2 objects/s recovering 2024-09-17T08:32:57.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:32:57.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:32:57.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[40649]: pgmap v1041: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 4 op/s; 74 KiB/s, 2 objects/s recovering 2024-09-17T08:32:57.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:32:57.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:56 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:32:57.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:56 smithi173 ceph-mon[33187]: pgmap v1041: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 4 op/s; 74 KiB/s, 2 objects/s recovering 2024-09-17T08:32:57.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:56 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:32:57.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:56 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:32:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:32:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:57 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:32:58.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:57 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:32:58.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:57 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:32:58.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:57 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:32:58.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:57 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:32:58.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-17T08:32:58.452 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:32:58.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:58 smithi173 ceph-mon[33187]: 2.c scrub starts 2024-09-17T08:32:58.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:58 smithi173 ceph-mon[33187]: 2.c scrub ok 2024-09-17T08:32:58.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:58 smithi173 ceph-mon[33187]: pgmap v1042: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 4 op/s; 67 KiB/s, 2 objects/s recovering 2024-09-17T08:32:58.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:58 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:32:58.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:58 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:32:59.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[40649]: 2.c scrub starts 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[40649]: 2.c scrub ok 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[40649]: pgmap v1042: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 4 op/s; 67 KiB/s, 2 objects/s recovering 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[29565]: 2.c scrub starts 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[29565]: 2.c scrub ok 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[29565]: pgmap v1042: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 4 op/s; 67 KiB/s, 2 objects/s recovering 2024-09-17T08:32:59.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:32:59.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:58 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:32:59.777 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: 5.16 deep-scrub starts 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: 5.16 deep-scrub ok 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T08:32:59.953 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:32:59 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:00.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: 5.16 deep-scrub starts 2024-09-17T08:33:00.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: 5.16 deep-scrub ok 2024-09-17T08:33:00.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:00.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:00.102 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: 5.16 deep-scrub starts 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: 5.16 deep-scrub ok 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T08:33:00.103 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:32:59 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:00.203 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:33:00.203 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:33:00.203 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-17T08:33:00.203 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:33:00.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-17T08:33:00.739 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:00.897 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 4.b scrub starts 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 4.b scrub ok 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: pgmap v1044: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:00.898 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:00 smithi173 ceph-mon[33187]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T08:33:00.918 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-17T08:33:01.117 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 4.b scrub starts 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 4.b scrub ok 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:33:01.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: pgmap v1044: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[40649]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 4.b scrub starts 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 4.b scrub ok 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: pgmap v1044: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:33:01.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3999363827' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:01.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:00 smithi079 ceph-mon[29565]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T08:33:01.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 4.18 deep-scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 4.18 deep-scrub ok 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 6.9 scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 6.9 scrub ok 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 4.18 deep-scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 4.18 deep-scrub ok 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 6.9 scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 6.9 scrub ok 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:33:01.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:01 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 4.18 deep-scrub starts 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 4.18 deep-scrub ok 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 6.9 scrub starts 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 6.9 scrub ok 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:33:01.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:01 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:33:02.789 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T08:33:02.882 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:33:02.883 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-17T08:33:02.883 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-17T08:33:02.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-17T08:33:02.883 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:33:02.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:02 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:33:02.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:02 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:33:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:02 smithi173 ceph-mon[33187]: pgmap v1046: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:02 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:03.112 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[40649]: pgmap v1046: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[29565]: pgmap v1046: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:03.113 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:02 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-17T08:33:03.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-17T08:33:03.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-17T08:33:03.652 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:03.757 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T08:33:03.787 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:04.075 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:04.076 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:03 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:33:04.139 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:33:04.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/227337777' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-17T08:33:04.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T08:33:04.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:04.140 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:03 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:04.876 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:04 smithi173 ceph-mon[33187]: 5.a scrub starts 2024-09-17T08:33:04.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:04 smithi173 ceph-mon[33187]: 5.a scrub ok 2024-09-17T08:33:04.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:04 smithi173 ceph-mon[33187]: pgmap v1048: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:04.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:04 smithi173 ceph-mon[33187]: 2.16 deep-scrub starts 2024-09-17T08:33:04.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:04 smithi173 ceph-mon[33187]: 2.16 deep-scrub ok 2024-09-17T08:33:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[40649]: 5.a scrub starts 2024-09-17T08:33:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[40649]: 5.a scrub ok 2024-09-17T08:33:05.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[40649]: pgmap v1048: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[40649]: 2.16 deep-scrub starts 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[40649]: 2.16 deep-scrub ok 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[29565]: 5.a scrub starts 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[29565]: 5.a scrub ok 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[29565]: pgmap v1048: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[29565]: 2.16 deep-scrub starts 2024-09-17T08:33:05.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:04 smithi079 ceph-mon[29565]: 2.16 deep-scrub ok 2024-09-17T08:33:05.421 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-17T08:33:05.421 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-17T08:33:05.421 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-17T08:33:05.421 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(5.3 REQUEST e435)", 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:32.910567+0000", 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.501940013000002, 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021868899999999999, 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910567+0000", 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.422 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910567+0000", 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910565+0000", 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.423 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910576+0000", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.115e-05 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910602+0000", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5446e-05 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910614+0000", 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.201e-05 2024-09-17T08:33:05.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910708+0000", 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3894999999999995e-05 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910728+0000", 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9816000000000001e-05 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.425 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.910786+0000", 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8428999999999999e-05 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "replica_scrub(pg: 5.3,from:0'0,to:426'10,epoch:435/434,start:5:c0000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:32.911750+0000", 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.500756916, 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000200423, 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.426 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911750+0000", 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911750+0000", 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.427 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911749+0000", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911760+0000", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1005999999999999e-05 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911782+0000", 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2166000000000001e-05 2024-09-17T08:33:05.428 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911796+0000", 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3623e-05 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911902+0000", 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010622499999999999 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.429 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911921+0000", 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8706e-05 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:32.911951+0000", 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.995e-05 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.430 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(5.3 epoch 435/434 rep_tid 277513 entries 435'11 (0'0) error 5:c609908c:::notify.5:head by client.24329.0:3085 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:33.104317+0000", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.308190727000003, 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045763599999999998, 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104317+0000", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.431 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104317+0000", 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104316+0000", 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104324+0000", 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4039999999999995e-06 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.432 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104339+0000", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4761e-05 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104352+0000", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3135e-05 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104421+0000", 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8732999999999994e-05 2024-09-17T08:33:05.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104774+0000", 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035343400000000002 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(5.9 epoch 435/434 rep_tid 159042 entries 435'11 (0'0) error 5:93e5b521:::notify.7:head by client.24329.0:3088 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:33.104373+0000", 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.308133873000003, 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049563400000000001, 2024-09-17T08:33:05.434 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104373+0000", 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104373+0000", 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.435 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104373+0000", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104451+0000", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8504999999999995e-05 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104467+0000", 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5614e-05 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.436 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104486+0000", 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9003000000000001e-05 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104551+0000", 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5229999999999997e-05 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104869+0000", 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031806800000000002 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(4.4 epoch 435/434 rep_tid 273297 entries 435'59 (0'0) error 4:22d26bf9:::data_loggenerations_metadata:head by client.24329.0:3086 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:33.104498+0000", 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.308009214000002, 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000520822, 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.438 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104498+0000", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104498+0000", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104498+0000", 2024-09-17T08:33:05.439 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104504+0000", 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1340000000000001e-06 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104515+0000", 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1338999999999999e-05 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.440 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104524+0000", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7250000000000003e-06 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104631+0000", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000106933 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.105019+0000", 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00038819100000000002 2024-09-17T08:33:05.441 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3087 5.1f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e435)", 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:33.104713+0000", 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.307794809000001, 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00093159700000000005, 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.442 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3087 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104713+0000", 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104713+0000", 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.443 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104711+0000", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104718+0000", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5830000000000003e-06 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104756+0000", 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8479000000000002e-05 2024-09-17T08:33:05.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104766+0000", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.2609999999999999e-06 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104817+0000", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1586999999999999e-05 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.104949+0000", 2024-09-17T08:33:05.445 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013212599999999999 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:33.105644+0000", 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00069477000000000004 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3096 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e435)", 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:38.104233+0000", 2024-09-17T08:33:05.446 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 27.308274065999999, 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045017400000000002, 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3096 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104233+0000", 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104233+0000", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104232+0000", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104240+0000", 2024-09-17T08:33:05.448 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.123e-06 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104254+0000", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3417e-05 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104291+0000", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7395999999999999e-05 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104430+0000", 2024-09-17T08:33:05.449 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013884100000000001 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104574+0000", 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014366699999999999 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.104683+0000", 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010987 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.450 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(5.9 RELEASE e436)", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:38.981997+0000", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 26.430510682000001, 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0014177510000000001, 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.981997+0000", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.451 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.981997+0000", 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.981996+0000", 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.981998+0000", 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.889e-06 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.452 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.982024+0000", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5919999999999999e-05 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.982051+0000", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6508000000000001e-05 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.982095+0000", 2024-09-17T08:33:05.453 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4338999999999997e-05 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.982105+0000", 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4190000000000008e-06 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.983375+0000", 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001270319 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.454 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.983386+0000", 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1131000000000001e-05 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:38.983414+0000", 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8425e-05 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3105 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-17T08:33:05.455 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:43.104389+0000", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 22.308118775000001, 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030470300000000003, 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3105 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104389+0000", 2024-09-17T08:33:05.456 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104389+0000", 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104388+0000", 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.457 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104393+0000", 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7720000000000002e-06 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104400+0000", 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8820000000000001e-06 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104410+0000", 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3759999999999997e-06 2024-09-17T08:33:05.458 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104510+0000", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9970999999999995e-05 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104599+0000", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8764000000000006e-05 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.104693+0000", 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4760999999999996e-05 2024-09-17T08:33:05.459 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPull(4.1d e438/437 cost 1049576)", 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:43.737981+0000", 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.674526107999998, 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024253299999999999, 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.460 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.737981+0000", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.737981+0000", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.737981+0000", 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.461 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.737988+0000", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4959999999999999e-06 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.737994+0000", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9220000000000002e-06 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.738003+0000", 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2530000000000004e-06 2024-09-17T08:33:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.738031+0000", 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8819e-05 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.738224+0000", 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000192354 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.463 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPull(3.2 e438/437 cost 1049576)", 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:43.760595+0000", 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.651912304, 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00027051799999999997, 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760595+0000", 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.464 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.465 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760595+0000", 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760594+0000", 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760602+0000", 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1579999999999999e-06 2024-09-17T08:33:05.466 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760611+0000", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3549999999999994e-06 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760616+0000", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4430000000000002e-06 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760648+0000", 2024-09-17T08:33:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.171e-05 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.760866+0000", 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021744599999999999 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPull(4.11 e438/437 cost 1049576)", 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:43.767981+0000", 2024-09-17T08:33:05.468 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.644526383999999, 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026535000000000003, 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.767981+0000", 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.767981+0000", 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.469 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.767980+0000", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999995 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.767988+0000", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9449999999999993e-06 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.767997+0000", 2024-09-17T08:33:05.470 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4910000000000005e-06 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.768001+0000", 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4630000000000003e-06 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.768033+0000", 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1912999999999998e-05 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.471 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.768246+0000", 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021312600000000001 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(4.11 REQUEST e438)", 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:43.986204+0000", 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 21.4263032, 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020964599999999999, 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.472 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986204+0000", 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986204+0000", 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.473 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986202+0000", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999981 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986215+0000", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2493e-05 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986250+0000", 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5630000000000003e-05 2024-09-17T08:33:05.474 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986271+0000", 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0871999999999999e-05 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986380+0000", 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000109072 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.475 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986389+0000", 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1509999999999993e-06 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:43.986414+0000", 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4411999999999999e-05 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3117 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e438)", 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:48.105097+0000", 2024-09-17T08:33:05.476 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 17.307410707999999, 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00038767800000000002, 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3117 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105097+0000", 2024-09-17T08:33:05.477 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105097+0000", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105095+0000", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999986 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105104+0000", 2024-09-17T08:33:05.478 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9029999999999993e-06 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105119+0000", 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4707e-05 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105133+0000", 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3726e-05 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105271+0000", 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000138648 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105373+0000", 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000101964 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.105484+0000", 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011099699999999999 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.480 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(osd.1.0:273299 3.0 e441/434 3:353ec8f9:::gateway.conf:head v 441'6, mlcod=0'0)", 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:48.504190+0000", 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 16.908316978999999, 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000504661, 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.481 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504190+0000", 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504190+0000", 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504189+0000", 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.482 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504198+0000", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4859999999999997e-06 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504212+0000", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3735e-05 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504218+0000", 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2829999999999998e-06 2024-09-17T08:33:05.483 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504259+0000", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1516e-05 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504281+0000", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1919000000000001e-05 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504658+0000", 2024-09-17T08:33:05.484 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00037683699999999998 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:48.504695+0000", 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6835000000000002e-05 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(6.b REQUEST e441)", 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:52.041376+0000", 2024-09-17T08:33:05.485 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 13.3711313, 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021816199999999999, 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041376+0000", 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041376+0000", 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.486 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041374+0000", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999976 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041387+0000", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.269e-05 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041421+0000", 2024-09-17T08:33:05.487 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4304999999999998e-05 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041439+0000", 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7893999999999998e-05 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.488 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041545+0000", 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00010629900000000001 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041558+0000", 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2995e-05 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:52.041594+0000", 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6152000000000002e-05 2024-09-17T08:33:05.489 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3126 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:53.106335+0000", 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 12.306172455, 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00053717999999999995, 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.490 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3126 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106335+0000", 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106335+0000", 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.491 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106332+0000", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.9999971 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106346+0000", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4384e-05 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106382+0000", 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5534999999999998e-05 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.492 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106401+0000", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8844000000000001e-05 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106570+0000", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000169226 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106759+0000", 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018902799999999999 2024-09-17T08:33:05.493 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:53.106872+0000", 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011307 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3135 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:32:58.107131+0000", 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 7.3053763370000002, 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00023894899999999999, 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.494 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3135 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107131+0000", 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.495 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107131+0000", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107130+0000", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107135+0000", 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0000000000000004e-06 2024-09-17T08:33:05.496 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107144+0000", 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8859999999999993e-06 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107152+0000", 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5499999999999995e-06 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.497 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107209+0000", 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7027999999999997e-05 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107280+0000", 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0752000000000003e-05 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:32:58.107370+0000", 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9779999999999998e-05 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.498 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(4.18 REQUEST e443)", 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:33:00.604285+0000", 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 4.8082221149999995, 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00098482999999999991, 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604285+0000", 2024-09-17T08:33:05.499 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604285+0000", 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604285+0000", 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967296 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.500 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604287+0000", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2709999999999999e-06 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604302+0000", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4647000000000001e-05 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604310+0000", 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1720000000000007e-06 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.501 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604361+0000", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1097000000000002e-05 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.604366+0000", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1270000000000002e-06 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.605219+0000", 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00085302499999999999 2024-09-17T08:33:05.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.605232+0000", 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2464e-05 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:00.605270+0000", 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8182000000000001e-05 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24329.0:3144 5.1f 5:f95f44c2:::notify.0:head [watch ping cookie 94434111863552 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e442)", 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-17T08:33:03.108175+0000", 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 2.3043323679999999, 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028880000000000003, 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24329", 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.79:0/2704626303", 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 3144 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.504 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108175+0000", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108175+0000", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108174+0000", 2024-09-17T08:33:05.505 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4294967295.999999 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108179+0000", 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.989e-06 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108187+0000", 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8590000000000005e-06 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-17T08:33:05.506 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108195+0000", 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1669999999999999e-06 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108291+0000", 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6250999999999997e-05 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108377+0000", 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5538999999999995e-05 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-17T08:33:05.507 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-17T08:33:03.108464+0000", 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6796e-05 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-17T08:33:05.508 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-17T08:33:06.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 1.8 deep-scrub starts 2024-09-17T08:33:06.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 1.8 deep-scrub ok 2024-09-17T08:33:06.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 2.f deep-scrub starts 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 2.f deep-scrub ok 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 1.8 deep-scrub starts 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 1.8 deep-scrub ok 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 2.f deep-scrub starts 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 2.f deep-scrub ok 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:33:06.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:05 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:33:06.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 1.8 deep-scrub starts 2024-09-17T08:33:06.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 1.8 deep-scrub ok 2024-09-17T08:33:06.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 2.f deep-scrub starts 2024-09-17T08:33:06.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 2.f deep-scrub ok 2024-09-17T08:33:06.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:33:06.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:05 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:33:06.810 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T08:33:07.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 4.15 deep-scrub starts 2024-09-17T08:33:07.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 4.15 deep-scrub ok 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: pgmap v1049: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4143070836' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 1.7 scrub starts 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[40649]: 1.7 scrub ok 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 4.15 deep-scrub starts 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 4.15 deep-scrub ok 2024-09-17T08:33:07.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: pgmap v1049: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4143070836' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 1.7 scrub starts 2024-09-17T08:33:07.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:06 smithi079 ceph-mon[29565]: 1.7 scrub ok 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 4.15 deep-scrub starts 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 4.15 deep-scrub ok 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: pgmap v1049: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4143070836' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 1.7 scrub starts 2024-09-17T08:33:07.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:06 smithi173 ceph-mon[33187]: 1.7 scrub ok 2024-09-17T08:33:07.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-17T08:33:07.753 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:08.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:08.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[29565]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:33:08.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:08.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:07 smithi079 ceph-mon[40649]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T08:33:08.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:08.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:08.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:33:08.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:33:08.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:08.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:07 smithi173 ceph-mon[33187]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T08:33:08.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: pgmap v1051: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 0 op/s 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 2.17 scrub starts 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[40649]: 2.17 scrub ok 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:33:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: pgmap v1051: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 0 op/s 2024-09-17T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 2.17 scrub starts 2024-09-17T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:08 smithi079 ceph-mon[29565]: 2.17 scrub ok 2024-09-17T08:33:09.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:33:09.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:33:09.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:33:09.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:33:09.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: pgmap v1051: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 644 B/s rd, 0 op/s 2024-09-17T08:33:09.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 2.17 scrub starts 2024-09-17T08:33:09.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:08 smithi173 ceph-mon[33187]: 2.17 scrub ok 2024-09-17T08:33:09.819 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T08:33:10.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:33:10.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:33:10.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:33:10.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:33:10.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:09 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:10.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:09 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:33:10.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:09 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:33:10.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:09 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:33:10.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:09 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:33:10.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:09 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-17T08:33:10.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-17T08:33:10.796 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:10.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 5.1d scrub starts 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 5.1d scrub ok 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: pgmap v1052: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 5.1a scrub starts 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: 5.1a scrub ok 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[29565]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T08:33:10.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 5.1d scrub starts 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 5.1d scrub ok 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: pgmap v1052: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 5.1a scrub starts 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: 5.1a scrub ok 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:10 smithi079 ceph-mon[40649]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 5.1d scrub starts 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 5.1d scrub ok 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: pgmap v1052: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 5.1a scrub starts 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: 5.1a scrub ok 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3131855244' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-17T08:33:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:10 smithi173 ceph-mon[33187]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T08:33:12.017 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:11 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:33:12.017 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:11 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:33:12.018 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:11 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:33:12.018 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:11 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:33:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:11 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:33:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:11 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:33:12.827 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T08:33:13.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 6.f deep-scrub starts 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 6.f deep-scrub ok 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 2.9 scrub starts 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 2.9 scrub ok 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: pgmap v1054: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:13.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 6.f deep-scrub starts 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 6.f deep-scrub ok 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 2.9 scrub starts 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 2.9 scrub ok 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: pgmap v1054: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:33:13.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:12 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:13.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 6.f deep-scrub starts 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 6.f deep-scrub ok 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 2.9 scrub starts 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 2.9 scrub ok 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: pgmap v1054: 142 pgs: 142 active+clean; 472 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:33:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:12 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:13.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-17T08:33:13.749 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:13.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: 4.10 scrub starts 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: 4.10 scrub ok 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: 1.6 scrub starts 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: 1.6 scrub ok 2024-09-17T08:33:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:13 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:14.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: 4.10 scrub starts 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: 4.10 scrub ok 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: 1.6 scrub starts 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: 1.6 scrub ok 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: 4.10 scrub starts 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: 4.10 scrub ok 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1251219582' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:14.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T08:33:14.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: 1.6 scrub starts 2024-09-17T08:33:14.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: 1.6 scrub ok 2024-09-17T08:33:14.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:13 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:15.069 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:33:15.070 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:33:15.070 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[40649]: pgmap v1056: 142 pgs: 142 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:15.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:33:15.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:33:15.070 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:14 smithi079 ceph-mon[29565]: pgmap v1056: 142 pgs: 142 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:15.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:14 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:33:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:14 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:33:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:14 smithi173 ceph-mon[33187]: pgmap v1056: 142 pgs: 142 active+clean; 420 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:15.844 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T08:33:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:33:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:33:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3330375001' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:15.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:33:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:33:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3330375001' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:15.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:15 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:16.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:15 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:33:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:15 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:33:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:15 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3330375001' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:15 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-17T08:33:16.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-17T08:33:16.857 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[40649]: pgmap v1057: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[40649]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T08:33:16.978 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:33:16.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:33:16.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[29565]: pgmap v1057: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:16.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:16.979 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:16 smithi079 ceph-mon[29565]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T08:33:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:16 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:33:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:16 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:33:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:16 smithi173 ceph-mon[33187]: pgmap v1057: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:16 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-17T08:33:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:16 smithi173 ceph-mon[33187]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T08:33:18.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:17 smithi079 ceph-mon[40649]: 4.8 scrub starts 2024-09-17T08:33:18.079 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:17 smithi079 ceph-mon[40649]: 4.8 scrub ok 2024-09-17T08:33:18.079 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:17 smithi079 ceph-mon[29565]: 4.8 scrub starts 2024-09-17T08:33:18.079 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:17 smithi079 ceph-mon[29565]: 4.8 scrub ok 2024-09-17T08:33:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:17 smithi173 ceph-mon[33187]: 4.8 scrub starts 2024-09-17T08:33:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:17 smithi173 ceph-mon[33187]: 4.8 scrub ok 2024-09-17T08:33:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T08:33:19.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: 2.e deep-scrub starts 2024-09-17T08:33:19.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: 2.e deep-scrub ok 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: pgmap v1059: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2001001865' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: 2.e deep-scrub starts 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: 2.e deep-scrub ok 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:33:19.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: pgmap v1059: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:19.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:19.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2001001865' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:18 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: 2.e deep-scrub starts 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: 2.e deep-scrub ok 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: pgmap v1059: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2001001865' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.146 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:18 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:19.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-17T08:33:19.745 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:33:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:33:20.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:20.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[40649]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T08:33:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:33:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:33:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:20.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:19 smithi079 ceph-mon[29565]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T08:33:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:19 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:33:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:19 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:33:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:19 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:19 smithi173 ceph-mon[33187]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T08:33:21.208 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[29565]: 5.17 scrub starts 2024-09-17T08:33:21.208 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[29565]: 5.17 scrub ok 2024-09-17T08:33:21.208 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[29565]: 6.1a scrub starts 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[29565]: 6.1a scrub ok 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[29565]: pgmap v1061: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[40649]: 5.17 scrub starts 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[40649]: 5.17 scrub ok 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[40649]: 6.1a scrub starts 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[40649]: 6.1a scrub ok 2024-09-17T08:33:21.209 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:20 smithi079 ceph-mon[40649]: pgmap v1061: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:20 smithi173 ceph-mon[33187]: 5.17 scrub starts 2024-09-17T08:33:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:20 smithi173 ceph-mon[33187]: 5.17 scrub ok 2024-09-17T08:33:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:20 smithi173 ceph-mon[33187]: 6.1a scrub starts 2024-09-17T08:33:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:20 smithi173 ceph-mon[33187]: 6.1a scrub ok 2024-09-17T08:33:21.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:20 smithi173 ceph-mon[33187]: pgmap v1061: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:22.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:21 smithi173 ceph-mon[33187]: 4.1a scrub starts 2024-09-17T08:33:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:21 smithi173 ceph-mon[33187]: 4.1a scrub ok 2024-09-17T08:33:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:21 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:22.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[40649]: 4.1a scrub starts 2024-09-17T08:33:22.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[40649]: 4.1a scrub ok 2024-09-17T08:33:22.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:22.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[29565]: 4.1a scrub starts 2024-09-17T08:33:22.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[29565]: 4.1a scrub ok 2024-09-17T08:33:22.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:21 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-17T08:33:22.615 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: 4.19 deep-scrub starts 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: 4.19 deep-scrub ok 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: pgmap v1062: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:22 smithi173 ceph-mon[33187]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T08:33:23.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: 4.19 deep-scrub starts 2024-09-17T08:33:23.323 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: 4.19 deep-scrub ok 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: pgmap v1062: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[29565]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: 4.19 deep-scrub starts 2024-09-17T08:33:23.324 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: 4.19 deep-scrub ok 2024-09-17T08:33:23.325 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: pgmap v1062: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:23.325 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:23.325 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:23.325 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/312058622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-17T08:33:23.325 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:22 smithi079 ceph-mon[40649]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T08:33:23.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-17T08:33:23.558 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:23.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-mon[33187]: 6.1f scrub starts 2024-09-17T08:33:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-mon[33187]: 6.1f scrub ok 2024-09-17T08:33:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-mon[33187]: 2.10 scrub starts 2024-09-17T08:33:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-mon[33187]: 2.10 scrub ok 2024-09-17T08:33:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:23.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[40649]: 6.1f scrub starts 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[40649]: 6.1f scrub ok 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[40649]: 2.10 scrub starts 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[40649]: 2.10 scrub ok 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[29565]: 6.1f scrub starts 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[29565]: 6.1f scrub ok 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[29565]: 2.10 scrub starts 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[29565]: 2.10 scrub ok 2024-09-17T08:33:23.977 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:23 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:25.078 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:24 smithi079 ceph-mon[40649]: pgmap v1064: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 649 B/s rd, 0 op/s 2024-09-17T08:33:25.079 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:24 smithi079 ceph-mon[29565]: pgmap v1064: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 649 B/s rd, 0 op/s 2024-09-17T08:33:25.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:24 smithi173 ceph-mon[33187]: pgmap v1064: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 649 B/s rd, 0 op/s 2024-09-17T08:33:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T08:33:26.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:25 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:33:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:25 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:33:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1661714174' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:25 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.256 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1661714174' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1661714174' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.257 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:25 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:26.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-17T08:33:26.804 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:27.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:33:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:33:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: pgmap v1065: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T08:33:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:26 smithi173 ceph-mon[33187]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: pgmap v1065: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[40649]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T08:33:27.228 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:33:27.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:33:27.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: pgmap v1065: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:27.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:27.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T08:33:27.229 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:26 smithi079 ceph-mon[29565]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T08:33:28.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: 6.11 scrub starts 2024-09-17T08:33:28.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: 6.11 scrub ok 2024-09-17T08:33:28.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:28.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:28.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: 2.a scrub starts 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[40649]: 2.a scrub ok 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: 6.11 scrub starts 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: 6.11 scrub ok 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: 2.a scrub starts 2024-09-17T08:33:28.009 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:27 smithi079 ceph-mon[29565]: 2.a scrub ok 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: 6.11 scrub starts 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: 6.11 scrub ok 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: 2.a scrub starts 2024-09-17T08:33:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:27 smithi173 ceph-mon[33187]: 2.a scrub ok 2024-09-17T08:33:28.900 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: 5.19 scrub starts 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: 5.19 scrub ok 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: 6.17 scrub starts 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: 6.17 scrub ok 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: pgmap v1068: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3174742532' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:28 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: 5.19 scrub starts 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: 5.19 scrub ok 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: 6.17 scrub starts 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: 6.17 scrub ok 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: pgmap v1068: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3174742532' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.225 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: 5.19 scrub starts 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: 5.19 scrub ok 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: 6.17 scrub starts 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: 6.17 scrub ok 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: pgmap v1068: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3174742532' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.226 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:28 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-17T08:33:29.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-17T08:33:29.743 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:33:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:33:30.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 1.5 scrub starts 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 1.5 scrub ok 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 5.14 scrub starts 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[40649]: 5.14 scrub ok 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 1.5 scrub starts 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 1.5 scrub ok 2024-09-17T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:30.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T08:33:30.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 5.14 scrub starts 2024-09-17T08:33:30.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:29 smithi079 ceph-mon[29565]: 5.14 scrub ok 2024-09-17T08:33:30.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 1.5 scrub starts 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 1.5 scrub ok 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 5.14 scrub starts 2024-09-17T08:33:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:29 smithi173 ceph-mon[33187]: 5.14 scrub ok 2024-09-17T08:33:31.213 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[40649]: pgmap v1070: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:33:31.213 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[40649]: 4.6 deep-scrub starts 2024-09-17T08:33:31.213 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[40649]: 4.6 deep-scrub ok 2024-09-17T08:33:31.214 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[40649]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T08:33:31.214 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[29565]: pgmap v1070: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:33:31.214 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[29565]: 4.6 deep-scrub starts 2024-09-17T08:33:31.214 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[29565]: 4.6 deep-scrub ok 2024-09-17T08:33:31.214 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:30 smithi079 ceph-mon[29565]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T08:33:31.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:30 smithi173 ceph-mon[33187]: pgmap v1070: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T08:33:31.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:30 smithi173 ceph-mon[33187]: 4.6 deep-scrub starts 2024-09-17T08:33:31.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:30 smithi173 ceph-mon[33187]: 4.6 deep-scrub ok 2024-09-17T08:33:31.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:30 smithi173 ceph-mon[33187]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T08:33:31.617 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T08:33:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: 6.2 deep-scrub starts 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: 6.2 deep-scrub ok 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: 5.7 scrub starts 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: 5.7 scrub ok 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/41830556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:31 smithi173 ceph-mon[33187]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: 6.2 deep-scrub starts 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: 6.2 deep-scrub ok 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: 5.7 scrub starts 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: 5.7 scrub ok 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/41830556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:32.277 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[40649]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: 6.2 deep-scrub starts 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: 6.2 deep-scrub ok 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: 5.7 scrub starts 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: 5.7 scrub ok 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/41830556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:32.278 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:31 smithi079 ceph-mon[29565]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T08:33:32.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-17T08:33:32.495 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:32 smithi173 ceph-mon[33187]: 1.3 scrub starts 2024-09-17T08:33:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:32 smithi173 ceph-mon[33187]: 1.3 scrub ok 2024-09-17T08:33:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:32 smithi173 ceph-mon[33187]: pgmap v1073: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-17T08:33:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:32 smithi173 ceph-mon[33187]: 4.3 scrub starts 2024-09-17T08:33:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:32 smithi173 ceph-mon[33187]: 4.3 scrub ok 2024-09-17T08:33:33.342 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:33.343 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[40649]: 1.3 scrub starts 2024-09-17T08:33:33.343 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[40649]: 1.3 scrub ok 2024-09-17T08:33:33.343 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[40649]: pgmap v1073: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-17T08:33:33.343 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[40649]: 4.3 scrub starts 2024-09-17T08:33:33.343 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[40649]: 4.3 scrub ok 2024-09-17T08:33:33.344 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[29565]: 1.3 scrub starts 2024-09-17T08:33:33.344 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[29565]: 1.3 scrub ok 2024-09-17T08:33:33.344 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[29565]: pgmap v1073: 142 pgs: 142 active+clean; 367 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-17T08:33:33.344 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[29565]: 4.3 scrub starts 2024-09-17T08:33:33.344 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:32 smithi079 ceph-mon[29565]: 4.3 scrub ok 2024-09-17T08:33:33.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: 2.d scrub starts 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: 2.d scrub ok 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: 5.d scrub starts 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: 5.d scrub ok 2024-09-17T08:33:34.173 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: 2.d scrub starts 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: 2.d scrub ok 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: 5.d scrub starts 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: 5.d scrub ok 2024-09-17T08:33:34.174 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:33 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: 2.d scrub starts 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: 2.d scrub ok 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: 5.d scrub starts 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: 5.d scrub ok 2024-09-17T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:33 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:34.923 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: 4.c scrub starts 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: 4.c scrub ok 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: 5.12 scrub starts 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: 5.12 scrub ok 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: pgmap v1074: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:34 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:35.242 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: 4.c scrub starts 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: 4.c scrub ok 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: 5.12 scrub starts 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: 5.12 scrub ok 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: pgmap v1074: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: 4.c scrub starts 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: 4.c scrub ok 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: 5.12 scrub starts 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: 5.12 scrub ok 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: pgmap v1074: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:35.243 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:34 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-17T08:33:35.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-17T08:33:36.101 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:36.221 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: 2.18 deep-scrub starts 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[29565]: 2.18 deep-scrub ok 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T08:33:36.222 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: 2.18 deep-scrub starts 2024-09-17T08:33:36.223 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:35 smithi079 ceph-mon[40649]: 2.18 deep-scrub ok 2024-09-17T08:33:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:33:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:33:36.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2129490991' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-17T08:33:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T08:33:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: 2.18 deep-scrub starts 2024-09-17T08:33:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:35 smithi173 ceph-mon[33187]: 2.18 deep-scrub ok 2024-09-17T08:33:37.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:36 smithi173 ceph-mon[33187]: pgmap v1076: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:36 smithi173 ceph-mon[33187]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T08:33:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:36 smithi173 ceph-mon[33187]: 1.1 deep-scrub starts 2024-09-17T08:33:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:36 smithi173 ceph-mon[33187]: 1.1 deep-scrub ok 2024-09-17T08:33:37.345 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[40649]: pgmap v1076: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:37.345 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[40649]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T08:33:37.345 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[40649]: 1.1 deep-scrub starts 2024-09-17T08:33:37.345 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[40649]: 1.1 deep-scrub ok 2024-09-17T08:33:37.346 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[29565]: pgmap v1076: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-09-17T08:33:37.346 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[29565]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T08:33:37.346 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[29565]: 1.1 deep-scrub starts 2024-09-17T08:33:37.346 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:36 smithi079 ceph-mon[29565]: 1.1 deep-scrub ok 2024-09-17T08:33:38.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:37 smithi173 ceph-mon[33187]: 5.1 scrub starts 2024-09-17T08:33:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:37 smithi173 ceph-mon[33187]: 5.1 scrub ok 2024-09-17T08:33:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:37 smithi173 ceph-mon[33187]: 3.2 scrub starts 2024-09-17T08:33:38.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:37 smithi173 ceph-mon[33187]: 3.2 scrub ok 2024-09-17T08:33:38.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[40649]: 5.1 scrub starts 2024-09-17T08:33:38.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[40649]: 5.1 scrub ok 2024-09-17T08:33:38.369 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[40649]: 3.2 scrub starts 2024-09-17T08:33:38.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[40649]: 3.2 scrub ok 2024-09-17T08:33:38.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[29565]: 5.1 scrub starts 2024-09-17T08:33:38.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[29565]: 5.1 scrub ok 2024-09-17T08:33:38.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[29565]: 3.2 scrub starts 2024-09-17T08:33:38.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:37 smithi079 ceph-mon[29565]: 3.2 scrub ok 2024-09-17T08:33:38.948 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: 2.1f deep-scrub starts 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: 2.1f deep-scrub ok 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: pgmap v1078: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: 6.c scrub starts 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: 6.c scrub ok 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1930832894' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:38 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.354 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: 2.1f deep-scrub starts 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: 2.1f deep-scrub ok 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: pgmap v1078: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: 6.c scrub starts 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: 6.c scrub ok 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1930832894' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:39.355 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: 2.1f deep-scrub starts 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: 2.1f deep-scrub ok 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: pgmap v1078: 142 pgs: 142 active+clean; 268 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: 6.c scrub starts 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: 6.c scrub ok 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1930832894' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.356 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:38 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:39.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-17T08:33:39.901 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:39 smithi173 ceph-mon[33187]: 1.8 scrub starts 2024-09-17T08:33:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:39 smithi173 ceph-mon[33187]: 1.8 scrub ok 2024-09-17T08:33:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:39 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:39 smithi173 ceph-mon[33187]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T08:33:40.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[40649]: 1.8 scrub starts 2024-09-17T08:33:40.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[40649]: 1.8 scrub ok 2024-09-17T08:33:40.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:40.308 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[40649]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T08:33:40.308 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[29565]: 1.8 scrub starts 2024-09-17T08:33:40.309 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[29565]: 1.8 scrub ok 2024-09-17T08:33:40.309 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:40.309 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:39 smithi079 ceph-mon[29565]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T08:33:41.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[29565]: pgmap v1081: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[29565]: 1.4 scrub starts 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[29565]: 1.4 scrub ok 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[29565]: 1.a deep-scrub starts 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[29565]: 1.a deep-scrub ok 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[40649]: pgmap v1081: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[40649]: 1.4 scrub starts 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[40649]: 1.4 scrub ok 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[40649]: 1.a deep-scrub starts 2024-09-17T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:40 smithi079 ceph-mon[40649]: 1.a deep-scrub ok 2024-09-17T08:33:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:40 smithi173 ceph-mon[33187]: pgmap v1081: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:40 smithi173 ceph-mon[33187]: 1.4 scrub starts 2024-09-17T08:33:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:40 smithi173 ceph-mon[33187]: 1.4 scrub ok 2024-09-17T08:33:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:40 smithi173 ceph-mon[33187]: 1.a deep-scrub starts 2024-09-17T08:33:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:40 smithi173 ceph-mon[33187]: 1.a deep-scrub ok 2024-09-17T08:33:42.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[29565]: 2.14 scrub starts 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[29565]: 2.14 scrub ok 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[29565]: 1.9 scrub starts 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[29565]: 1.9 scrub ok 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[40649]: 2.14 scrub starts 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[40649]: 2.14 scrub ok 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[40649]: 1.9 scrub starts 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[40649]: 1.9 scrub ok 2024-09-17T08:33:42.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:41 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:41 smithi173 ceph-mon[33187]: 2.14 scrub starts 2024-09-17T08:33:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:41 smithi173 ceph-mon[33187]: 2.14 scrub ok 2024-09-17T08:33:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:41 smithi173 ceph-mon[33187]: 1.9 scrub starts 2024-09-17T08:33:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:41 smithi173 ceph-mon[33187]: 1.9 scrub ok 2024-09-17T08:33:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:41 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-17T08:33:42.625 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T08:33:43.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:42 smithi173 ceph-mon[33187]: 6.e scrub starts 2024-09-17T08:33:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:42 smithi173 ceph-mon[33187]: 6.e scrub ok 2024-09-17T08:33:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:42 smithi173 ceph-mon[33187]: pgmap v1082: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:42 smithi173 ceph-mon[33187]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T08:33:43.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:43.369 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[29565]: 6.e scrub starts 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[29565]: 6.e scrub ok 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[29565]: pgmap v1082: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[29565]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[40649]: 6.e scrub starts 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[40649]: 6.e scrub ok 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[40649]: pgmap v1082: 142 pgs: 142 active+clean; 163 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/848942922' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-17T08:33:43.370 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:42 smithi079 ceph-mon[40649]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T08:33:43.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-17T08:33:43.635 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:43.960 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:44.052 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[40649]: 2.19 scrub starts 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[40649]: 2.19 scrub ok 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[40649]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[29565]: 2.19 scrub starts 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[29565]: 2.19 scrub ok 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[29565]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T08:33:44.053 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:43 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:44.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:43 smithi173 ceph-mon[33187]: 2.19 scrub starts 2024-09-17T08:33:44.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:43 smithi173 ceph-mon[33187]: 2.19 scrub ok 2024-09-17T08:33:44.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:43 smithi173 ceph-mon[33187]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T08:33:44.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:43 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[40649]: pgmap v1085: 142 pgs: 142 active+clean; 315 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[40649]: 1.2 scrub starts 2024-09-17T08:33:45.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[40649]: 1.2 scrub ok 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[40649]: 6.1d scrub starts 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[40649]: 6.1d scrub ok 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[29565]: pgmap v1085: 142 pgs: 142 active+clean; 315 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[29565]: 1.2 scrub starts 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[29565]: 1.2 scrub ok 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[29565]: 6.1d scrub starts 2024-09-17T08:33:45.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:45 smithi079 ceph-mon[29565]: 6.1d scrub ok 2024-09-17T08:33:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:45 smithi173 ceph-mon[33187]: pgmap v1085: 142 pgs: 142 active+clean; 315 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:45 smithi173 ceph-mon[33187]: 1.2 scrub starts 2024-09-17T08:33:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:45 smithi173 ceph-mon[33187]: 1.2 scrub ok 2024-09-17T08:33:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:45 smithi173 ceph-mon[33187]: 6.1d scrub starts 2024-09-17T08:33:45.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:45 smithi173 ceph-mon[33187]: 6.1d scrub ok 2024-09-17T08:33:46.635 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T08:33:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:46 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1872771665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:46 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:47.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:46 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1872771665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:47.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:46 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:47.008 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:46 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1872771665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:47.008 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:46 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:47.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-17T08:33:47.672 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:47.792 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[29565]: pgmap v1086: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[29565]: 5.1e scrub starts 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[29565]: 5.1e scrub ok 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[29565]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[29565]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[40649]: pgmap v1086: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[40649]: 5.1e scrub starts 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[40649]: 5.1e scrub ok 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[40649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:47.793 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:47 smithi079 ceph-mon[40649]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T08:33:47.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:47 smithi173 ceph-mon[33187]: pgmap v1086: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:47 smithi173 ceph-mon[33187]: 5.1e scrub starts 2024-09-17T08:33:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:47 smithi173 ceph-mon[33187]: 5.1e scrub ok 2024-09-17T08:33:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:47 smithi173 ceph-mon[33187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:47.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:47 smithi173 ceph-mon[33187]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 2.0 scrub starts 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 2.0 scrub ok 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 5.3 scrub starts 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 5.3 scrub ok 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 5.8 scrub starts 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: 5.8 scrub ok 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 2.0 scrub starts 2024-09-17T08:33:48.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 2.0 scrub ok 2024-09-17T08:33:48.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 5.3 scrub starts 2024-09-17T08:33:48.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 5.3 scrub ok 2024-09-17T08:33:48.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 5.8 scrub starts 2024-09-17T08:33:48.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: 5.8 scrub ok 2024-09-17T08:33:48.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:48 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 2.0 scrub starts 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 2.0 scrub ok 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 5.3 scrub starts 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 5.3 scrub ok 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 5.8 scrub starts 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: 5.8 scrub ok 2024-09-17T08:33:48.891 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:48 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:33:49.641 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T08:33:49.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: pgmap v1088: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: 5.c scrub starts 2024-09-17T08:33:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: 5.c scrub ok 2024-09-17T08:33:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: 4.17 scrub starts 2024-09-17T08:33:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: 4.17 scrub ok 2024-09-17T08:33:49.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:49 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:49.991 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: pgmap v1088: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: 5.c scrub starts 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: 5.c scrub ok 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: 4.17 scrub starts 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: 4.17 scrub ok 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: pgmap v1088: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: 5.c scrub starts 2024-09-17T08:33:49.992 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: 5.c scrub ok 2024-09-17T08:33:49.993 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: 4.17 scrub starts 2024-09-17T08:33:49.993 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: 4.17 scrub ok 2024-09-17T08:33:49.993 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:49 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-17T08:33:50.419 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T08:33:50.419 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T08:33:50.419 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T08:33:50.419 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T08:33:50.419 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T08:33:50.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:33:50.626 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: 4.2 scrub starts 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: 4.2 scrub ok 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: 2.1b scrub starts 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[40649]: 2.1b scrub ok 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: 4.2 scrub starts 2024-09-17T08:33:50.902 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: 4.2 scrub ok 2024-09-17T08:33:50.903 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:50.903 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T08:33:50.903 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: 2.1b scrub starts 2024-09-17T08:33:50.903 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:50 smithi079 ceph-mon[29565]: 2.1b scrub ok 2024-09-17T08:33:50.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: 4.2 scrub starts 2024-09-17T08:33:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: 4.2 scrub ok 2024-09-17T08:33:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/1420405816' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-17T08:33:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T08:33:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: 2.1b scrub starts 2024-09-17T08:33:50.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:50 smithi173 ceph-mon[33187]: 2.1b scrub ok 2024-09-17T08:33:51.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: pgmap v1090: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: 6.16 scrub starts 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: 6.16 scrub ok 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: 4.14 scrub starts 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: 4.14 scrub ok 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[40649]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T08:33:51.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: pgmap v1090: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: 6.16 scrub starts 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: 6.16 scrub ok 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: 4.14 scrub starts 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: 4.14 scrub ok 2024-09-17T08:33:51.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:51 smithi079 ceph-mon[29565]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: pgmap v1090: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: 6.16 scrub starts 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: 6.16 scrub ok 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: 4.14 scrub starts 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: 4.14 scrub ok 2024-09-17T08:33:51.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:51 smithi173 ceph-mon[33187]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T08:33:52.604 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:33:52.604 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":465,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","created":"2024-09-17T08:04:41.066270+0000","modified":"2024-09-17T08:33:51.614462+0000","last_up_change":"2024-09-17T08:32:31.631853+0000","last_in_change":"2024-09-17T08:32:42.689250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:07:44.689342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"441","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"332","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:09:53.075171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-17T08:09:53.309442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:09:54.771473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:09:56.745903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:09:58.808366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"793e335a-ea62-499c-af46-e48c3190408a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6803","nonce":2553823997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6805","nonce":2553823997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6809","nonce":2553823997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6807","nonce":2553823997}]},"public_addr":"172.21.15.79:6803/2553823997","cluster_addr":"172.21.15.79:6805/2553823997","heartbeat_back_addr":"172.21.15.79:6809/2553823997","heartbeat_front_addr":"172.21.15.79:6807/2553823997","state":["exists","up"]},{"osd":1,"uuid":"dda3e35f-7364-47fa-af06-610c50ac856d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6811","nonce":221772614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6813","nonce":221772614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6817","nonce":221772614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6815","nonce":221772614}]},"public_addr":"172.21.15.79:6811/221772614","cluster_addr":"172.21.15.79:6813/221772614","heartbeat_back_addr":"172.21.15.79:6817/221772614","heartbeat_front_addr":"172.21.15.79:6815/221772614","state":["exists","up"]},{"osd":2,"uuid":"e121b6c9-f67b-4802-a92b-9c709ff433c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":204,"up_from":227,"up_thru":460,"down_at":205,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6819","nonce":359954782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6821","nonce":359954782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6825","nonce":359954782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6823","nonce":359954782}]},"public_addr":"172.21.15.79:6819/359954782","cluster_addr":"172.21.15.79:6821/359954782","heartbeat_back_addr":"172.21.15.79:6825/359954782","heartbeat_front_addr":"172.21.15.79:6823/359954782","state":["exists","up"]},{"osd":3,"uuid":"69af955a-9dbe-4d86-8278-27df098a3886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":440,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6827","nonce":3442203570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6829","nonce":3442203570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6833","nonce":3442203570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6831","nonce":3442203570}]},"public_addr":"172.21.15.79:6827/3442203570","cluster_addr":"172.21.15.79:6829/3442203570","heartbeat_back_addr":"172.21.15.79:6833/3442203570","heartbeat_front_addr":"172.21.15.79:6831/3442203570","state":["exists","up"]},{"osd":4,"uuid":"2ab07ff0-d7f3-4fca-a97b-843f7e69fc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6801","nonce":2625332597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6803","nonce":2625332597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6807","nonce":2625332597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6805","nonce":2625332597}]},"public_addr":"172.21.15.173:6801/2625332597","cluster_addr":"172.21.15.173:6803/2625332597","heartbeat_back_addr":"172.21.15.173:6807/2625332597","heartbeat_front_addr":"172.21.15.173:6805/2625332597","state":["exists","up"]},{"osd":5,"uuid":"2fc412d9-7b28-43b0-b7d3-80fff14855ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":453,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6809","nonce":329100432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6811","nonce":329100432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6815","nonce":329100432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6813","nonce":329100432}]},"public_addr":"172.21.15.173:6809/329100432","cluster_addr":"172.21.15.173:6811/329100432","heartbeat_back_addr":"172.21.15.173:6815/329100432","heartbeat_front_addr":"172.21.15.173:6813/329100432","state":["exists","up"]},{"osd":6,"uuid":"19197da7-927a-4422-8d18-ca9918136919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":50,"last_clean_end":424,"up_from":434,"up_thru":440,"down_at":425,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6817","nonce":2093386064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6819","nonce":2093386064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6823","nonce":2093386064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6821","nonce":2093386064}]},"public_addr":"172.21.15.173:6817/2093386064","cluster_addr":"172.21.15.173:6819/2093386064","heartbeat_back_addr":"172.21.15.173:6823/2093386064","heartbeat_front_addr":"172.21.15.173:6821/2093386064","state":["exists","up"]},{"osd":7,"uuid":"b32fec83-5977-488d-b36c-bba34b9f3dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":456,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6825","nonce":777262325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6827","nonce":777262325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6831","nonce":777262325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6829","nonce":777262325}]},"public_addr":"172.21.15.173:6825/777262325","cluster_addr":"172.21.15.173:6827/777262325","heartbeat_back_addr":"172.21.15.173:6831/777262325","heartbeat_front_addr":"172.21.15.173:6829/777262325","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:37.205506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:26.357415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:50.273420+0000","dead_epoch":204},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:32:58.645887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:06.058594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:22.194493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:32:38.057159+0000","dead_epoch":424},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:02.325860+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.10","osds":[4,5,6]},{"pgid":"5.2","osds":[5,6,2]}],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":7,"to":2},{"from":4,"to":0}]},{"pgid":"2.12","mappings":[{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":0}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"6.b","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/3019822778":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3759482241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/2023211636":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/128416645":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2554615564":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3326230156":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1830656502":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1850619208":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6801/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2454308049":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/107735338":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/3951775509":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6800/3253737241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/3253737241":"2024-09-18T08:10:31.196134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:33:52.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[40649]: 2.1 scrub starts 2024-09-17T08:33:52.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[40649]: 2.1 scrub ok 2024-09-17T08:33:52.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2648372710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:52.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[29565]: 2.1 scrub starts 2024-09-17T08:33:52.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[29565]: 2.1 scrub ok 2024-09-17T08:33:52.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:52 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2648372710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:52.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:52 smithi173 ceph-mon[33187]: 2.1 scrub starts 2024-09-17T08:33:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:52 smithi173 ceph-mon[33187]: 2.1 scrub ok 2024-09-17T08:33:52.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:52 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2648372710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:53.229 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:33:53 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:33:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:33:53.253 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T08:33:53.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T08:33:53.453 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:53.849 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:53 smithi079 ceph-mon[40649]: pgmap v1093: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:53.850 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:53 smithi079 ceph-mon[29565]: pgmap v1093: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:53.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:53 smithi173 ceph-mon[33187]: pgmap v1093: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:33:53.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:33:53 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:33:54.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:54 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:33:54.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:54 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:33:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:54 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:33:54.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:54 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:33:54.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:54 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:33:54.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:54 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:33:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:33:55.317 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":466,"fsid":"66e8930a-74cb-11ef-bceb-c7b262605968","created":"2024-09-17T08:04:41.066270+0000","modified":"2024-09-17T08:33:54.662313+0000","last_up_change":"2024-09-17T08:32:31.631853+0000","last_in_change":"2024-09-17T08:32:42.689250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T08:07:44.689342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"441","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"332","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T08:09:53.075171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-17T08:09:53.309442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T08:09:54.771473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T08:09:56.745903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T08:09:58.808366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"793e335a-ea62-499c-af46-e48c3190408a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6803","nonce":2553823997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6805","nonce":2553823997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6809","nonce":2553823997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2553823997},{"type":"v1","addr":"172.21.15.79:6807","nonce":2553823997}]},"public_addr":"172.21.15.79:6803/2553823997","cluster_addr":"172.21.15.79:6805/2553823997","heartbeat_back_addr":"172.21.15.79:6809/2553823997","heartbeat_front_addr":"172.21.15.79:6807/2553823997","state":["exists","up"]},{"osd":1,"uuid":"dda3e35f-7364-47fa-af06-610c50ac856d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6811","nonce":221772614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6813","nonce":221772614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6817","nonce":221772614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":221772614},{"type":"v1","addr":"172.21.15.79:6815","nonce":221772614}]},"public_addr":"172.21.15.79:6811/221772614","cluster_addr":"172.21.15.79:6813/221772614","heartbeat_back_addr":"172.21.15.79:6817/221772614","heartbeat_front_addr":"172.21.15.79:6815/221772614","state":["exists","up"]},{"osd":2,"uuid":"e121b6c9-f67b-4802-a92b-9c709ff433c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":204,"up_from":227,"up_thru":460,"down_at":205,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6819","nonce":359954782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6821","nonce":359954782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6825","nonce":359954782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":359954782},{"type":"v1","addr":"172.21.15.79:6823","nonce":359954782}]},"public_addr":"172.21.15.79:6819/359954782","cluster_addr":"172.21.15.79:6821/359954782","heartbeat_back_addr":"172.21.15.79:6825/359954782","heartbeat_front_addr":"172.21.15.79:6823/359954782","state":["exists","up"]},{"osd":3,"uuid":"69af955a-9dbe-4d86-8278-27df098a3886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":440,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6827","nonce":3442203570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6829","nonce":3442203570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6833","nonce":3442203570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":3442203570},{"type":"v1","addr":"172.21.15.79:6831","nonce":3442203570}]},"public_addr":"172.21.15.79:6827/3442203570","cluster_addr":"172.21.15.79:6829/3442203570","heartbeat_back_addr":"172.21.15.79:6833/3442203570","heartbeat_front_addr":"172.21.15.79:6831/3442203570","state":["exists","up"]},{"osd":4,"uuid":"2ab07ff0-d7f3-4fca-a97b-843f7e69fc35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":437,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6801","nonce":2625332597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6803","nonce":2625332597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6807","nonce":2625332597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2625332597},{"type":"v1","addr":"172.21.15.173:6805","nonce":2625332597}]},"public_addr":"172.21.15.173:6801/2625332597","cluster_addr":"172.21.15.173:6803/2625332597","heartbeat_back_addr":"172.21.15.173:6807/2625332597","heartbeat_front_addr":"172.21.15.173:6805/2625332597","state":["exists","up"]},{"osd":5,"uuid":"2fc412d9-7b28-43b0-b7d3-80fff14855ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":453,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6809","nonce":329100432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6811","nonce":329100432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6815","nonce":329100432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":329100432},{"type":"v1","addr":"172.21.15.173:6813","nonce":329100432}]},"public_addr":"172.21.15.173:6809/329100432","cluster_addr":"172.21.15.173:6811/329100432","heartbeat_back_addr":"172.21.15.173:6815/329100432","heartbeat_front_addr":"172.21.15.173:6813/329100432","state":["exists","up"]},{"osd":6,"uuid":"19197da7-927a-4422-8d18-ca9918136919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":50,"last_clean_end":424,"up_from":434,"up_thru":440,"down_at":425,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6817","nonce":2093386064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6819","nonce":2093386064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6823","nonce":2093386064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2093386064},{"type":"v1","addr":"172.21.15.173:6821","nonce":2093386064}]},"public_addr":"172.21.15.173:6817/2093386064","cluster_addr":"172.21.15.173:6819/2093386064","heartbeat_back_addr":"172.21.15.173:6823/2093386064","heartbeat_front_addr":"172.21.15.173:6821/2093386064","state":["exists","up"]},{"osd":7,"uuid":"b32fec83-5977-488d-b36c-bba34b9f3dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":456,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6825","nonce":777262325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6827","nonce":777262325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6831","nonce":777262325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":777262325},{"type":"v1","addr":"172.21.15.173:6829","nonce":777262325}]},"public_addr":"172.21.15.173:6825/777262325","cluster_addr":"172.21.15.173:6827/777262325","heartbeat_back_addr":"172.21.15.173:6831/777262325","heartbeat_front_addr":"172.21.15.173:6829/777262325","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:37.205506+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:26.357415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:50.273420+0000","dead_epoch":204},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:32:58.645887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:06.058594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:22.194493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:54.059804+0000","dead_epoch":424},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T08:33:02.325860+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.10","osds":[4,5,6]},{"pgid":"5.2","osds":[5,6,2]}],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":7,"to":2},{"from":4,"to":0}]},{"pgid":"2.12","mappings":[{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":0}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"6.b","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/3019822778":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3759482241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/2023211636":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/128416645":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2554615564":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/3326230156":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/673928727":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1830656502":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/1850619208":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6800/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:6801/610659042":"2024-09-18T08:05:11.956861+0000","172.21.15.79:0/2454308049":"2024-09-18T08:10:31.196134+0000","172.21.15.79:0/107735338":"2024-09-18T08:05:34.382508+0000","172.21.15.79:0/3951775509":"2024-09-18T08:05:34.382508+0000","172.21.15.79:6800/3253737241":"2024-09-18T08:10:31.196134+0000","172.21.15.79:6801/3253737241":"2024-09-18T08:10:31.196134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T08:33:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:55 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:55.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:55 smithi173 ceph-mon[33187]: pgmap v1094: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:55 smithi173 ceph-mon[33187]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T08:33:55.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:55 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/284052543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[29565]: pgmap v1094: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[29565]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/284052543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[40649]: pgmap v1094: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[40649]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T08:33:56.013 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:55 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/284052543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T08:33:56.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T08:33:56.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T08:33:56.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-17T08:33:56.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-17T08:33:56.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-17T08:33:56.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-17T08:33:56.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-17T08:33:56.438 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:56.490 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: 2.8 scrub starts 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: 2.8 scrub ok 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: pgmap v1096: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[40649]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T08:33:56.841 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: 2.8 scrub starts 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: 2.8 scrub ok 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: pgmap v1096: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:33:56.842 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:56 smithi079 ceph-mon[29565]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: 2.8 scrub starts 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: 2.8 scrub ok 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: pgmap v1096: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' 2024-09-17T08:33:56.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:56 smithi173 ceph-mon[33187]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T08:33:57.528 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.528 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.528 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.529 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.529 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.534 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:33:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 2.1d scrub starts 2024-09-17T08:33:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 2.1d scrub ok 2024-09-17T08:33:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 6.1e scrub starts 2024-09-17T08:33:57.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 6.1e scrub ok 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 4.11 scrub starts 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[29565]: 4.11 scrub ok 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 2.1d scrub starts 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 2.1d scrub ok 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 6.1e scrub starts 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 6.1e scrub ok 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 4.11 scrub starts 2024-09-17T08:33:57.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:57 smithi079 ceph-mon[40649]: 4.11 scrub ok 2024-09-17T08:33:57.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 2.1d scrub starts 2024-09-17T08:33:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 2.1d scrub ok 2024-09-17T08:33:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 6.1e scrub starts 2024-09-17T08:33:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 6.1e scrub ok 2024-09-17T08:33:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 4.11 scrub starts 2024-09-17T08:33:57.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:57 smithi173 ceph-mon[33187]: 4.11 scrub ok 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[29565]: pgmap v1098: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[29565]: 5.0 deep-scrub starts 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[29565]: 5.0 deep-scrub ok 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[40649]: pgmap v1098: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[40649]: 5.0 deep-scrub starts 2024-09-17T08:33:58.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:58 smithi079 ceph-mon[40649]: 5.0 deep-scrub ok 2024-09-17T08:33:58.976 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:58 smithi173 ceph-mon[33187]: pgmap v1098: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-17T08:33:58.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:58 smithi173 ceph-mon[33187]: 5.0 deep-scrub starts 2024-09-17T08:33:58.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:58 smithi173 ceph-mon[33187]: 5.0 deep-scrub ok 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 2.5 scrub starts 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 2.5 scrub ok 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 5.15 scrub starts 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 5.15 scrub ok 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 4.9 deep-scrub starts 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 4.9 deep-scrub ok 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 6.1 scrub starts 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 6.1 scrub ok 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 4.5 deep-scrub starts 2024-09-17T08:33:59.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[29565]: 4.5 deep-scrub ok 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 2.5 scrub starts 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 2.5 scrub ok 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 5.15 scrub starts 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 5.15 scrub ok 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 4.9 deep-scrub starts 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 4.9 deep-scrub ok 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 6.1 scrub starts 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 6.1 scrub ok 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 4.5 deep-scrub starts 2024-09-17T08:33:59.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:33:59 smithi079 ceph-mon[40649]: 4.5 deep-scrub ok 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 2.5 scrub starts 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 2.5 scrub ok 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 5.15 scrub starts 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 5.15 scrub ok 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 4.9 deep-scrub starts 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 4.9 deep-scrub ok 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 6.1 scrub starts 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 6.1 scrub ok 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 4.5 deep-scrub starts 2024-09-17T08:33:59.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:33:59 smithi173 ceph-mon[33187]: 4.5 deep-scrub ok 2024-09-17T08:34:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[40649]: 2.7 scrub starts 2024-09-17T08:34:00.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[40649]: 2.7 scrub ok 2024-09-17T08:34:00.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[40649]: pgmap v1099: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[29565]: 2.7 scrub starts 2024-09-17T08:34:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[29565]: 2.7 scrub ok 2024-09-17T08:34:00.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:00 smithi079 ceph-mon[29565]: pgmap v1099: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:00 smithi173 ceph-mon[33187]: 2.7 scrub starts 2024-09-17T08:34:00.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:00 smithi173 ceph-mon[33187]: 2.7 scrub ok 2024-09-17T08:34:00.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:00 smithi173 ceph-mon[33187]: pgmap v1099: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:01.008 INFO:teuthology.orchestra.run.smithi079.stdout:115964117306 2024-09-17T08:34:01.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-17T08:34:01.970 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:01 smithi079 ceph-mon[29565]: 5.1b deep-scrub starts 2024-09-17T08:34:01.970 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:01 smithi079 ceph-mon[29565]: 5.1b deep-scrub ok 2024-09-17T08:34:01.971 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:01 smithi079 ceph-mon[40649]: 5.1b deep-scrub starts 2024-09-17T08:34:01.971 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:01 smithi079 ceph-mon[40649]: 5.1b deep-scrub ok 2024-09-17T08:34:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:01 smithi173 ceph-mon[33187]: 5.1b deep-scrub starts 2024-09-17T08:34:01.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:01 smithi173 ceph-mon[33187]: 5.1b deep-scrub ok 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[40649]: 5.9 scrub starts 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[40649]: 5.9 scrub ok 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[40649]: pgmap v1100: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[40649]: 4.0 scrub starts 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[40649]: 4.0 scrub ok 2024-09-17T08:34:02.827 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[29565]: 5.9 scrub starts 2024-09-17T08:34:02.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[29565]: 5.9 scrub ok 2024-09-17T08:34:02.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[29565]: pgmap v1100: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:02.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[29565]: 4.0 scrub starts 2024-09-17T08:34:02.828 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:02 smithi079 ceph-mon[29565]: 4.0 scrub ok 2024-09-17T08:34:02.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:02 smithi173 ceph-mon[33187]: 5.9 scrub starts 2024-09-17T08:34:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:02 smithi173 ceph-mon[33187]: 5.9 scrub ok 2024-09-17T08:34:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:02 smithi173 ceph-mon[33187]: pgmap v1100: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:02 smithi173 ceph-mon[33187]: 4.0 scrub starts 2024-09-17T08:34:02.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:02 smithi173 ceph-mon[33187]: 4.0 scrub ok 2024-09-17T08:34:03.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:34:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:03.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:34:03 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:34:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:03 smithi173 ceph-mon[33187]: 6.1b scrub starts 2024-09-17T08:34:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:03 smithi173 ceph-mon[33187]: 6.1b scrub ok 2024-09-17T08:34:03.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:03 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:04.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[40649]: 6.1b scrub starts 2024-09-17T08:34:04.050 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[40649]: 6.1b scrub ok 2024-09-17T08:34:04.051 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:04.051 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[29565]: 6.1b scrub starts 2024-09-17T08:34:04.051 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[29565]: 6.1b scrub ok 2024-09-17T08:34:04.051 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:03 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:04.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 5.6 scrub starts 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 5.6 scrub ok 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 2.13 scrub starts 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 2.13 scrub ok 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: pgmap v1101: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 564 B/s rd, 0 op/s 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 6.b scrub starts 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[29565]: 6.b scrub ok 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 5.6 scrub starts 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 5.6 scrub ok 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 2.13 scrub starts 2024-09-17T08:34:04.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 2.13 scrub ok 2024-09-17T08:34:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: pgmap v1101: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 564 B/s rd, 0 op/s 2024-09-17T08:34:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 6.b scrub starts 2024-09-17T08:34:04.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:04 smithi079 ceph-mon[40649]: 6.b scrub ok 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 5.6 scrub starts 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 5.6 scrub ok 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 2.13 scrub starts 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 2.13 scrub ok 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: pgmap v1101: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 564 B/s rd, 0 op/s 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 6.b scrub starts 2024-09-17T08:34:04.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:04 smithi173 ceph-mon[33187]: 6.b scrub ok 2024-09-17T08:34:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:253403070760 2024-09-17T08:34:05.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-17T08:34:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 2.2 scrub starts 2024-09-17T08:34:05.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 2.2 scrub ok 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 4.13 scrub starts 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 4.13 scrub ok 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 4.1d scrub starts 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[40649]: 4.1d scrub ok 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 2.2 scrub starts 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 2.2 scrub ok 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 4.13 scrub starts 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 4.13 scrub ok 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 4.1d scrub starts 2024-09-17T08:34:05.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:05 smithi079 ceph-mon[29565]: 4.1d scrub ok 2024-09-17T08:34:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 2.2 scrub starts 2024-09-17T08:34:05.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 2.2 scrub ok 2024-09-17T08:34:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 4.13 scrub starts 2024-09-17T08:34:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 4.13 scrub ok 2024-09-17T08:34:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 4.1d scrub starts 2024-09-17T08:34:05.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:05 smithi173 ceph-mon[33187]: 4.1d scrub ok 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[40649]: 6.a scrub starts 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[40649]: 6.a scrub ok 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[40649]: pgmap v1102: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[40649]: 5.13 scrub starts 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[40649]: 5.13 scrub ok 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[29565]: 6.a scrub starts 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[29565]: 6.a scrub ok 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[29565]: pgmap v1102: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[29565]: 5.13 scrub starts 2024-09-17T08:34:06.863 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:06 smithi079 ceph-mon[29565]: 5.13 scrub ok 2024-09-17T08:34:06.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:06 smithi173 ceph-mon[33187]: 6.a scrub starts 2024-09-17T08:34:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:06 smithi173 ceph-mon[33187]: 6.a scrub ok 2024-09-17T08:34:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:06 smithi173 ceph-mon[33187]: pgmap v1102: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:06 smithi173 ceph-mon[33187]: 5.13 scrub starts 2024-09-17T08:34:06.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:06 smithi173 ceph-mon[33187]: 5.13 scrub ok 2024-09-17T08:34:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:07 smithi173 ceph-mon[33187]: 4.f scrub starts 2024-09-17T08:34:07.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:07 smithi173 ceph-mon[33187]: 4.f scrub ok 2024-09-17T08:34:08.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:07 smithi079 ceph-mon[40649]: 4.f scrub starts 2024-09-17T08:34:08.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:07 smithi079 ceph-mon[40649]: 4.f scrub ok 2024-09-17T08:34:08.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:07 smithi079 ceph-mon[29565]: 4.f scrub starts 2024-09-17T08:34:08.116 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:07 smithi079 ceph-mon[29565]: 4.f scrub ok 2024-09-17T08:34:08.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 6.15 scrub starts 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 6.15 scrub ok 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 2.3 scrub starts 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 2.3 scrub ok 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: pgmap v1103: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 4.4 deep-scrub starts 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[29565]: 4.4 deep-scrub ok 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 6.15 scrub starts 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 6.15 scrub ok 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 2.3 scrub starts 2024-09-17T08:34:08.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 2.3 scrub ok 2024-09-17T08:34:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: pgmap v1103: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:34:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 4.4 deep-scrub starts 2024-09-17T08:34:08.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:08 smithi079 ceph-mon[40649]: 4.4 deep-scrub ok 2024-09-17T08:34:08.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 6.15 scrub starts 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 6.15 scrub ok 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 2.3 scrub starts 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 2.3 scrub ok 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: pgmap v1103: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 4.4 deep-scrub starts 2024-09-17T08:34:08.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:08 smithi173 ceph-mon[33187]: 4.4 deep-scrub ok 2024-09-17T08:34:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:09 smithi173 ceph-mon[33187]: 4.1f scrub starts 2024-09-17T08:34:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:09 smithi173 ceph-mon[33187]: 4.1f scrub ok 2024-09-17T08:34:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:09 smithi173 ceph-mon[33187]: 6.18 deep-scrub starts 2024-09-17T08:34:09.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:09 smithi173 ceph-mon[33187]: 6.18 deep-scrub ok 2024-09-17T08:34:10.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[29565]: 4.1f scrub starts 2024-09-17T08:34:10.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[29565]: 4.1f scrub ok 2024-09-17T08:34:10.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[29565]: 6.18 deep-scrub starts 2024-09-17T08:34:10.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[29565]: 6.18 deep-scrub ok 2024-09-17T08:34:10.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[40649]: 4.1f scrub starts 2024-09-17T08:34:10.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[40649]: 4.1f scrub ok 2024-09-17T08:34:10.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[40649]: 6.18 deep-scrub starts 2024-09-17T08:34:10.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:09 smithi079 ceph-mon[40649]: 6.18 deep-scrub ok 2024-09-17T08:34:10.861 INFO:teuthology.orchestra.run.smithi079.stdout:1864015806485 2024-09-17T08:34:10.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-17T08:34:11.057 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[40649]: 5.10 scrub starts 2024-09-17T08:34:11.058 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[40649]: 5.10 scrub ok 2024-09-17T08:34:11.058 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[40649]: pgmap v1104: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[29565]: 5.10 scrub starts 2024-09-17T08:34:11.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[29565]: 5.10 scrub ok 2024-09-17T08:34:11.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:10 smithi079 ceph-mon[29565]: pgmap v1104: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:10 smithi173 ceph-mon[33187]: 5.10 scrub starts 2024-09-17T08:34:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:10 smithi173 ceph-mon[33187]: 5.10 scrub ok 2024-09-17T08:34:11.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:10 smithi173 ceph-mon[33187]: pgmap v1104: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:11.981 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 6.3 deep-scrub starts 2024-09-17T08:34:11.981 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 6.3 deep-scrub ok 2024-09-17T08:34:11.981 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 5.2 scrub starts 2024-09-17T08:34:11.981 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 5.2 scrub ok 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 6.14 scrub starts 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[29565]: 6.14 scrub ok 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 6.3 deep-scrub starts 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 6.3 deep-scrub ok 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 5.2 scrub starts 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 5.2 scrub ok 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 6.14 scrub starts 2024-09-17T08:34:11.982 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:11 smithi079 ceph-mon[40649]: 6.14 scrub ok 2024-09-17T08:34:12.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 6.3 deep-scrub starts 2024-09-17T08:34:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 6.3 deep-scrub ok 2024-09-17T08:34:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 5.2 scrub starts 2024-09-17T08:34:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 5.2 scrub ok 2024-09-17T08:34:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 6.14 scrub starts 2024-09-17T08:34:12.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:11 smithi173 ceph-mon[33187]: 6.14 scrub ok 2024-09-17T08:34:13.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 6.4 scrub starts 2024-09-17T08:34:13.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 6.4 scrub ok 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 5.18 scrub starts 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 5.18 scrub ok 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: pgmap v1105: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 6.6 scrub starts 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[40649]: 6.6 scrub ok 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 6.4 scrub starts 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 6.4 scrub ok 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 5.18 scrub starts 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 5.18 scrub ok 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: pgmap v1105: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 6.6 scrub starts 2024-09-17T08:34:13.030 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:12 smithi079 ceph-mon[29565]: 6.6 scrub ok 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 6.4 scrub starts 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 6.4 scrub ok 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 5.18 scrub starts 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 5.18 scrub ok 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: pgmap v1105: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 6.6 scrub starts 2024-09-17T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:12 smithi173 ceph-mon[33187]: 6.6 scrub ok 2024-09-17T08:34:13.271 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:13.289 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:34:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:13.289 INFO:teuthology.orchestra.run.smithi079.stdout:176093659442 2024-09-17T08:34:13.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-17T08:34:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[29565]: 5.e scrub starts 2024-09-17T08:34:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[29565]: 5.e scrub ok 2024-09-17T08:34:13.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[29565]: 4.1e scrub starts 2024-09-17T08:34:13.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[29565]: 4.1e scrub ok 2024-09-17T08:34:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[40649]: 5.e scrub starts 2024-09-17T08:34:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[40649]: 5.e scrub ok 2024-09-17T08:34:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[40649]: 4.1e scrub starts 2024-09-17T08:34:13.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:13 smithi079 ceph-mon[40649]: 4.1e scrub ok 2024-09-17T08:34:13.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:34:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:34:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:13 smithi173 ceph-mon[33187]: 5.e scrub starts 2024-09-17T08:34:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:13 smithi173 ceph-mon[33187]: 5.e scrub ok 2024-09-17T08:34:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:13 smithi173 ceph-mon[33187]: 4.1e scrub starts 2024-09-17T08:34:13.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:13 smithi173 ceph-mon[33187]: 4.1e scrub ok 2024-09-17T08:34:14.211 INFO:teuthology.orchestra.run.smithi079.stdout:34359738696 2024-09-17T08:34:14.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T08:34:14.889 INFO:teuthology.orchestra.run.smithi079.stdout:55834575174 2024-09-17T08:34:14.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T08:34:14.928 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: 2.1c scrub starts 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: 2.1c scrub ok 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: 6.9 deep-scrub starts 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: 6.9 deep-scrub ok 2024-09-17T08:34:15.072 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:15.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: pgmap v1106: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.073 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[29565]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T08:34:15.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: 2.1c scrub starts 2024-09-17T08:34:15.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: 2.1c scrub ok 2024-09-17T08:34:15.073 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: 6.9 deep-scrub starts 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: 6.9 deep-scrub ok 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: pgmap v1106: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.074 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:14 smithi079 ceph-mon[40649]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T08:34:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: 2.1c scrub starts 2024-09-17T08:34:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: 2.1c scrub ok 2024-09-17T08:34:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:34:15.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:34:15.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: 6.9 deep-scrub starts 2024-09-17T08:34:15.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: 6.9 deep-scrub ok 2024-09-17T08:34:15.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:15.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: pgmap v1106: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:15.225 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:14 smithi173 ceph-mon[33187]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T08:34:15.430 INFO:teuthology.orchestra.run.smithi079.stdout:137438953784 2024-09-17T08:34:15.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-17T08:34:16.007 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[29565]: 4.d scrub starts 2024-09-17T08:34:16.007 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[29565]: 4.d scrub ok 2024-09-17T08:34:16.007 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[29565]: 2.6 scrub starts 2024-09-17T08:34:16.007 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[29565]: 2.6 scrub ok 2024-09-17T08:34:16.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[40649]: 4.d scrub starts 2024-09-17T08:34:16.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[40649]: 4.d scrub ok 2024-09-17T08:34:16.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[40649]: 2.6 scrub starts 2024-09-17T08:34:16.008 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:15 smithi079 ceph-mon[40649]: 2.6 scrub ok 2024-09-17T08:34:16.029 INFO:teuthology.orchestra.run.smithi079.stdout:974957576359 2024-09-17T08:34:16.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T08:34:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:15 smithi173 ceph-mon[33187]: 4.d scrub starts 2024-09-17T08:34:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:15 smithi173 ceph-mon[33187]: 4.d scrub ok 2024-09-17T08:34:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:15 smithi173 ceph-mon[33187]: 2.6 scrub starts 2024-09-17T08:34:16.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:15 smithi173 ceph-mon[33187]: 2.6 scrub ok 2024-09-17T08:34:16.804 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:16.804 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:16.804 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:17.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[40649]: 2.b scrub starts 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[40649]: 2.b scrub ok 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[40649]: pgmap v1108: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[40649]: 4.7 scrub starts 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[40649]: 4.7 scrub ok 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[29565]: 2.b scrub starts 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[29565]: 2.b scrub ok 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[29565]: pgmap v1108: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[29565]: 4.7 scrub starts 2024-09-17T08:34:17.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:16 smithi079 ceph-mon[29565]: 4.7 scrub ok 2024-09-17T08:34:17.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:16 smithi173 ceph-mon[33187]: 2.b scrub starts 2024-09-17T08:34:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:16 smithi173 ceph-mon[33187]: 2.b scrub ok 2024-09-17T08:34:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:16 smithi173 ceph-mon[33187]: pgmap v1108: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:16 smithi173 ceph-mon[33187]: 4.7 scrub starts 2024-09-17T08:34:17.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:16 smithi173 ceph-mon[33187]: 4.7 scrub ok 2024-09-17T08:34:18.116 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[29565]: 1.7 scrub starts 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[29565]: 1.7 scrub ok 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[29565]: 3.0 scrub starts 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[29565]: 3.0 scrub ok 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[40649]: 1.7 scrub starts 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[40649]: 1.7 scrub ok 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[40649]: 3.0 scrub starts 2024-09-17T08:34:18.117 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:17 smithi079 ceph-mon[40649]: 3.0 scrub ok 2024-09-17T08:34:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:17 smithi173 ceph-mon[33187]: 1.7 scrub starts 2024-09-17T08:34:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:17 smithi173 ceph-mon[33187]: 1.7 scrub ok 2024-09-17T08:34:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:17 smithi173 ceph-mon[33187]: 3.0 scrub starts 2024-09-17T08:34:18.125 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:17 smithi173 ceph-mon[33187]: 3.0 scrub ok 2024-09-17T08:34:18.983 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[29565]: 6.8 scrub starts 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[29565]: 6.8 scrub ok 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[29565]: pgmap v1109: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[40649]: 6.8 scrub starts 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[40649]: 6.8 scrub ok 2024-09-17T08:34:19.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[40649]: pgmap v1109: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:19.096 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:18 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:19.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:18 smithi173 ceph-mon[33187]: 6.8 scrub starts 2024-09-17T08:34:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:18 smithi173 ceph-mon[33187]: 6.8 scrub ok 2024-09-17T08:34:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:18 smithi173 ceph-mon[33187]: pgmap v1109: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:19.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:18 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 6.1c deep-scrub starts 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 6.1c deep-scrub ok 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 6.19 scrub starts 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 6.19 scrub ok 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 2.1a scrub starts 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 2.1a scrub ok 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 4.16 scrub starts 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[29565]: 4.16 scrub ok 2024-09-17T08:34:20.025 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 6.1c deep-scrub starts 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 6.1c deep-scrub ok 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 6.19 scrub starts 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 6.19 scrub ok 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 2.1a scrub starts 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 2.1a scrub ok 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 4.16 scrub starts 2024-09-17T08:34:20.026 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:19 smithi079 ceph-mon[40649]: 4.16 scrub ok 2024-09-17T08:34:20.201 INFO:teuthology.orchestra.run.smithi079.stdout:176093659445 2024-09-17T08:34:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 6.1c deep-scrub starts 2024-09-17T08:34:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 6.1c deep-scrub ok 2024-09-17T08:34:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 6.19 scrub starts 2024-09-17T08:34:20.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 6.19 scrub ok 2024-09-17T08:34:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 2.1a scrub starts 2024-09-17T08:34:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 2.1a scrub ok 2024-09-17T08:34:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 4.16 scrub starts 2024-09-17T08:34:20.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:19 smithi173 ceph-mon[33187]: 4.16 scrub ok 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 6.5 scrub starts 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 6.5 scrub ok 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 4.e scrub starts 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 4.e scrub ok 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 5.b scrub starts 2024-09-17T08:34:21.065 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 5.b scrub ok 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: pgmap v1110: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 2.4 scrub starts 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: 2.4 scrub ok 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/955427616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 6.5 scrub starts 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 6.5 scrub ok 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 4.e scrub starts 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 4.e scrub ok 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 5.b scrub starts 2024-09-17T08:34:21.066 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 5.b scrub ok 2024-09-17T08:34:21.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: pgmap v1110: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:21.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 2.4 scrub starts 2024-09-17T08:34:21.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: 2.4 scrub ok 2024-09-17T08:34:21.067 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:20 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/955427616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:34:21.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659442 got 176093659445 for osd.5 2024-09-17T08:34:21.086 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:21.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 6.5 scrub starts 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 6.5 scrub ok 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 4.e scrub starts 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 4.e scrub ok 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 5.b scrub starts 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 5.b scrub ok 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: pgmap v1110: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 2.4 scrub starts 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: 2.4 scrub ok 2024-09-17T08:34:21.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:20 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/955427616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[29565]: 4.18 scrub starts 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[29565]: 4.18 scrub ok 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[29565]: 6.10 scrub starts 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[29565]: 6.10 scrub ok 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[40649]: 4.18 scrub starts 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[40649]: 4.18 scrub ok 2024-09-17T08:34:21.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[40649]: 6.10 scrub starts 2024-09-17T08:34:21.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:21 smithi079 ceph-mon[40649]: 6.10 scrub ok 2024-09-17T08:34:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:21 smithi173 ceph-mon[33187]: 4.18 scrub starts 2024-09-17T08:34:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:21 smithi173 ceph-mon[33187]: 4.18 scrub ok 2024-09-17T08:34:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:21 smithi173 ceph-mon[33187]: 6.10 scrub starts 2024-09-17T08:34:22.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:21 smithi173 ceph-mon[33187]: 6.10 scrub ok 2024-09-17T08:34:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:115964117310 2024-09-17T08:34:23.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: 5.4 deep-scrub starts 2024-09-17T08:34:23.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: 5.4 deep-scrub ok 2024-09-17T08:34:23.027 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: 4.1b scrub starts 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: 4.1b scrub ok 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: pgmap v1111: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/2137672703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: 5.4 deep-scrub starts 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: 5.4 deep-scrub ok 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: 4.1b scrub starts 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: 4.1b scrub ok 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: pgmap v1111: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:23.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:22 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/2137672703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:34:23.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: 5.4 deep-scrub starts 2024-09-17T08:34:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: 5.4 deep-scrub ok 2024-09-17T08:34:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: 4.1b scrub starts 2024-09-17T08:34:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: 4.1b scrub ok 2024-09-17T08:34:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: pgmap v1111: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:23.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:22 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/2137672703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T08:34:23.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:34:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:23.725 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:23.725 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 4.15 scrub starts 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 4.15 scrub ok 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 4.1 scrub starts 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 4.1 scrub ok 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 3.1 scrub starts 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[40649]: 3.1 scrub ok 2024-09-17T08:34:23.972 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 4.15 scrub starts 2024-09-17T08:34:23.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 4.15 scrub ok 2024-09-17T08:34:23.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 4.1 scrub starts 2024-09-17T08:34:23.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 4.1 scrub ok 2024-09-17T08:34:23.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 3.1 scrub starts 2024-09-17T08:34:23.973 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:23 smithi079 ceph-mon[29565]: 3.1 scrub ok 2024-09-17T08:34:23.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 4.15 scrub starts 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 4.15 scrub ok 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 4.1 scrub starts 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 4.1 scrub ok 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 3.1 scrub starts 2024-09-17T08:34:23.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:23 smithi173 ceph-mon[33187]: 3.1 scrub ok 2024-09-17T08:34:24.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117306 got 115964117310 for osd.3 2024-09-17T08:34:24.626 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:24.872 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: 5.a deep-scrub starts 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: 5.a deep-scrub ok 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: pgmap v1112: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: 4.b deep-scrub starts 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[40649]: 4.b deep-scrub ok 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: 5.a deep-scrub starts 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: 5.a deep-scrub ok 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:24.873 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: pgmap v1112: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:24.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: 4.b deep-scrub starts 2024-09-17T08:34:24.874 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:24 smithi079 ceph-mon[29565]: 4.b deep-scrub ok 2024-09-17T08:34:25.069 INFO:teuthology.orchestra.run.smithi079.stdout:253403070764 2024-09-17T08:34:25.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: 5.a deep-scrub starts 2024-09-17T08:34:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: 5.a deep-scrub ok 2024-09-17T08:34:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: pgmap v1112: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: 4.b deep-scrub starts 2024-09-17T08:34:25.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:24 smithi173 ceph-mon[33187]: 4.b deep-scrub ok 2024-09-17T08:34:25.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:25 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/775251764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:34:25.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:25 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/775251764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:34:26.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:25 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/775251764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[40649]: 1.0 scrub starts 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[40649]: 1.0 scrub ok 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[40649]: pgmap v1113: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[29565]: 1.0 scrub starts 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[29565]: 1.0 scrub ok 2024-09-17T08:34:27.058 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:26 smithi079 ceph-mon[29565]: pgmap v1113: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:26 smithi173 ceph-mon[33187]: 1.0 scrub starts 2024-09-17T08:34:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:26 smithi173 ceph-mon[33187]: 1.0 scrub ok 2024-09-17T08:34:27.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:26 smithi173 ceph-mon[33187]: pgmap v1113: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 2.12 scrub starts 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 2.12 scrub ok 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 2.11 scrub starts 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 2.11 scrub ok 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 6.0 scrub starts 2024-09-17T08:34:27.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[40649]: 6.0 scrub ok 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 2.12 scrub starts 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 2.12 scrub ok 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 2.11 scrub starts 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 2.11 scrub ok 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 6.0 scrub starts 2024-09-17T08:34:27.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:27 smithi079 ceph-mon[29565]: 6.0 scrub ok 2024-09-17T08:34:28.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070760 got 253403070764 for osd.7 2024-09-17T08:34:28.165 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:28.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 2.12 scrub starts 2024-09-17T08:34:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 2.12 scrub ok 2024-09-17T08:34:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 2.11 scrub starts 2024-09-17T08:34:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 2.11 scrub ok 2024-09-17T08:34:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 6.0 scrub starts 2024-09-17T08:34:28.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:27 smithi173 ceph-mon[33187]: 6.0 scrub ok 2024-09-17T08:34:28.530 INFO:teuthology.orchestra.run.smithi079.stdout:34359738699 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: 6.12 scrub starts 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: 6.12 scrub ok 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: 2.f scrub starts 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: 2.f scrub ok 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: pgmap v1114: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.045 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/781366068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: 6.12 scrub starts 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: 6.12 scrub ok 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: 2.f scrub starts 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: 2.f scrub ok 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: pgmap v1114: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.046 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:28 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/781366068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:34:29.131 INFO:teuthology.orchestra.run.smithi079.stdout:55834575177 2024-09-17T08:34:29.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: 6.12 scrub starts 2024-09-17T08:34:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: 6.12 scrub ok 2024-09-17T08:34:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: 2.f scrub starts 2024-09-17T08:34:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: 2.f scrub ok 2024-09-17T08:34:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: pgmap v1114: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:29.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:28 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/781366068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T08:34:29.299 INFO:teuthology.orchestra.run.smithi079.stdout:1864015806489 2024-09-17T08:34:29.875 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575174 got 55834575177 for osd.1 2024-09-17T08:34:29.876 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:30.114 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[40649]: 5.1f scrub starts 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[40649]: 5.1f scrub ok 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3729590822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3697561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[29565]: 5.1f scrub starts 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[29565]: 5.1f scrub ok 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3729590822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:34:30.115 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:29 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3697561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:34:30.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:29 smithi173 ceph-mon[33187]: 5.1f scrub starts 2024-09-17T08:34:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:29 smithi173 ceph-mon[33187]: 5.1f scrub ok 2024-09-17T08:34:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3729590822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T08:34:30.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:29 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3697561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T08:34:30.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 1864015806485 got 1864015806489 for osd.6 2024-09-17T08:34:30.794 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:31.063 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:30 smithi079 ceph-mon[29565]: pgmap v1115: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:31.063 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:30 smithi079 ceph-mon[40649]: pgmap v1115: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:31.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:30 smithi173 ceph-mon[33187]: pgmap v1115: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:31.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738696 got 34359738699 for osd.0 2024-09-17T08:34:31.622 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 2.c deep-scrub starts 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 2.c deep-scrub ok 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 4.8 deep-scrub starts 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 4.8 deep-scrub ok 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 4.10 deep-scrub starts 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 4.10 deep-scrub ok 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 2.1e scrub starts 2024-09-17T08:34:32.028 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[40649]: 2.1e scrub ok 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 2.c deep-scrub starts 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 2.c deep-scrub ok 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 4.8 deep-scrub starts 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 4.8 deep-scrub ok 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 4.10 deep-scrub starts 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 4.10 deep-scrub ok 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 2.1e scrub starts 2024-09-17T08:34:32.029 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:31 smithi079 ceph-mon[29565]: 2.1e scrub ok 2024-09-17T08:34:32.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 2.c deep-scrub starts 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 2.c deep-scrub ok 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 4.8 deep-scrub starts 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 4.8 deep-scrub ok 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 4.10 deep-scrub starts 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 4.10 deep-scrub ok 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 2.1e scrub starts 2024-09-17T08:34:32.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:31 smithi173 ceph-mon[33187]: 2.1e scrub ok 2024-09-17T08:34:32.286 INFO:teuthology.orchestra.run.smithi079.stdout:137438953788 2024-09-17T08:34:32.534 INFO:teuthology.orchestra.run.smithi079.stdout:974957576363 2024-09-17T08:34:33.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:34:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[29565]: 2.15 scrub starts 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[29565]: 2.15 scrub ok 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[29565]: pgmap v1116: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/3789166296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/613325419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:34:33.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[40649]: 2.15 scrub starts 2024-09-17T08:34:33.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[40649]: 2.15 scrub ok 2024-09-17T08:34:33.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[40649]: pgmap v1116: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:33.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/3789166296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:34:33.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:32 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/613325419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:34:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:32 smithi173 ceph-mon[33187]: 2.15 scrub starts 2024-09-17T08:34:33.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:32 smithi173 ceph-mon[33187]: 2.15 scrub ok 2024-09-17T08:34:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:32 smithi173 ceph-mon[33187]: pgmap v1116: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T08:34:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/3789166296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T08:34:33.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:32 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/613325419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T08:34:33.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 974957576359 got 974957576363 for osd.2 2024-09-17T08:34:33.340 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:33.837 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953784 got 137438953788 for osd.4 2024-09-17T08:34:33.838 DEBUG:teuthology.parallel:result is None 2024-09-17T08:34:33.838 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T08:34:33.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:34:33.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 4.1c scrub starts 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 4.1c scrub ok 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 5.1a deep-scrub starts 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 5.1a deep-scrub ok 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 4.6 scrub starts 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: 4.6 scrub ok 2024-09-17T08:34:33.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T08:34:33.975 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:33 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:34.040 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:34:34.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 4.1c scrub starts 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 4.1c scrub ok 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 5.1a deep-scrub starts 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 5.1a deep-scrub ok 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 4.6 scrub starts 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: 4.6 scrub ok 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:34:34.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 4.1c scrub starts 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 4.1c scrub ok 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 5.1a deep-scrub starts 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 5.1a deep-scrub ok 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 4.6 scrub starts 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: 4.6 scrub ok 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T08:34:34.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:33 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T08:34:35.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: 2.17 deep-scrub starts 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: 2.17 deep-scrub ok 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: pgmap v1118: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: purged_snaps scrub starts 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[29565]: purged_snaps scrub ok 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: 2.17 deep-scrub starts 2024-09-17T08:34:35.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: 2.17 deep-scrub ok 2024-09-17T08:34:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: pgmap v1118: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: purged_snaps scrub starts 2024-09-17T08:34:35.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:34 smithi079 ceph-mon[40649]: purged_snaps scrub ok 2024-09-17T08:34:35.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: 2.17 deep-scrub starts 2024-09-17T08:34:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: 2.17 deep-scrub ok 2024-09-17T08:34:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: from='client.24415 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T08:34:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: pgmap v1118: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T08:34:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: purged_snaps scrub starts 2024-09-17T08:34:35.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:34 smithi173 ceph-mon[33187]: purged_snaps scrub ok 2024-09-17T08:34:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 6.f scrub starts 2024-09-17T08:34:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 6.f scrub ok 2024-09-17T08:34:35.869 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.16 scrub starts 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.16 scrub ok 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.11 scrub starts 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.11 scrub ok 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.1c scrub starts 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: 5.1c scrub ok 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[40649]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 6.f scrub starts 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 6.f scrub ok 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.16 scrub starts 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.16 scrub ok 2024-09-17T08:34:35.870 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.11 scrub starts 2024-09-17T08:34:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.11 scrub ok 2024-09-17T08:34:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.1c scrub starts 2024-09-17T08:34:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: 5.1c scrub ok 2024-09-17T08:34:35.871 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:35 smithi079 ceph-mon[29565]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T08:34:35.992 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:34:35.994 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 6.f scrub starts 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 6.f scrub ok 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.16 scrub starts 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.16 scrub ok 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.11 scrub starts 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.11 scrub ok 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.1c scrub starts 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: 5.1c scrub ok 2024-09-17T08:34:36.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:35 smithi173 ceph-mon[33187]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T08:34:36.798 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":1120,"stamp":"2024-09-17T08:34:35.742540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2684,"num_read_kb":2436,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":8551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2634,"ondisk_log_size":2634,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5111504,"kb_used_data":8644,"kb_used_omap":18,"kb_used_meta":5102765,"kb_avail":744849712,"statfs":{"total":767960285184,"available":762726105088,"internally_reserved":0,"allocated":8851456,"data_stored":5963125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18443,"internal_metadata":5225232373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002055"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941384+0000","last_change":"2024-09-17T08:34:02.793177+0000","last_active":"2024-09-17T08:34:13.941384+0000","last_peered":"2024-09-17T08:34:13.941384+0000","last_clean":"2024-09-17T08:34:13.941384+0000","last_became_active":"2024-09-17T08:32:43.740289+0000","last_became_peered":"2024-09-17T08:32:43.740289+0000","last_unstale":"2024-09-17T08:34:13.941384+0000","last_undegraded":"2024-09-17T08:34:13.941384+0000","last_fullsized":"2024-09-17T08:34:13.941384+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:02.793098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:34:02.793098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:09.068341+0000","scrub_duration":0.001793773,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":545,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621593+0000","last_change":"2024-09-17T08:33:37.625623+0000","last_active":"2024-09-17T08:33:56.621593+0000","last_peered":"2024-09-17T08:33:56.621593+0000","last_clean":"2024-09-17T08:33:56.621593+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:33:56.621593+0000","last_undegraded":"2024-09-17T08:33:56.621593+0000","last_fullsized":"2024-09-17T08:33:56.621593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:37.625546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:37.625546+0000","last_clean_scrub_stamp":"2024-09-17T08:33:37.625546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.915425+0000","scrub_duration":0.0015847700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"449'46","reported_seq":76,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941951+0000","last_change":"2024-09-17T08:33:20.979714+0000","last_active":"2024-09-17T08:34:13.941951+0000","last_peered":"2024-09-17T08:34:13.941951+0000","last_clean":"2024-09-17T08:34:13.941951+0000","last_became_active":"2024-09-17T08:32:43.735736+0000","last_became_peered":"2024-09-17T08:32:43.735736+0000","last_unstale":"2024-09-17T08:34:13.941951+0000","last_undegraded":"2024-09-17T08:34:13.941951+0000","last_fullsized":"2024-09-17T08:34:13.941951+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"449'46","last_scrub_stamp":"2024-09-17T08:33:20.979618+0000","last_deep_scrub":"449'46","last_deep_scrub_stamp":"2024-09-17T08:33:20.979618+0000","last_clean_scrub_stamp":"2024-09-17T08:33:20.979618+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.412419+0000","scrub_duration":0.0056654280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":563,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748263+0000","last_change":"2024-09-17T08:34:11.211850+0000","last_active":"2024-09-17T08:34:13.748263+0000","last_peered":"2024-09-17T08:34:13.748263+0000","last_clean":"2024-09-17T08:34:13.748263+0000","last_became_active":"2024-09-17T08:32:32.655817+0000","last_became_peered":"2024-09-17T08:32:32.655817+0000","last_unstale":"2024-09-17T08:34:13.748263+0000","last_undegraded":"2024-09-17T08:34:13.748263+0000","last_fullsized":"2024-09-17T08:34:13.748263+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.211748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.211748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.616905+0000","scrub_duration":0.0013268399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.585175+0000","last_change":"2024-09-17T08:34:30.585175+0000","last_active":"2024-09-17T08:34:30.585175+0000","last_peered":"2024-09-17T08:34:30.585175+0000","last_clean":"2024-09-17T08:34:30.585175+0000","last_became_active":"2024-09-17T08:32:43.724367+0000","last_became_peered":"2024-09-17T08:32:43.724367+0000","last_unstale":"2024-09-17T08:34:30.585175+0000","last_undegraded":"2024-09-17T08:34:30.585175+0000","last_fullsized":"2024-09-17T08:34:30.585175+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:30.585076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.585076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:58.518150+0000","scrub_duration":0.001590901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"389'25","reported_seq":723,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.274650+0000","last_change":"2024-09-17T08:34:20.274650+0000","last_active":"2024-09-17T08:34:20.274650+0000","last_peered":"2024-09-17T08:34:20.274650+0000","last_clean":"2024-09-17T08:34:20.274650+0000","last_became_active":"2024-09-17T08:32:32.657238+0000","last_became_peered":"2024-09-17T08:32:32.657238+0000","last_unstale":"2024-09-17T08:34:20.274650+0000","last_undegraded":"2024-09-17T08:34:20.274650+0000","last_fullsized":"2024-09-17T08:34:20.274650+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:20.274538+0000","last_deep_scrub":"389'25","last_deep_scrub_stamp":"2024-09-17T08:33:00.607405+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.274538+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:45.475985+0000","scrub_duration":0.002096868,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":412,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751811+0000","last_change":"2024-09-17T08:33:27.421746+0000","last_active":"2024-09-17T08:34:13.751811+0000","last_peered":"2024-09-17T08:34:13.751811+0000","last_clean":"2024-09-17T08:34:13.751811+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:34:13.751811+0000","last_undegraded":"2024-09-17T08:34:13.751811+0000","last_fullsized":"2024-09-17T08:34:13.751811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:27.421667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:27.421667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.079946+0000","scrub_duration":0.0015319369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":545,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748478+0000","last_change":"2024-09-17T08:33:19.427099+0000","last_active":"2024-09-17T08:34:13.748478+0000","last_peered":"2024-09-17T08:34:13.748478+0000","last_clean":"2024-09-17T08:34:13.748478+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:34:13.748478+0000","last_undegraded":"2024-09-17T08:34:13.748478+0000","last_fullsized":"2024-09-17T08:34:13.748478+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:19.427014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:33:19.427014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:46.542638+0000","scrub_duration":0.0015870909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":568,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006250+0000","last_change":"2024-09-17T08:33:56.027534+0000","last_active":"2024-09-17T08:33:57.006250+0000","last_peered":"2024-09-17T08:33:57.006250+0000","last_clean":"2024-09-17T08:33:57.006250+0000","last_became_active":"2024-09-17T08:32:32.662767+0000","last_became_peered":"2024-09-17T08:32:32.662767+0000","last_unstale":"2024-09-17T08:33:57.006250+0000","last_undegraded":"2024-09-17T08:33:57.006250+0000","last_fullsized":"2024-09-17T08:33:57.006250+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:56.027450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.027450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.835214+0000","scrub_duration":0.001371197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"389'11","reported_seq":609,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:21.248269+0000","last_change":"2024-09-17T08:34:21.248269+0000","last_active":"2024-09-17T08:34:21.248269+0000","last_peered":"2024-09-17T08:34:21.248269+0000","last_clean":"2024-09-17T08:34:21.248269+0000","last_became_active":"2024-09-17T08:32:43.729898+0000","last_became_peered":"2024-09-17T08:32:43.729898+0000","last_unstale":"2024-09-17T08:34:21.248269+0000","last_undegraded":"2024-09-17T08:34:21.248269+0000","last_fullsized":"2024-09-17T08:34:21.248269+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'11","last_scrub_stamp":"2024-09-17T08:34:21.248164+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:34:21.248164+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.360498+0000","scrub_duration":0.0019473509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":589,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.162251+0000","last_change":"2024-09-17T08:34:32.162330+0000","last_active":"2024-09-17T08:34:32.162251+0000","last_peered":"2024-09-17T08:34:32.162251+0000","last_clean":"2024-09-17T08:34:32.162251+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:34:32.162251+0000","last_undegraded":"2024-09-17T08:34:32.162251+0000","last_fullsized":"2024-09-17T08:34:32.162251+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:32.162234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:32.162234+0000","last_clean_scrub_stamp":"2024-09-17T08:34:32.162234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.055336+0000","scrub_duration":0.001400957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":416,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.149569+0000","last_change":"2024-09-17T08:34:18.149654+0000","last_active":"2024-09-17T08:34:18.149569+0000","last_peered":"2024-09-17T08:34:18.149569+0000","last_clean":"2024-09-17T08:34:18.149569+0000","last_became_active":"2024-09-17T08:33:29.919824+0000","last_became_peered":"2024-09-17T08:33:29.919824+0000","last_unstale":"2024-09-17T08:34:18.149569+0000","last_undegraded":"2024-09-17T08:34:18.149569+0000","last_fullsized":"2024-09-17T08:34:18.149569+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:18.149552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.149552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:44.933358+0000","scrub_duration":0.0013843569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":581,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.043536+0000","last_change":"2024-09-17T08:34:13.043536+0000","last_active":"2024-09-17T08:34:13.043536+0000","last_peered":"2024-09-17T08:34:13.043536+0000","last_clean":"2024-09-17T08:34:13.043536+0000","last_became_active":"2024-09-17T08:33:35.947238+0000","last_became_peered":"2024-09-17T08:33:35.947238+0000","last_unstale":"2024-09-17T08:34:13.043536+0000","last_undegraded":"2024-09-17T08:34:13.043536+0000","last_fullsized":"2024-09-17T08:34:13.043536+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:13.043424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:31:43.907526+0000","last_clean_scrub_stamp":"2024-09-17T08:34:13.043424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.360395+0000","scrub_duration":0.001518716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"389'25","reported_seq":607,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748854+0000","last_change":"2024-09-17T08:33:20.460326+0000","last_active":"2024-09-17T08:34:13.748854+0000","last_peered":"2024-09-17T08:34:13.748854+0000","last_clean":"2024-09-17T08:34:13.748854+0000","last_became_active":"2024-09-17T08:32:43.716624+0000","last_became_peered":"2024-09-17T08:32:43.716624+0000","last_unstale":"2024-09-17T08:34:13.748854+0000","last_undegraded":"2024-09-17T08:34:13.748854+0000","last_fullsized":"2024-09-17T08:34:13.748854+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:33:20.460237+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:33:20.460237+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:43.172916+0000","scrub_duration":0.0020359200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:00.160240+0000","last_change":"2024-09-17T08:34:00.160240+0000","last_active":"2024-09-17T08:34:00.160240+0000","last_peered":"2024-09-17T08:34:00.160240+0000","last_clean":"2024-09-17T08:34:00.160240+0000","last_became_active":"2024-09-17T08:33:29.926134+0000","last_became_peered":"2024-09-17T08:33:29.926134+0000","last_unstale":"2024-09-17T08:34:00.160240+0000","last_undegraded":"2024-09-17T08:34:00.160240+0000","last_fullsized":"2024-09-17T08:34:00.160240+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:00.160141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:00.160141+0000","last_clean_scrub_stamp":"2024-09-17T08:34:00.160141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:01.181743+0000","scrub_duration":0.0013840759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":556,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.940985+0000","last_change":"2024-09-17T08:34:08.656708+0000","last_active":"2024-09-17T08:34:13.940985+0000","last_peered":"2024-09-17T08:34:13.940985+0000","last_clean":"2024-09-17T08:34:13.940985+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:34:13.940985+0000","last_undegraded":"2024-09-17T08:34:13.940985+0000","last_fullsized":"2024-09-17T08:34:13.940985+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:08.656623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:08.656623+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.656623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:27.881160+0000","scrub_duration":0.0013164769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"75'130","reported_seq":334,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.525693+0000","last_change":"2024-09-17T08:33:38.946466+0000","last_active":"2024-09-17T08:34:07.525693+0000","last_peered":"2024-09-17T08:34:07.525693+0000","last_clean":"2024-09-17T08:34:07.525693+0000","last_became_active":"2024-09-17T08:33:29.922540+0000","last_became_peered":"2024-09-17T08:33:29.922540+0000","last_unstale":"2024-09-17T08:34:07.525693+0000","last_undegraded":"2024-09-17T08:34:07.525693+0000","last_fullsized":"2024-09-17T08:34:07.525693+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:38.946332+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:04.082189+0000","last_clean_scrub_stamp":"2024-09-17T08:33:38.946332+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.611131+0000","scrub_duration":0.0029773289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"441'53","reported_seq":696,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.126664+0000","last_change":"2024-09-17T08:34:14.126664+0000","last_active":"2024-09-17T08:34:14.126664+0000","last_peered":"2024-09-17T08:34:14.126664+0000","last_clean":"2024-09-17T08:34:14.126664+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:34:14.126664+0000","last_undegraded":"2024-09-17T08:34:14.126664+0000","last_fullsized":"2024-09-17T08:34:14.126664+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'53","last_scrub_stamp":"2024-09-17T08:34:14.126538+0000","last_deep_scrub":"441'53","last_deep_scrub_stamp":"2024-09-17T08:32:48.696251+0000","last_clean_scrub_stamp":"2024-09-17T08:34:14.126538+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.522869+0000","scrub_duration":0.0029383730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":492,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:15.105153+0000","last_change":"2024-09-17T08:34:15.105241+0000","last_active":"2024-09-17T08:34:15.105153+0000","last_peered":"2024-09-17T08:34:15.105153+0000","last_clean":"2024-09-17T08:34:15.105153+0000","last_became_active":"2024-09-17T08:33:35.943108+0000","last_became_peered":"2024-09-17T08:33:35.943108+0000","last_unstale":"2024-09-17T08:34:15.105153+0000","last_undegraded":"2024-09-17T08:34:15.105153+0000","last_fullsized":"2024-09-17T08:34:15.105153+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:15.105136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:15.105136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.748291+0000","scrub_duration":0.0016169859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":424,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752168+0000","last_change":"2024-09-17T08:33:48.336927+0000","last_active":"2024-09-17T08:34:13.752168+0000","last_peered":"2024-09-17T08:34:13.752168+0000","last_clean":"2024-09-17T08:34:13.752168+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:34:13.752168+0000","last_undegraded":"2024-09-17T08:34:13.752168+0000","last_fullsized":"2024-09-17T08:34:13.752168+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:48.336829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.336829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:14.362638+0000","scrub_duration":0.001767219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":507,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006198+0000","last_change":"2024-09-17T08:33:43.628804+0000","last_active":"2024-09-17T08:33:57.006198+0000","last_peered":"2024-09-17T08:33:57.006198+0000","last_clean":"2024-09-17T08:33:57.006198+0000","last_became_active":"2024-09-17T08:33:43.628507+0000","last_became_peered":"2024-09-17T08:33:43.628507+0000","last_unstale":"2024-09-17T08:33:57.006198+0000","last_undegraded":"2024-09-17T08:33:57.006198+0000","last_fullsized":"2024-09-17T08:33:57.006198+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:11.020257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:11.020257+0000","last_clean_scrub_stamp":"2024-09-17T08:33:11.020257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:33.185557+0000","scrub_duration":0.0014410639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'130","reported_seq":830,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.005407+0000","last_change":"2024-09-17T08:33:40.996286+0000","last_active":"2024-09-17T08:33:57.005407+0000","last_peered":"2024-09-17T08:33:57.005407+0000","last_clean":"2024-09-17T08:33:57.005407+0000","last_became_active":"2024-09-17T08:33:35.945893+0000","last_became_peered":"2024-09-17T08:33:35.945893+0000","last_unstale":"2024-09-17T08:33:57.005407+0000","last_undegraded":"2024-09-17T08:33:57.005407+0000","last_fullsized":"2024-09-17T08:33:57.005407+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:40.996158+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:40.996158+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:03.023145+0000","scrub_duration":0.0020294979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"389'35","reported_seq":639,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748701+0000","last_change":"2024-09-17T08:33:33.388095+0000","last_active":"2024-09-17T08:34:13.748701+0000","last_peered":"2024-09-17T08:34:13.748701+0000","last_clean":"2024-09-17T08:34:13.748701+0000","last_became_active":"2024-09-17T08:32:43.723164+0000","last_became_peered":"2024-09-17T08:32:43.723164+0000","last_unstale":"2024-09-17T08:34:13.748701+0000","last_undegraded":"2024-09-17T08:34:13.748701+0000","last_fullsized":"2024-09-17T08:34:13.748701+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'35","last_scrub_stamp":"2024-09-17T08:33:33.387988+0000","last_deep_scrub":"389'35","last_deep_scrub_stamp":"2024-09-17T08:31:35.795025+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.387988+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:46.096005+0000","scrub_duration":0.0021778019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751941+0000","last_change":"2024-09-17T08:33:26.451014+0000","last_active":"2024-09-17T08:34:13.751941+0000","last_peered":"2024-09-17T08:34:13.751941+0000","last_clean":"2024-09-17T08:34:13.751941+0000","last_became_active":"2024-09-17T08:32:43.724631+0000","last_became_peered":"2024-09-17T08:32:43.724631+0000","last_unstale":"2024-09-17T08:34:13.751941+0000","last_undegraded":"2024-09-17T08:34:13.751941+0000","last_fullsized":"2024-09-17T08:34:13.751941+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:26.450926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:33:26.450926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:56.297495+0000","scrub_duration":0.001529709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"461'14","reported_seq":796,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122156+0000","last_change":"2024-09-17T08:33:43.627521+0000","last_active":"2024-09-17T08:34:23.122156+0000","last_peered":"2024-09-17T08:34:23.122156+0000","last_clean":"2024-09-17T08:34:23.122156+0000","last_became_active":"2024-09-17T08:33:43.626800+0000","last_became_peered":"2024-09-17T08:33:43.626800+0000","last_unstale":"2024-09-17T08:34:23.122156+0000","last_undegraded":"2024-09-17T08:34:23.122156+0000","last_fullsized":"2024-09-17T08:34:23.122156+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:33:33.074745+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.074745+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.351318+0000","scrub_duration":0.00170889,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":551,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748932+0000","last_change":"2024-09-17T08:33:41.379831+0000","last_active":"2024-09-17T08:34:13.748932+0000","last_peered":"2024-09-17T08:34:13.748932+0000","last_clean":"2024-09-17T08:34:13.748932+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:34:13.748932+0000","last_undegraded":"2024-09-17T08:34:13.748932+0000","last_fullsized":"2024-09-17T08:34:13.748932+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:41.379728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:33:41.379728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.381031+0000","scrub_duration":0.001680792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'130","reported_seq":334,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.807175+0000","last_change":"2024-09-17T08:33:39.960080+0000","last_active":"2024-09-17T08:34:07.807175+0000","last_peered":"2024-09-17T08:34:07.807175+0000","last_clean":"2024-09-17T08:34:07.807175+0000","last_became_active":"2024-09-17T08:33:29.924947+0000","last_became_peered":"2024-09-17T08:33:29.924947+0000","last_unstale":"2024-09-17T08:34:07.807175+0000","last_undegraded":"2024-09-17T08:34:07.807175+0000","last_fullsized":"2024-09-17T08:34:07.807175+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:39.959922+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:39.959922+0000","last_clean_scrub_stamp":"2024-09-17T08:33:39.959922+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.413246+0000","scrub_duration":0.0015588970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"390'46","reported_seq":412,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:06.322261+0000","last_change":"2024-09-17T08:34:06.322261+0000","last_active":"2024-09-17T08:34:06.322261+0000","last_peered":"2024-09-17T08:34:06.322261+0000","last_clean":"2024-09-17T08:34:06.322261+0000","last_became_active":"2024-09-17T08:33:43.628298+0000","last_became_peered":"2024-09-17T08:33:43.628298+0000","last_unstale":"2024-09-17T08:34:06.322261+0000","last_undegraded":"2024-09-17T08:34:06.322261+0000","last_fullsized":"2024-09-17T08:34:06.322261+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"390'46","last_scrub_stamp":"2024-09-17T08:34:06.322182+0000","last_deep_scrub":"390'46","last_deep_scrub_stamp":"2024-09-17T08:32:48.046645+0000","last_clean_scrub_stamp":"2024-09-17T08:34:06.322182+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.119548+0000","scrub_duration":0.0024513059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751748+0000","last_change":"2024-09-17T08:33:11.641804+0000","last_active":"2024-09-17T08:34:13.751748+0000","last_peered":"2024-09-17T08:34:13.751748+0000","last_clean":"2024-09-17T08:34:13.751748+0000","last_became_active":"2024-09-17T08:32:43.739708+0000","last_became_peered":"2024-09-17T08:32:43.739708+0000","last_unstale":"2024-09-17T08:34:13.751748+0000","last_undegraded":"2024-09-17T08:34:13.751748+0000","last_fullsized":"2024-09-17T08:34:13.751748+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:11.641709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:11.641709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001805903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":834,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:28.123872+0000","last_change":"2024-09-17T08:34:12.169517+0000","last_active":"2024-09-17T08:34:28.123872+0000","last_peered":"2024-09-17T08:34:28.123872+0000","last_clean":"2024-09-17T08:34:28.123872+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:34:28.123872+0000","last_undegraded":"2024-09-17T08:34:28.123872+0000","last_fullsized":"2024-09-17T08:34:28.123872+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:34:12.169395+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:32:26.893190+0000","last_clean_scrub_stamp":"2024-09-17T08:34:12.169395+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.055112+0000","scrub_duration":0.001885255,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":130,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621637+0000","last_change":"2024-09-17T08:33:29.923536+0000","last_active":"2024-09-17T08:33:56.621637+0000","last_peered":"2024-09-17T08:33:56.621637+0000","last_clean":"2024-09-17T08:33:56.621637+0000","last_became_active":"2024-09-17T08:33:29.923207+0000","last_became_peered":"2024-09-17T08:33:29.923207+0000","last_unstale":"2024-09-17T08:33:56.621637+0000","last_undegraded":"2024-09-17T08:33:56.621637+0000","last_fullsized":"2024-09-17T08:33:56.621637+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:18.584274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:18.584274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:21.742281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"389'32","reported_seq":638,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.225988+0000","last_change":"2024-09-17T08:34:19.225988+0000","last_active":"2024-09-17T08:34:19.225988+0000","last_peered":"2024-09-17T08:34:19.225988+0000","last_clean":"2024-09-17T08:34:19.225988+0000","last_became_active":"2024-09-17T08:32:32.650732+0000","last_became_peered":"2024-09-17T08:32:32.650732+0000","last_unstale":"2024-09-17T08:34:19.225988+0000","last_undegraded":"2024-09-17T08:34:19.225988+0000","last_fullsized":"2024-09-17T08:34:19.225988+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'32","last_scrub_stamp":"2024-09-17T08:34:19.225854+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.225854+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:43.080057+0000","scrub_duration":0.0022183519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":499,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006461+0000","last_change":"2024-09-17T08:33:55.012426+0000","last_active":"2024-09-17T08:33:57.006461+0000","last_peered":"2024-09-17T08:33:57.006461+0000","last_clean":"2024-09-17T08:33:57.006461+0000","last_became_active":"2024-09-17T08:33:35.942593+0000","last_became_peered":"2024-09-17T08:33:35.942593+0000","last_unstale":"2024-09-17T08:33:57.006461+0000","last_undegraded":"2024-09-17T08:33:57.006461+0000","last_fullsized":"2024-09-17T08:33:57.006461+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:55.012273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:40.836520+0000","last_clean_scrub_stamp":"2024-09-17T08:33:55.012273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.482273+0000","scrub_duration":0.0030888970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":409,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622035+0000","last_change":"2024-09-17T08:33:30.129443+0000","last_active":"2024-09-17T08:33:56.622035+0000","last_peered":"2024-09-17T08:33:56.622035+0000","last_clean":"2024-09-17T08:33:56.622035+0000","last_became_active":"2024-09-17T08:33:30.128925+0000","last_became_peered":"2024-09-17T08:33:30.128925+0000","last_unstale":"2024-09-17T08:33:56.622035+0000","last_undegraded":"2024-09-17T08:33:56.622035+0000","last_fullsized":"2024-09-17T08:33:56.622035+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:02.592944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:33:02.592944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:27.673186+0000","scrub_duration":0.0089623930000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941933+0000","last_change":"2024-09-17T08:33:37.809677+0000","last_active":"2024-09-17T08:34:13.941933+0000","last_peered":"2024-09-17T08:34:13.941933+0000","last_clean":"2024-09-17T08:34:13.941933+0000","last_became_active":"2024-09-17T08:32:43.729208+0000","last_became_peered":"2024-09-17T08:32:43.729208+0000","last_unstale":"2024-09-17T08:34:13.941933+0000","last_undegraded":"2024-09-17T08:34:13.941933+0000","last_fullsized":"2024-09-17T08:34:13.941933+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:37.809568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:37.809568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.772366+0000","scrub_duration":0.001885627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'130","reported_seq":351,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.942015+0000","last_change":"2024-09-17T08:33:39.785292+0000","last_active":"2024-09-17T08:34:13.942015+0000","last_peered":"2024-09-17T08:34:13.942015+0000","last_clean":"2024-09-17T08:34:13.942015+0000","last_became_active":"2024-09-17T08:32:46.735714+0000","last_became_peered":"2024-09-17T08:32:46.735714+0000","last_unstale":"2024-09-17T08:34:13.942015+0000","last_undegraded":"2024-09-17T08:34:13.942015+0000","last_fullsized":"2024-09-17T08:34:13.942015+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:39.785202+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_clean_scrub_stamp":"2024-09-17T08:33:39.785202+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.471841+0000","scrub_duration":0.0023755920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"389'49","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.241886+0000","last_change":"2024-09-17T08:34:22.241886+0000","last_active":"2024-09-17T08:34:22.241886+0000","last_peered":"2024-09-17T08:34:22.241886+0000","last_clean":"2024-09-17T08:34:22.241886+0000","last_became_active":"2024-09-17T08:32:32.659219+0000","last_became_peered":"2024-09-17T08:32:32.659219+0000","last_unstale":"2024-09-17T08:34:22.241886+0000","last_undegraded":"2024-09-17T08:34:22.241886+0000","last_fullsized":"2024-09-17T08:34:22.241886+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'49","last_scrub_stamp":"2024-09-17T08:34:22.241752+0000","last_deep_scrub":"349'35","last_deep_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.241752+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:44.566917+0000","scrub_duration":0.0023798640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":389,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128268+0000","last_change":"2024-09-17T08:33:59.514914+0000","last_active":"2024-09-17T08:34:33.128268+0000","last_peered":"2024-09-17T08:34:33.128268+0000","last_clean":"2024-09-17T08:34:33.128268+0000","last_became_active":"2024-09-17T08:32:32.667307+0000","last_became_peered":"2024-09-17T08:32:32.667307+0000","last_unstale":"2024-09-17T08:34:33.128268+0000","last_undegraded":"2024-09-17T08:34:33.128268+0000","last_fullsized":"2024-09-17T08:34:33.128268+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:59.514789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:33:59.514789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.885022+0000","scrub_duration":0.001575105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"438'13","reported_seq":80,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.808301+0000","last_change":"2024-09-17T08:33:57.801381+0000","last_active":"2024-09-17T08:34:32.808301+0000","last_peered":"2024-09-17T08:34:32.808301+0000","last_clean":"2024-09-17T08:34:32.808301+0000","last_became_active":"2024-09-17T08:32:43.722709+0000","last_became_peered":"2024-09-17T08:32:43.722709+0000","last_unstale":"2024-09-17T08:34:32.808301+0000","last_undegraded":"2024-09-17T08:34:32.808301+0000","last_fullsized":"2024-09-17T08:34:32.808301+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"438'13","last_scrub_stamp":"2024-09-17T08:33:57.801285+0000","last_deep_scrub":"438'13","last_deep_scrub_stamp":"2024-09-17T08:33:57.801285+0000","last_clean_scrub_stamp":"2024-09-17T08:33:57.801285+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:06.984521+0000","scrub_duration":0.002669893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":580,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:10.036971+0000","last_change":"2024-09-17T08:34:10.036971+0000","last_active":"2024-09-17T08:34:10.036971+0000","last_peered":"2024-09-17T08:34:10.036971+0000","last_clean":"2024-09-17T08:34:10.036971+0000","last_became_active":"2024-09-17T08:32:32.663724+0000","last_became_peered":"2024-09-17T08:32:32.663724+0000","last_unstale":"2024-09-17T08:34:10.036971+0000","last_undegraded":"2024-09-17T08:34:10.036971+0000","last_fullsized":"2024-09-17T08:34:10.036971+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.036857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:10.036857+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.036857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.389069+0000","scrub_duration":0.0015775369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'130","reported_seq":476,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748588+0000","last_change":"2024-09-17T08:33:28.447426+0000","last_active":"2024-09-17T08:34:13.748588+0000","last_peered":"2024-09-17T08:34:13.748588+0000","last_clean":"2024-09-17T08:34:13.748588+0000","last_became_active":"2024-09-17T08:32:32.654594+0000","last_became_peered":"2024-09-17T08:32:32.654594+0000","last_unstale":"2024-09-17T08:34:13.748588+0000","last_undegraded":"2024-09-17T08:34:13.748588+0000","last_fullsized":"2024-09-17T08:34:13.748588+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:28.447311+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:31:46.794529+0000","last_clean_scrub_stamp":"2024-09-17T08:33:28.447311+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:47.424437+0000","scrub_duration":0.0015252760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"389'42","reported_seq":71,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941066+0000","last_change":"2024-09-17T08:34:01.843260+0000","last_active":"2024-09-17T08:34:13.941066+0000","last_peered":"2024-09-17T08:34:13.941066+0000","last_clean":"2024-09-17T08:34:13.941066+0000","last_became_active":"2024-09-17T08:32:43.718925+0000","last_became_peered":"2024-09-17T08:32:43.718925+0000","last_unstale":"2024-09-17T08:34:13.941066+0000","last_undegraded":"2024-09-17T08:34:13.941066+0000","last_fullsized":"2024-09-17T08:34:13.941066+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'42","last_scrub_stamp":"2024-09-17T08:34:01.843171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:01.843171+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.113360+0000","scrub_duration":0.0026746589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":439,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:15.137628+0000","last_change":"2024-09-17T08:34:15.137628+0000","last_active":"2024-09-17T08:34:15.137628+0000","last_peered":"2024-09-17T08:34:15.137628+0000","last_clean":"2024-09-17T08:34:15.137628+0000","last_became_active":"2024-09-17T08:32:32.660955+0000","last_became_peered":"2024-09-17T08:32:32.660955+0000","last_unstale":"2024-09-17T08:34:15.137628+0000","last_undegraded":"2024-09-17T08:34:15.137628+0000","last_fullsized":"2024-09-17T08:34:15.137628+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:15.137550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:29:37.364612+0000","last_clean_scrub_stamp":"2024-09-17T08:34:15.137550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:41.429516+0000","scrub_duration":0.001326121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":549,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748263+0000","last_change":"2024-09-17T08:33:36.352045+0000","last_active":"2024-09-17T08:34:13.748263+0000","last_peered":"2024-09-17T08:34:13.748263+0000","last_clean":"2024-09-17T08:34:13.748263+0000","last_became_active":"2024-09-17T08:32:43.732117+0000","last_became_peered":"2024-09-17T08:32:43.732117+0000","last_unstale":"2024-09-17T08:34:13.748263+0000","last_undegraded":"2024-09-17T08:34:13.748263+0000","last_fullsized":"2024-09-17T08:34:13.748263+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:36.351945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:33:36.351945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:48.985504+0000","scrub_duration":0.0020477490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":577,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748773+0000","last_change":"2024-09-17T08:33:30.387806+0000","last_active":"2024-09-17T08:34:13.748773+0000","last_peered":"2024-09-17T08:34:13.748773+0000","last_clean":"2024-09-17T08:34:13.748773+0000","last_became_active":"2024-09-17T08:32:43.720906+0000","last_became_peered":"2024-09-17T08:32:43.720906+0000","last_unstale":"2024-09-17T08:34:13.748773+0000","last_undegraded":"2024-09-17T08:34:13.748773+0000","last_fullsized":"2024-09-17T08:34:13.748773+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:30.387707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:30.387707+0000","last_clean_scrub_stamp":"2024-09-17T08:33:30.387707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.786564+0000","scrub_duration":0.001561303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"75'130","reported_seq":475,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127677+0000","last_change":"2024-09-17T08:33:12.831461+0000","last_active":"2024-09-17T08:34:33.127677+0000","last_peered":"2024-09-17T08:34:33.127677+0000","last_clean":"2024-09-17T08:34:33.127677+0000","last_became_active":"2024-09-17T08:32:46.734867+0000","last_became_peered":"2024-09-17T08:32:46.734867+0000","last_unstale":"2024-09-17T08:34:33.127677+0000","last_undegraded":"2024-09-17T08:34:33.127677+0000","last_fullsized":"2024-09-17T08:34:33.127677+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:12.831216+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:31:33.537488+0000","last_clean_scrub_stamp":"2024-09-17T08:33:12.831216+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.830657+0000","scrub_duration":0.001778567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"389'39","reported_seq":609,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622023+0000","last_change":"2024-09-17T08:33:31.734577+0000","last_active":"2024-09-17T08:33:56.622023+0000","last_peered":"2024-09-17T08:33:56.622023+0000","last_clean":"2024-09-17T08:33:56.622023+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:33:56.622023+0000","last_undegraded":"2024-09-17T08:33:56.622023+0000","last_fullsized":"2024-09-17T08:33:56.622023+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'39","last_scrub_stamp":"2024-09-17T08:33:31.734487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.734487+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:47.409141+0000","scrub_duration":0.0030547149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":562,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:58.066945+0000","last_change":"2024-09-17T08:33:58.066945+0000","last_active":"2024-09-17T08:33:58.066945+0000","last_peered":"2024-09-17T08:33:58.066945+0000","last_clean":"2024-09-17T08:33:58.066945+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:33:58.066945+0000","last_undegraded":"2024-09-17T08:33:58.066945+0000","last_fullsized":"2024-09-17T08:33:58.066945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:33:58.066822+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.066822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:03.198571+0000","scrub_duration":0.0019489920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":190,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:10.282195+0000","last_change":"2024-09-17T08:34:10.282195+0000","last_active":"2024-09-17T08:34:10.282195+0000","last_peered":"2024-09-17T08:34:10.282195+0000","last_clean":"2024-09-17T08:34:10.282195+0000","last_became_active":"2024-09-17T08:33:29.925585+0000","last_became_peered":"2024-09-17T08:33:29.925585+0000","last_unstale":"2024-09-17T08:34:10.282195+0000","last_undegraded":"2024-09-17T08:34:10.282195+0000","last_fullsized":"2024-09-17T08:34:10.282195+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.282097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.282097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.953681+0000","scrub_duration":0.0016866299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":455,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350406+0000","last_change":"2024-09-17T08:33:58.275497+0000","last_active":"2024-09-17T08:34:19.350406+0000","last_peered":"2024-09-17T08:34:19.350406+0000","last_clean":"2024-09-17T08:34:19.350406+0000","last_became_active":"2024-09-17T08:32:32.670334+0000","last_became_peered":"2024-09-17T08:32:32.670334+0000","last_unstale":"2024-09-17T08:34:19.350406+0000","last_undegraded":"2024-09-17T08:34:19.350406+0000","last_fullsized":"2024-09-17T08:34:19.350406+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:58.275424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.275424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.713250+0000","scrub_duration":0.0013653510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"75'130","reported_seq":471,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128192+0000","last_change":"2024-09-17T08:34:16.249901+0000","last_active":"2024-09-17T08:34:33.128192+0000","last_peered":"2024-09-17T08:34:33.128192+0000","last_clean":"2024-09-17T08:34:33.128192+0000","last_became_active":"2024-09-17T08:32:46.733976+0000","last_became_peered":"2024-09-17T08:32:46.733976+0000","last_unstale":"2024-09-17T08:34:33.128192+0000","last_undegraded":"2024-09-17T08:34:33.128192+0000","last_fullsized":"2024-09-17T08:34:33.128192+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:34:16.249788+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:34:16.249788+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.912187+0000","scrub_duration":0.001733671,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"389'35","reported_seq":472,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350443+0000","last_change":"2024-09-17T08:33:49.301865+0000","last_active":"2024-09-17T08:34:19.350443+0000","last_peered":"2024-09-17T08:34:19.350443+0000","last_clean":"2024-09-17T08:34:19.350443+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:34:19.350443+0000","last_undegraded":"2024-09-17T08:34:19.350443+0000","last_fullsized":"2024-09-17T08:34:19.350443+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'35","last_scrub_stamp":"2024-09-17T08:33:49.301712+0000","last_deep_scrub":"220'25","last_deep_scrub_stamp":"2024-09-17T08:29:30.429348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:49.301712+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.681674+0000","scrub_duration":0.00257275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":425,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.068278+0000","last_change":"2024-09-17T08:34:20.068278+0000","last_active":"2024-09-17T08:34:20.068278+0000","last_peered":"2024-09-17T08:34:20.068278+0000","last_clean":"2024-09-17T08:34:20.068278+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:34:20.068278+0000","last_undegraded":"2024-09-17T08:34:20.068278+0000","last_fullsized":"2024-09-17T08:34:20.068278+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:20.068140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.068140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.774839+0000","scrub_duration":0.001743334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"435'11","reported_seq":862,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122164+0000","last_change":"2024-09-17T08:33:47.576848+0000","last_active":"2024-09-17T08:34:23.122164+0000","last_peered":"2024-09-17T08:34:23.122164+0000","last_clean":"2024-09-17T08:34:23.122164+0000","last_became_active":"2024-09-17T08:32:32.664207+0000","last_became_peered":"2024-09-17T08:32:32.664207+0000","last_unstale":"2024-09-17T08:34:23.122164+0000","last_undegraded":"2024-09-17T08:34:23.122164+0000","last_fullsized":"2024-09-17T08:34:23.122164+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'11","last_scrub_stamp":"2024-09-17T08:33:47.576751+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:33:47.576751+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.314296+0000","scrub_duration":0.0021314350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":574,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:26.757471+0000","last_change":"2024-09-17T08:34:26.757543+0000","last_active":"2024-09-17T08:34:26.757471+0000","last_peered":"2024-09-17T08:34:26.757471+0000","last_clean":"2024-09-17T08:34:26.757471+0000","last_became_active":"2024-09-17T08:32:43.720606+0000","last_became_peered":"2024-09-17T08:32:43.720606+0000","last_unstale":"2024-09-17T08:34:26.757471+0000","last_undegraded":"2024-09-17T08:34:26.757471+0000","last_fullsized":"2024-09-17T08:34:26.757471+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.757459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.757459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:54.097535+0000","scrub_duration":0.001314274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'130","reported_seq":321,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621316+0000","last_change":"2024-09-17T08:33:35.993275+0000","last_active":"2024-09-17T08:33:56.621316+0000","last_peered":"2024-09-17T08:33:56.621316+0000","last_clean":"2024-09-17T08:33:56.621316+0000","last_became_active":"2024-09-17T08:33:29.923602+0000","last_became_peered":"2024-09-17T08:33:29.923602+0000","last_unstale":"2024-09-17T08:33:56.621316+0000","last_undegraded":"2024-09-17T08:33:56.621316+0000","last_fullsized":"2024-09-17T08:33:56.621316+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:35.993184+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:35.993184+0000","last_clean_scrub_stamp":"2024-09-17T08:33:35.993184+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:37.358223+0000","scrub_duration":0.001483306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"467'70","reported_seq":822,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.809299+0000","last_change":"2024-09-17T08:34:08.184942+0000","last_active":"2024-09-17T08:34:32.809299+0000","last_peered":"2024-09-17T08:34:32.809299+0000","last_clean":"2024-09-17T08:34:32.809299+0000","last_became_active":"2024-09-17T08:32:43.736276+0000","last_became_peered":"2024-09-17T08:32:43.736276+0000","last_unstale":"2024-09-17T08:34:32.809299+0000","last_undegraded":"2024-09-17T08:34:32.809299+0000","last_fullsized":"2024-09-17T08:34:32.809299+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'70","last_scrub_stamp":"2024-09-17T08:34:08.184827+0000","last_deep_scrub":"467'70","last_deep_scrub_stamp":"2024-09-17T08:34:08.184827+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.184827+0000","objects_scrubbed":10,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.992157+0000","scrub_duration":0.0068099370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":411,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:04.230356+0000","last_change":"2024-09-17T08:34:04.230356+0000","last_active":"2024-09-17T08:34:04.230356+0000","last_peered":"2024-09-17T08:34:04.230356+0000","last_clean":"2024-09-17T08:34:04.230356+0000","last_became_active":"2024-09-17T08:33:29.924321+0000","last_became_peered":"2024-09-17T08:33:29.924321+0000","last_unstale":"2024-09-17T08:34:04.230356+0000","last_undegraded":"2024-09-17T08:34:04.230356+0000","last_fullsized":"2024-09-17T08:34:04.230356+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:04.230259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.230259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.324120+0000","scrub_duration":0.00140515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":559,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621912+0000","last_change":"2024-09-17T08:33:34.711717+0000","last_active":"2024-09-17T08:33:56.621912+0000","last_peered":"2024-09-17T08:33:56.621912+0000","last_clean":"2024-09-17T08:33:56.621912+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:33:56.621912+0000","last_undegraded":"2024-09-17T08:33:56.621912+0000","last_fullsized":"2024-09-17T08:33:56.621912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:34.711633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:22.920405+0000","last_clean_scrub_stamp":"2024-09-17T08:33:34.711633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:38.556910+0000","scrub_duration":0.001465062,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941865+0000","last_change":"2024-09-17T08:34:11.793143+0000","last_active":"2024-09-17T08:34:13.941865+0000","last_peered":"2024-09-17T08:34:13.941865+0000","last_clean":"2024-09-17T08:34:13.941865+0000","last_became_active":"2024-09-17T08:32:43.727553+0000","last_became_peered":"2024-09-17T08:32:43.727553+0000","last_unstale":"2024-09-17T08:34:13.941865+0000","last_undegraded":"2024-09-17T08:34:13.941865+0000","last_fullsized":"2024-09-17T08:34:13.941865+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.793062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.793062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:25.899589+0000","scrub_duration":0.001569097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'130","reported_seq":781,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.005407+0000","last_change":"2024-09-17T08:33:43.900682+0000","last_active":"2024-09-17T08:33:57.005407+0000","last_peered":"2024-09-17T08:33:57.005407+0000","last_clean":"2024-09-17T08:33:57.005407+0000","last_became_active":"2024-09-17T08:33:35.942576+0000","last_became_peered":"2024-09-17T08:33:35.942576+0000","last_unstale":"2024-09-17T08:33:57.005407+0000","last_undegraded":"2024-09-17T08:33:57.005407+0000","last_fullsized":"2024-09-17T08:33:57.005407+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:43.900574+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:43.900574+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:54.476276+0000","scrub_duration":0.001917321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"389'21","reported_seq":91,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:16.094444+0000","last_change":"2024-09-17T08:34:16.094444+0000","last_active":"2024-09-17T08:34:16.094444+0000","last_peered":"2024-09-17T08:34:16.094444+0000","last_clean":"2024-09-17T08:34:16.094444+0000","last_became_active":"2024-09-17T08:31:50.001044+0000","last_became_peered":"2024-09-17T08:31:50.001044+0000","last_unstale":"2024-09-17T08:34:16.094444+0000","last_undegraded":"2024-09-17T08:34:16.094444+0000","last_fullsized":"2024-09-17T08:34:16.094444+0000","mapping_epoch":423,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":424,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'21","last_scrub_stamp":"2024-09-17T08:34:16.094348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:16.094348+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.549686+0000","scrub_duration":0.0024298119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"441'6","reported_seq":1600,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.864908+0000","last_change":"2024-09-17T08:34:17.075627+0000","last_active":"2024-09-17T08:34:32.864908+0000","last_peered":"2024-09-17T08:34:32.864908+0000","last_clean":"2024-09-17T08:34:32.864908+0000","last_became_active":"2024-09-17T08:32:32.661051+0000","last_became_peered":"2024-09-17T08:32:32.661051+0000","last_unstale":"2024-09-17T08:34:32.864908+0000","last_undegraded":"2024-09-17T08:34:32.864908+0000","last_fullsized":"2024-09-17T08:34:32.864908+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'6","last_scrub_stamp":"2024-09-17T08:34:17.075546+0000","last_deep_scrub":"441'6","last_deep_scrub_stamp":"2024-09-17T08:32:48.504881+0000","last_clean_scrub_stamp":"2024-09-17T08:34:17.075546+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:42.865178+0000","scrub_duration":0.0016728159999999999,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1134,"num_read_kb":1129,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":483,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.004456+0000","last_change":"2024-09-17T08:33:51.083094+0000","last_active":"2024-09-17T08:33:57.004456+0000","last_peered":"2024-09-17T08:33:57.004456+0000","last_clean":"2024-09-17T08:33:57.004456+0000","last_became_active":"2024-09-17T08:33:43.736595+0000","last_became_peered":"2024-09-17T08:33:43.736595+0000","last_unstale":"2024-09-17T08:33:57.004456+0000","last_undegraded":"2024-09-17T08:33:57.004456+0000","last_fullsized":"2024-09-17T08:33:57.004456+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:51.083004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:33:51.083004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.527861+0000","scrub_duration":0.001394564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":549,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:03.266126+0000","last_change":"2024-09-17T08:34:03.266126+0000","last_active":"2024-09-17T08:34:03.266126+0000","last_peered":"2024-09-17T08:34:03.266126+0000","last_clean":"2024-09-17T08:34:03.266126+0000","last_became_active":"2024-09-17T08:33:43.627052+0000","last_became_peered":"2024-09-17T08:33:43.627052+0000","last_unstale":"2024-09-17T08:34:03.266126+0000","last_undegraded":"2024-09-17T08:34:03.266126+0000","last_fullsized":"2024-09-17T08:34:03.266126+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.265988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.265988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.762258+0000","scrub_duration":0.0017509660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":501,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.116086+0000","last_change":"2024-09-17T08:34:19.116175+0000","last_active":"2024-09-17T08:34:19.116086+0000","last_peered":"2024-09-17T08:34:19.116086+0000","last_clean":"2024-09-17T08:34:19.116086+0000","last_became_active":"2024-09-17T08:33:35.944536+0000","last_became_peered":"2024-09-17T08:33:35.944536+0000","last_unstale":"2024-09-17T08:34:19.116086+0000","last_undegraded":"2024-09-17T08:34:19.116086+0000","last_fullsized":"2024-09-17T08:34:19.116086+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:19.116070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:34.623625+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.116070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:32.674800+0000","scrub_duration":0.0015923300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'130","reported_seq":477,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751852+0000","last_change":"2024-09-17T08:33:31.394511+0000","last_active":"2024-09-17T08:34:13.751852+0000","last_peered":"2024-09-17T08:34:13.751852+0000","last_clean":"2024-09-17T08:34:13.751852+0000","last_became_active":"2024-09-17T08:32:32.664164+0000","last_became_peered":"2024-09-17T08:32:32.664164+0000","last_unstale":"2024-09-17T08:34:13.751852+0000","last_undegraded":"2024-09-17T08:34:13.751852+0000","last_fullsized":"2024-09-17T08:34:13.751852+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:31.394422+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.394422+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:53.574905+0000","scrub_duration":0.00180649,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"389'42","reported_seq":679,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.695355+0000","last_change":"2024-09-17T08:34:32.695426+0000","last_active":"2024-09-17T08:34:32.695355+0000","last_peered":"2024-09-17T08:34:32.695355+0000","last_clean":"2024-09-17T08:34:32.695355+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:34:32.695355+0000","last_undegraded":"2024-09-17T08:34:32.695355+0000","last_fullsized":"2024-09-17T08:34:32.695355+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'42","last_scrub_stamp":"2024-09-17T08:34:32.695342+0000","last_deep_scrub":"389'42","last_deep_scrub_stamp":"2024-09-17T08:33:29.760243+0000","last_clean_scrub_stamp":"2024-09-17T08:34:32.695342+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:53.701113+0000","scrub_duration":0.0019870500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":589,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.827367+0000","last_change":"2024-09-17T08:34:22.827478+0000","last_active":"2024-09-17T08:34:22.827367+0000","last_peered":"2024-09-17T08:34:22.827367+0000","last_clean":"2024-09-17T08:34:22.827367+0000","last_became_active":"2024-09-17T08:32:43.738598+0000","last_became_peered":"2024-09-17T08:32:43.738598+0000","last_unstale":"2024-09-17T08:34:22.827367+0000","last_undegraded":"2024-09-17T08:34:22.827367+0000","last_fullsized":"2024-09-17T08:34:22.827367+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:34:22.827339+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.827339+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.148205+0000","scrub_duration":0.001894073,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":511,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006496+0000","last_change":"2024-09-17T08:33:46.954112+0000","last_active":"2024-09-17T08:33:57.006496+0000","last_peered":"2024-09-17T08:33:57.006496+0000","last_clean":"2024-09-17T08:33:57.006496+0000","last_became_active":"2024-09-17T08:33:35.942772+0000","last_became_peered":"2024-09-17T08:33:35.942772+0000","last_unstale":"2024-09-17T08:33:57.006496+0000","last_undegraded":"2024-09-17T08:33:57.006496+0000","last_fullsized":"2024-09-17T08:33:57.006496+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:46.953992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_clean_scrub_stamp":"2024-09-17T08:33:46.953992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:06.776876+0000","scrub_duration":0.0025532979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622050+0000","last_change":"2024-09-17T08:33:31.027945+0000","last_active":"2024-09-17T08:33:56.622050+0000","last_peered":"2024-09-17T08:33:56.622050+0000","last_clean":"2024-09-17T08:33:56.622050+0000","last_became_active":"2024-09-17T08:33:29.923055+0000","last_became_peered":"2024-09-17T08:33:29.923055+0000","last_unstale":"2024-09-17T08:33:56.622050+0000","last_undegraded":"2024-09-17T08:33:56.622050+0000","last_fullsized":"2024-09-17T08:33:56.622050+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:31.027838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.027838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.760376+0000","scrub_duration":0.0016732629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":458,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350509+0000","last_change":"2024-09-17T08:34:11.209317+0000","last_active":"2024-09-17T08:34:19.350509+0000","last_peered":"2024-09-17T08:34:19.350509+0000","last_clean":"2024-09-17T08:34:19.350509+0000","last_became_active":"2024-09-17T08:32:43.735221+0000","last_became_peered":"2024-09-17T08:32:43.735221+0000","last_unstale":"2024-09-17T08:34:19.350509+0000","last_undegraded":"2024-09-17T08:34:19.350509+0000","last_fullsized":"2024-09-17T08:34:19.350509+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.209241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.209241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:40.277382+0000","scrub_duration":0.0017053770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"389'45","reported_seq":377,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.345333+0000","last_change":"2024-09-17T08:33:58.554487+0000","last_active":"2024-09-17T08:34:35.345333+0000","last_peered":"2024-09-17T08:34:35.345333+0000","last_clean":"2024-09-17T08:34:35.345333+0000","last_became_active":"2024-09-17T08:32:32.655755+0000","last_became_peered":"2024-09-17T08:32:32.655755+0000","last_unstale":"2024-09-17T08:34:35.345333+0000","last_undegraded":"2024-09-17T08:34:35.345333+0000","last_fullsized":"2024-09-17T08:34:35.345333+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'45","last_scrub_stamp":"2024-09-17T08:33:58.554370+0000","last_deep_scrub":"389'45","last_deep_scrub_stamp":"2024-09-17T08:33:58.554370+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.554370+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:19.186909+0000","scrub_duration":0.0026170529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"419'3","reported_seq":57,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941222+0000","last_change":"2024-09-17T08:33:36.855970+0000","last_active":"2024-09-17T08:34:13.941222+0000","last_peered":"2024-09-17T08:34:13.941222+0000","last_clean":"2024-09-17T08:34:13.941222+0000","last_became_active":"2024-09-17T08:32:43.727064+0000","last_became_peered":"2024-09-17T08:32:43.727064+0000","last_unstale":"2024-09-17T08:34:13.941222+0000","last_undegraded":"2024-09-17T08:34:13.941222+0000","last_fullsized":"2024-09-17T08:34:13.941222+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"419'3","last_scrub_stamp":"2024-09-17T08:33:36.855865+0000","last_deep_scrub":"419'3","last_deep_scrub_stamp":"2024-09-17T08:31:37.564007+0000","last_clean_scrub_stamp":"2024-09-17T08:33:36.855865+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:51.886060+0000","scrub_duration":0.0019571559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.253240+0000","last_change":"2024-09-17T08:34:07.253240+0000","last_active":"2024-09-17T08:34:07.253240+0000","last_peered":"2024-09-17T08:34:07.253240+0000","last_clean":"2024-09-17T08:34:07.253240+0000","last_became_active":"2024-09-17T08:33:29.928307+0000","last_became_peered":"2024-09-17T08:33:29.928307+0000","last_unstale":"2024-09-17T08:34:07.253240+0000","last_undegraded":"2024-09-17T08:34:07.253240+0000","last_fullsized":"2024-09-17T08:34:07.253240+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:07.253146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:34:07.253146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.900418+0000","scrub_duration":0.0015987600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":765,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:25.189082+0000","last_change":"2024-09-17T08:34:25.189180+0000","last_active":"2024-09-17T08:34:25.189082+0000","last_peered":"2024-09-17T08:34:25.189082+0000","last_clean":"2024-09-17T08:34:25.189082+0000","last_became_active":"2024-09-17T08:33:35.944526+0000","last_became_peered":"2024-09-17T08:33:35.944526+0000","last_unstale":"2024-09-17T08:34:25.189082+0000","last_undegraded":"2024-09-17T08:34:25.189082+0000","last_fullsized":"2024-09-17T08:34:25.189082+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:34:25.189065+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:34:25.189065+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:28.139259+0000","scrub_duration":0.001807061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":568,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:21.180010+0000","last_change":"2024-09-17T08:34:21.180115+0000","last_active":"2024-09-17T08:34:21.180010+0000","last_peered":"2024-09-17T08:34:21.180010+0000","last_clean":"2024-09-17T08:34:21.180010+0000","last_became_active":"2024-09-17T08:33:35.946777+0000","last_became_peered":"2024-09-17T08:33:35.946777+0000","last_unstale":"2024-09-17T08:34:21.180010+0000","last_undegraded":"2024-09-17T08:34:21.180010+0000","last_fullsized":"2024-09-17T08:34:21.180010+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:21.179992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:21.179992+0000","last_clean_scrub_stamp":"2024-09-17T08:34:21.179992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.445761+0000","scrub_duration":0.0016641130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":130,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622114+0000","last_change":"2024-09-17T08:33:30.129157+0000","last_active":"2024-09-17T08:33:56.622114+0000","last_peered":"2024-09-17T08:33:56.622114+0000","last_clean":"2024-09-17T08:33:56.622114+0000","last_became_active":"2024-09-17T08:33:30.128228+0000","last_became_peered":"2024-09-17T08:33:30.128228+0000","last_unstale":"2024-09-17T08:33:56.622114+0000","last_undegraded":"2024-09-17T08:33:56.622114+0000","last_fullsized":"2024-09-17T08:33:56.622114+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:15.007821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:15.007821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:21.142102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"390'51","reported_seq":427,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127037+0000","last_change":"2024-09-17T08:33:11.877859+0000","last_active":"2024-09-17T08:34:33.127037+0000","last_peered":"2024-09-17T08:34:33.127037+0000","last_clean":"2024-09-17T08:34:33.127037+0000","last_became_active":"2024-09-17T08:32:32.665974+0000","last_became_peered":"2024-09-17T08:32:32.665974+0000","last_unstale":"2024-09-17T08:34:33.127037+0000","last_undegraded":"2024-09-17T08:34:33.127037+0000","last_fullsized":"2024-09-17T08:34:33.127037+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"390'51","last_scrub_stamp":"2024-09-17T08:33:11.877729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:11.877729+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:38.447825+0000","scrub_duration":0.0027582140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":462,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.201455+0000","last_change":"2024-09-17T08:34:30.201519+0000","last_active":"2024-09-17T08:34:30.201455+0000","last_peered":"2024-09-17T08:34:30.201455+0000","last_clean":"2024-09-17T08:34:30.201455+0000","last_became_active":"2024-09-17T08:33:43.737691+0000","last_became_peered":"2024-09-17T08:33:43.737691+0000","last_unstale":"2024-09-17T08:34:30.201455+0000","last_undegraded":"2024-09-17T08:34:30.201455+0000","last_fullsized":"2024-09-17T08:34:30.201455+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:30.201439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:30.201439+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.201439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:57.677602+0000","scrub_duration":0.0018739329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":546,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.315388+0000","last_change":"2024-09-17T08:34:19.315448+0000","last_active":"2024-09-17T08:34:19.315388+0000","last_peered":"2024-09-17T08:34:19.315388+0000","last_clean":"2024-09-17T08:34:19.315388+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:34:19.315388+0000","last_undegraded":"2024-09-17T08:34:19.315388+0000","last_fullsized":"2024-09-17T08:34:19.315388+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:19.315374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.315374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.174069+0000","scrub_duration":0.0016805679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":494,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:17.130432+0000","last_change":"2024-09-17T08:34:17.130527+0000","last_active":"2024-09-17T08:34:17.130432+0000","last_peered":"2024-09-17T08:34:17.130432+0000","last_clean":"2024-09-17T08:34:17.130432+0000","last_became_active":"2024-09-17T08:33:35.945813+0000","last_became_peered":"2024-09-17T08:33:35.945813+0000","last_unstale":"2024-09-17T08:34:17.130432+0000","last_undegraded":"2024-09-17T08:34:17.130432+0000","last_fullsized":"2024-09-17T08:34:17.130432+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:17.130400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:17.130400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.878747+0000","scrub_duration":0.001414242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"468'25","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.781037+0000","last_change":"2024-09-17T08:34:23.781128+0000","last_active":"2024-09-17T08:34:23.781037+0000","last_peered":"2024-09-17T08:34:23.781037+0000","last_clean":"2024-09-17T08:34:23.781037+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:34:23.781037+0000","last_undegraded":"2024-09-17T08:34:23.781037+0000","last_fullsized":"2024-09-17T08:34:23.781037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'25","last_scrub_stamp":"2024-09-17T08:34:23.781020+0000","last_deep_scrub":"468'25","last_deep_scrub_stamp":"2024-09-17T08:34:23.781020+0000","last_clean_scrub_stamp":"2024-09-17T08:34:23.781020+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.965648+0000","scrub_duration":0.0047644130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":442,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752158+0000","last_change":"2024-09-17T08:33:32.444569+0000","last_active":"2024-09-17T08:34:13.752158+0000","last_peered":"2024-09-17T08:34:13.752158+0000","last_clean":"2024-09-17T08:34:13.752158+0000","last_became_active":"2024-09-17T08:32:43.726028+0000","last_became_peered":"2024-09-17T08:32:43.726028+0000","last_unstale":"2024-09-17T08:34:13.752158+0000","last_undegraded":"2024-09-17T08:34:13.752158+0000","last_fullsized":"2024-09-17T08:34:13.752158+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:32.444495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:33:32.444495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:45.215832+0000","scrub_duration":0.0015980600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":519,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.288213+0000","last_change":"2024-09-17T08:34:23.288287+0000","last_active":"2024-09-17T08:34:23.288213+0000","last_peered":"2024-09-17T08:34:23.288213+0000","last_clean":"2024-09-17T08:34:23.288213+0000","last_became_active":"2024-09-17T08:33:43.629024+0000","last_became_peered":"2024-09-17T08:33:43.629024+0000","last_unstale":"2024-09-17T08:34:23.288213+0000","last_undegraded":"2024-09-17T08:34:23.288213+0000","last_fullsized":"2024-09-17T08:34:23.288213+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:23.288196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:23.288196+0000","last_clean_scrub_stamp":"2024-09-17T08:34:23.288196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:40.271669+0000","scrub_duration":0.0015586490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":554,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.647087+0000","last_change":"2024-09-17T08:34:13.647087+0000","last_active":"2024-09-17T08:34:13.647087+0000","last_peered":"2024-09-17T08:34:13.647087+0000","last_clean":"2024-09-17T08:34:13.647087+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:34:13.647087+0000","last_undegraded":"2024-09-17T08:34:13.647087+0000","last_fullsized":"2024-09-17T08:34:13.647087+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:13.646989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:13.646989+0000","last_clean_scrub_stamp":"2024-09-17T08:34:13.646989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.988115+0000","scrub_duration":0.0015587999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"389'46","reported_seq":562,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.219371+0000","last_change":"2024-09-17T08:34:30.219459+0000","last_active":"2024-09-17T08:34:30.219371+0000","last_peered":"2024-09-17T08:34:30.219371+0000","last_clean":"2024-09-17T08:34:30.219371+0000","last_became_active":"2024-09-17T08:33:29.926228+0000","last_became_peered":"2024-09-17T08:33:29.926228+0000","last_unstale":"2024-09-17T08:34:30.219371+0000","last_undegraded":"2024-09-17T08:34:30.219371+0000","last_fullsized":"2024-09-17T08:34:30.219371+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'46","last_scrub_stamp":"2024-09-17T08:34:30.219352+0000","last_deep_scrub":"389'46","last_deep_scrub_stamp":"2024-09-17T08:34:30.219352+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.219352+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:59.962987+0000","scrub_duration":0.0026729029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":507,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:12.113769+0000","last_change":"2024-09-17T08:33:43.627938+0000","last_active":"2024-09-17T08:34:12.113769+0000","last_peered":"2024-09-17T08:34:12.113769+0000","last_clean":"2024-09-17T08:34:12.113769+0000","last_became_active":"2024-09-17T08:33:43.627449+0000","last_became_peered":"2024-09-17T08:33:43.627449+0000","last_unstale":"2024-09-17T08:34:12.113769+0000","last_undegraded":"2024-09-17T08:34:12.113769+0000","last_fullsized":"2024-09-17T08:34:12.113769+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:16.966496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:16.966496+0000","last_clean_scrub_stamp":"2024-09-17T08:33:16.966496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:42.809931+0000","scrub_duration":0.0035556149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"435'11","reported_seq":872,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122522+0000","last_change":"2024-09-17T08:34:01.124122+0000","last_active":"2024-09-17T08:34:23.122522+0000","last_peered":"2024-09-17T08:34:23.122522+0000","last_clean":"2024-09-17T08:34:23.122522+0000","last_became_active":"2024-09-17T08:32:32.655178+0000","last_became_peered":"2024-09-17T08:32:32.655178+0000","last_unstale":"2024-09-17T08:34:23.122522+0000","last_undegraded":"2024-09-17T08:34:23.122522+0000","last_fullsized":"2024-09-17T08:34:23.122522+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'11","last_scrub_stamp":"2024-09-17T08:34:01.124010+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_clean_scrub_stamp":"2024-09-17T08:34:01.124010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:16.952157+0000","scrub_duration":0.0018368589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":266,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:05.245370+0000","last_change":"2024-09-17T08:34:05.245370+0000","last_active":"2024-09-17T08:34:05.245370+0000","last_peered":"2024-09-17T08:34:05.245370+0000","last_clean":"2024-09-17T08:34:05.245370+0000","last_became_active":"2024-09-17T08:33:29.925368+0000","last_became_peered":"2024-09-17T08:33:29.925368+0000","last_unstale":"2024-09-17T08:34:05.245370+0000","last_undegraded":"2024-09-17T08:34:05.245370+0000","last_fullsized":"2024-09-17T08:34:05.245370+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:05.245271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:34:05.245271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:33.409968+0000","scrub_duration":0.0013971459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"467'29","reported_seq":679,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748311+0000","last_change":"2024-09-17T08:33:58.242429+0000","last_active":"2024-09-17T08:34:13.748311+0000","last_peered":"2024-09-17T08:34:13.748311+0000","last_clean":"2024-09-17T08:34:13.748311+0000","last_became_active":"2024-09-17T08:32:43.732127+0000","last_became_peered":"2024-09-17T08:32:43.732127+0000","last_unstale":"2024-09-17T08:34:13.748311+0000","last_undegraded":"2024-09-17T08:34:13.748311+0000","last_fullsized":"2024-09-17T08:34:13.748311+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'29","last_scrub_stamp":"2024-09-17T08:33:58.242288+0000","last_deep_scrub":"467'29","last_deep_scrub_stamp":"2024-09-17T08:33:58.242288+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.242288+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.757095+0000","scrub_duration":0.0047194519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"444'4","reported_seq":569,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:27.344832+0000","last_change":"2024-09-17T08:34:27.344832+0000","last_active":"2024-09-17T08:34:27.344832+0000","last_peered":"2024-09-17T08:34:27.344832+0000","last_clean":"2024-09-17T08:34:27.344832+0000","last_became_active":"2024-09-17T08:29:02.550936+0000","last_became_peered":"2024-09-17T08:29:02.550936+0000","last_unstale":"2024-09-17T08:34:27.344832+0000","last_undegraded":"2024-09-17T08:34:27.344832+0000","last_fullsized":"2024-09-17T08:34:27.344832+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"444'4","last_scrub_stamp":"2024-09-17T08:34:27.344696+0000","last_deep_scrub":"444'4","last_deep_scrub_stamp":"2024-09-17T08:33:04.659998+0000","last_clean_scrub_stamp":"2024-09-17T08:34:27.344696+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:57.338812+0000","scrub_duration":0.001830786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":487,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.004269+0000","last_change":"2024-09-17T08:33:48.044948+0000","last_active":"2024-09-17T08:33:57.004269+0000","last_peered":"2024-09-17T08:33:57.004269+0000","last_clean":"2024-09-17T08:33:57.004269+0000","last_became_active":"2024-09-17T08:33:43.736688+0000","last_became_peered":"2024-09-17T08:33:43.736688+0000","last_unstale":"2024-09-17T08:33:57.004269+0000","last_undegraded":"2024-09-17T08:33:57.004269+0000","last_fullsized":"2024-09-17T08:33:57.004269+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:48.044860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.044860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.103510+0000","scrub_duration":0.0013201689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941349+0000","last_change":"2024-09-17T08:34:03.743212+0000","last_active":"2024-09-17T08:34:13.941349+0000","last_peered":"2024-09-17T08:34:13.941349+0000","last_clean":"2024-09-17T08:34:13.941349+0000","last_became_active":"2024-09-17T08:32:43.739607+0000","last_became_peered":"2024-09-17T08:32:43.739607+0000","last_unstale":"2024-09-17T08:34:13.941349+0000","last_undegraded":"2024-09-17T08:34:13.941349+0000","last_fullsized":"2024-09-17T08:34:13.941349+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.743082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.743082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.330450+0000","scrub_duration":0.001381841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"389'25","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.731731+0000","last_change":"2024-09-17T08:34:18.731815+0000","last_active":"2024-09-17T08:34:18.731731+0000","last_peered":"2024-09-17T08:34:18.731731+0000","last_clean":"2024-09-17T08:34:18.731731+0000","last_became_active":"2024-09-17T08:32:43.730052+0000","last_became_peered":"2024-09-17T08:32:43.730052+0000","last_unstale":"2024-09-17T08:34:18.731731+0000","last_undegraded":"2024-09-17T08:34:18.731731+0000","last_fullsized":"2024-09-17T08:34:18.731731+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:18.731717+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.731717+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:32.200082+0000","scrub_duration":0.00231562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":501,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.004340+0000","last_change":"2024-09-17T08:33:43.737320+0000","last_active":"2024-09-17T08:33:57.004340+0000","last_peered":"2024-09-17T08:33:57.004340+0000","last_clean":"2024-09-17T08:33:57.004340+0000","last_became_active":"2024-09-17T08:33:43.737045+0000","last_became_peered":"2024-09-17T08:33:43.737045+0000","last_unstale":"2024-09-17T08:33:57.004340+0000","last_undegraded":"2024-09-17T08:33:57.004340+0000","last_fullsized":"2024-09-17T08:33:57.004340+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:22.520435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:22.520435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.272245+0000","scrub_duration":0.025124259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941649+0000","last_change":"2024-09-17T08:33:18.963625+0000","last_active":"2024-09-17T08:34:13.941649+0000","last_peered":"2024-09-17T08:34:13.941649+0000","last_clean":"2024-09-17T08:34:13.941649+0000","last_became_active":"2024-09-17T08:32:43.722547+0000","last_became_peered":"2024-09-17T08:32:43.722547+0000","last_unstale":"2024-09-17T08:34:13.941649+0000","last_undegraded":"2024-09-17T08:34:13.941649+0000","last_fullsized":"2024-09-17T08:34:13.941649+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:18.963447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:18.963447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017451299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":474,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:10.294438+0000","last_change":"2024-09-17T08:34:10.294438+0000","last_active":"2024-09-17T08:34:10.294438+0000","last_peered":"2024-09-17T08:34:10.294438+0000","last_clean":"2024-09-17T08:34:10.294438+0000","last_became_active":"2024-09-17T08:33:43.628666+0000","last_became_peered":"2024-09-17T08:33:43.628666+0000","last_unstale":"2024-09-17T08:34:10.294438+0000","last_undegraded":"2024-09-17T08:34:10.294438+0000","last_fullsized":"2024-09-17T08:34:10.294438+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.294355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.294355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.035808+0000","scrub_duration":0.0014631900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":330,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127027+0000","last_change":"2024-09-17T08:34:26.354336+0000","last_active":"2024-09-17T08:34:33.127027+0000","last_peered":"2024-09-17T08:34:33.127027+0000","last_clean":"2024-09-17T08:34:33.127027+0000","last_became_active":"2024-09-17T08:32:32.658304+0000","last_became_peered":"2024-09-17T08:32:32.658304+0000","last_unstale":"2024-09-17T08:34:33.127027+0000","last_undegraded":"2024-09-17T08:34:33.127027+0000","last_fullsized":"2024-09-17T08:34:33.127027+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.354233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.354233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:37.372385+0000","scrub_duration":0.001699737,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"420'21","reported_seq":61,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941575+0000","last_change":"2024-09-17T08:33:48.668393+0000","last_active":"2024-09-17T08:34:13.941575+0000","last_peered":"2024-09-17T08:34:13.941575+0000","last_clean":"2024-09-17T08:34:13.941575+0000","last_became_active":"2024-09-17T08:32:43.725971+0000","last_became_peered":"2024-09-17T08:32:43.725971+0000","last_unstale":"2024-09-17T08:34:13.941575+0000","last_undegraded":"2024-09-17T08:34:13.941575+0000","last_fullsized":"2024-09-17T08:34:13.941575+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"420'21","last_scrub_stamp":"2024-09-17T08:33:48.668286+0000","last_deep_scrub":"420'21","last_deep_scrub_stamp":"2024-09-17T08:31:40.137904+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.668286+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:52.935284+0000","scrub_duration":0.001802606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621952+0000","last_change":"2024-09-17T08:33:29.921338+0000","last_active":"2024-09-17T08:33:56.621952+0000","last_peered":"2024-09-17T08:33:56.621952+0000","last_clean":"2024-09-17T08:33:56.621952+0000","last_became_active":"2024-09-17T08:33:29.919398+0000","last_became_peered":"2024-09-17T08:33:29.919398+0000","last_unstale":"2024-09-17T08:33:56.621952+0000","last_undegraded":"2024-09-17T08:33:56.621952+0000","last_fullsized":"2024-09-17T08:33:56.621952+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:58.139916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:58.139916+0000","last_clean_scrub_stamp":"2024-09-17T08:32:58.139916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:20.150445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":574,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:08.043793+0000","last_change":"2024-09-17T08:34:07.080340+0000","last_active":"2024-09-17T08:34:08.043793+0000","last_peered":"2024-09-17T08:34:08.043793+0000","last_clean":"2024-09-17T08:34:08.043793+0000","last_became_active":"2024-09-17T08:32:32.651944+0000","last_became_peered":"2024-09-17T08:32:32.651944+0000","last_unstale":"2024-09-17T08:34:08.043793+0000","last_undegraded":"2024-09-17T08:34:08.043793+0000","last_fullsized":"2024-09-17T08:34:08.043793+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:07.080233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:07.080233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:21.237210+0000","scrub_duration":0.00163532,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"389'30","reported_seq":65,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941975+0000","last_change":"2024-09-17T08:33:50.746646+0000","last_active":"2024-09-17T08:34:13.941975+0000","last_peered":"2024-09-17T08:34:13.941975+0000","last_clean":"2024-09-17T08:34:13.941975+0000","last_became_active":"2024-09-17T08:32:43.727080+0000","last_became_peered":"2024-09-17T08:32:43.727080+0000","last_unstale":"2024-09-17T08:34:13.941975+0000","last_undegraded":"2024-09-17T08:34:13.941975+0000","last_fullsized":"2024-09-17T08:34:13.941975+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'30","last_scrub_stamp":"2024-09-17T08:33:50.746545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:50.746545+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:13.481858+0000","scrub_duration":0.001928164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":200,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:26.126564+0000","last_change":"2024-09-17T08:34:26.126647+0000","last_active":"2024-09-17T08:34:26.126564+0000","last_peered":"2024-09-17T08:34:26.126564+0000","last_clean":"2024-09-17T08:34:26.126564+0000","last_became_active":"2024-09-17T08:33:29.923325+0000","last_became_peered":"2024-09-17T08:33:29.923325+0000","last_unstale":"2024-09-17T08:34:26.126564+0000","last_undegraded":"2024-09-17T08:34:26.126564+0000","last_fullsized":"2024-09-17T08:34:26.126564+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.126549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.126549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:52.507737+0000","scrub_duration":0.0015897789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"455'13","reported_seq":167,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122543+0000","last_change":"2024-09-17T08:33:58.156020+0000","last_active":"2024-09-17T08:34:23.122543+0000","last_peered":"2024-09-17T08:34:23.122543+0000","last_clean":"2024-09-17T08:34:23.122543+0000","last_became_active":"2024-09-17T08:33:29.921687+0000","last_became_peered":"2024-09-17T08:33:29.921687+0000","last_unstale":"2024-09-17T08:34:23.122543+0000","last_undegraded":"2024-09-17T08:34:23.122543+0000","last_fullsized":"2024-09-17T08:34:23.122543+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"455'13","last_scrub_stamp":"2024-09-17T08:33:58.155909+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:29:34.411246+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.155909+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:17.174516+0000","scrub_duration":0.001838604,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":556,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:11.972638+0000","last_change":"2024-09-17T08:33:50.561826+0000","last_active":"2024-09-17T08:34:11.972638+0000","last_peered":"2024-09-17T08:34:11.972638+0000","last_clean":"2024-09-17T08:34:11.972638+0000","last_became_active":"2024-09-17T08:32:43.719215+0000","last_became_peered":"2024-09-17T08:32:43.719215+0000","last_unstale":"2024-09-17T08:34:11.972638+0000","last_undegraded":"2024-09-17T08:34:11.972638+0000","last_fullsized":"2024-09-17T08:34:11.972638+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:50.561717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:50.561717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:09.215291+0000","scrub_duration":0.0019649379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"389'25","reported_seq":440,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.126546+0000","last_change":"2024-09-17T08:34:22.126546+0000","last_active":"2024-09-17T08:34:22.126546+0000","last_peered":"2024-09-17T08:34:22.126546+0000","last_clean":"2024-09-17T08:34:22.126546+0000","last_became_active":"2024-09-17T08:33:29.926041+0000","last_became_peered":"2024-09-17T08:33:29.926041+0000","last_unstale":"2024-09-17T08:34:22.126546+0000","last_undegraded":"2024-09-17T08:34:22.126546+0000","last_fullsized":"2024-09-17T08:34:22.126546+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:22.126408+0000","last_deep_scrub":"389'25","last_deep_scrub_stamp":"2024-09-17T08:33:05.055099+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.126408+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:47.692473+0000","scrub_duration":0.0022084919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":562,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:03.545529+0000","last_change":"2024-09-17T08:34:03.545529+0000","last_active":"2024-09-17T08:34:03.545529+0000","last_peered":"2024-09-17T08:34:03.545529+0000","last_clean":"2024-09-17T08:34:03.545529+0000","last_became_active":"2024-09-17T08:32:43.730244+0000","last_became_peered":"2024-09-17T08:32:43.730244+0000","last_unstale":"2024-09-17T08:34:03.545529+0000","last_undegraded":"2024-09-17T08:34:03.545529+0000","last_fullsized":"2024-09-17T08:34:03.545529+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.545422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.545422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:04.748468+0000","scrub_duration":0.001371415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"438'14","reported_seq":80,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.809819+0000","last_change":"2024-09-17T08:33:28.955404+0000","last_active":"2024-09-17T08:34:32.809819+0000","last_peered":"2024-09-17T08:34:32.809819+0000","last_clean":"2024-09-17T08:34:32.809819+0000","last_became_active":"2024-09-17T08:32:43.729362+0000","last_became_peered":"2024-09-17T08:32:43.729362+0000","last_unstale":"2024-09-17T08:34:32.809819+0000","last_undegraded":"2024-09-17T08:34:32.809819+0000","last_fullsized":"2024-09-17T08:34:32.809819+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"438'14","last_scrub_stamp":"2024-09-17T08:33:28.955300+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:33:28.955300+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:39.293174+0000","scrub_duration":0.0021009459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":533,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.749126+0000","last_change":"2024-09-17T08:33:27.443109+0000","last_active":"2024-09-17T08:34:13.749126+0000","last_peered":"2024-09-17T08:34:13.749126+0000","last_clean":"2024-09-17T08:34:13.749126+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:34:13.749126+0000","last_undegraded":"2024-09-17T08:34:13.749126+0000","last_fullsized":"2024-09-17T08:34:13.749126+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:27.443000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_clean_scrub_stamp":"2024-09-17T08:33:27.443000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:51.341041+0000","scrub_duration":0.001620898,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"389'21","reported_seq":469,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.349850+0000","last_change":"2024-09-17T08:33:07.573106+0000","last_active":"2024-09-17T08:34:14.349850+0000","last_peered":"2024-09-17T08:34:14.349850+0000","last_clean":"2024-09-17T08:34:14.349850+0000","last_became_active":"2024-09-17T08:32:32.671197+0000","last_became_peered":"2024-09-17T08:32:32.671197+0000","last_unstale":"2024-09-17T08:34:14.349850+0000","last_undegraded":"2024-09-17T08:34:14.349850+0000","last_fullsized":"2024-09-17T08:34:14.349850+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'21","last_scrub_stamp":"2024-09-17T08:33:07.572956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:07.572956+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026949299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":325,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128506+0000","last_change":"2024-09-17T08:33:40.661731+0000","last_active":"2024-09-17T08:34:33.128506+0000","last_peered":"2024-09-17T08:34:33.128506+0000","last_clean":"2024-09-17T08:34:33.128506+0000","last_became_active":"2024-09-17T08:32:43.715260+0000","last_became_peered":"2024-09-17T08:32:43.715260+0000","last_unstale":"2024-09-17T08:34:33.128506+0000","last_undegraded":"2024-09-17T08:34:33.128506+0000","last_fullsized":"2024-09-17T08:34:33.128506+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:40.661630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:34.199418+0000","last_clean_scrub_stamp":"2024-09-17T08:33:40.661630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.017334+0000","scrub_duration":0.001546813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941256+0000","last_change":"2024-09-17T08:34:05.804038+0000","last_active":"2024-09-17T08:34:13.941256+0000","last_peered":"2024-09-17T08:34:13.941256+0000","last_clean":"2024-09-17T08:34:13.941256+0000","last_became_active":"2024-09-17T08:32:43.735363+0000","last_became_peered":"2024-09-17T08:32:43.735363+0000","last_unstale":"2024-09-17T08:34:13.941256+0000","last_undegraded":"2024-09-17T08:34:13.941256+0000","last_fullsized":"2024-09-17T08:34:13.941256+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:05.803956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:53.069071+0000","last_clean_scrub_stamp":"2024-09-17T08:34:05.803956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.364498+0000","scrub_duration":0.0011965089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":556,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.793270+0000","last_change":"2024-09-17T08:34:20.793335+0000","last_active":"2024-09-17T08:34:20.793270+0000","last_peered":"2024-09-17T08:34:20.793270+0000","last_clean":"2024-09-17T08:34:20.793270+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:34:20.793270+0000","last_undegraded":"2024-09-17T08:34:20.793270+0000","last_fullsized":"2024-09-17T08:34:20.793270+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:20.793258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.793258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.060833+0000","scrub_duration":0.00185181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"389'27","reported_seq":634,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748598+0000","last_change":"2024-09-17T08:34:04.330881+0000","last_active":"2024-09-17T08:34:13.748598+0000","last_peered":"2024-09-17T08:34:13.748598+0000","last_clean":"2024-09-17T08:34:13.748598+0000","last_became_active":"2024-09-17T08:32:32.651582+0000","last_became_peered":"2024-09-17T08:32:32.651582+0000","last_unstale":"2024-09-17T08:34:13.748598+0000","last_undegraded":"2024-09-17T08:34:13.748598+0000","last_fullsized":"2024-09-17T08:34:13.748598+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:34:04.330744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.330744+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.852415+0000","scrub_duration":0.002077307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":423,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:31.211527+0000","last_change":"2024-09-17T08:34:31.211527+0000","last_active":"2024-09-17T08:34:31.211527+0000","last_peered":"2024-09-17T08:34:31.211527+0000","last_clean":"2024-09-17T08:34:31.211527+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:34:31.211527+0000","last_undegraded":"2024-09-17T08:34:31.211527+0000","last_fullsized":"2024-09-17T08:34:31.211527+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:31.211431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:31.211431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:38.017157+0000","scrub_duration":0.0017236840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.349874+0000","last_change":"2024-09-17T08:33:33.427392+0000","last_active":"2024-09-17T08:34:14.349874+0000","last_peered":"2024-09-17T08:34:14.349874+0000","last_clean":"2024-09-17T08:34:14.349874+0000","last_became_active":"2024-09-17T08:32:43.725035+0000","last_became_peered":"2024-09-17T08:32:43.725035+0000","last_unstale":"2024-09-17T08:34:14.349874+0000","last_undegraded":"2024-09-17T08:34:14.349874+0000","last_fullsized":"2024-09-17T08:34:14.349874+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:33.427308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.427308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:54.408162+0000","scrub_duration":0.001853267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941599+0000","last_change":"2024-09-17T08:33:25.988653+0000","last_active":"2024-09-17T08:34:13.941599+0000","last_peered":"2024-09-17T08:34:13.941599+0000","last_clean":"2024-09-17T08:34:13.941599+0000","last_became_active":"2024-09-17T08:32:43.734497+0000","last_became_peered":"2024-09-17T08:32:43.734497+0000","last_unstale":"2024-09-17T08:34:13.941599+0000","last_undegraded":"2024-09-17T08:34:13.941599+0000","last_fullsized":"2024-09-17T08:34:13.941599+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:25.988536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:25.988536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021364700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"468'14","reported_seq":280,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.284798+0000","last_change":"2024-09-17T08:34:30.284798+0000","last_active":"2024-09-17T08:34:30.284798+0000","last_peered":"2024-09-17T08:34:30.284798+0000","last_clean":"2024-09-17T08:34:30.284798+0000","last_became_active":"2024-09-17T08:32:32.656521+0000","last_became_peered":"2024-09-17T08:32:32.656521+0000","last_unstale":"2024-09-17T08:34:30.284798+0000","last_undegraded":"2024-09-17T08:34:30.284798+0000","last_fullsized":"2024-09-17T08:34:30.284798+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'14","last_scrub_stamp":"2024-09-17T08:34:30.284687+0000","last_deep_scrub":"468'14","last_deep_scrub_stamp":"2024-09-17T08:34:30.284687+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.284687+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:45.057295+0000","scrub_duration":0.0032137440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":233,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.943339+0000","last_change":"2024-09-17T08:33:29.925087+0000","last_active":"2024-09-17T08:34:13.943339+0000","last_peered":"2024-09-17T08:34:13.943339+0000","last_clean":"2024-09-17T08:34:13.943339+0000","last_became_active":"2024-09-17T08:33:29.924785+0000","last_became_peered":"2024-09-17T08:33:29.924785+0000","last_unstale":"2024-09-17T08:34:13.943339+0000","last_undegraded":"2024-09-17T08:34:13.943339+0000","last_fullsized":"2024-09-17T08:34:13.943339+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:03.997048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:03.997048+0000","last_clean_scrub_stamp":"2024-09-17T08:33:03.997048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":311,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:34.306676+0000","last_change":"2024-09-17T08:34:34.306676+0000","last_active":"2024-09-17T08:34:34.306676+0000","last_peered":"2024-09-17T08:34:34.306676+0000","last_clean":"2024-09-17T08:34:34.306676+0000","last_became_active":"2024-09-17T08:32:32.664391+0000","last_became_peered":"2024-09-17T08:32:32.664391+0000","last_unstale":"2024-09-17T08:34:34.306676+0000","last_undegraded":"2024-09-17T08:34:34.306676+0000","last_fullsized":"2024-09-17T08:34:34.306676+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:34.306553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:34:34.306553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:49.216675+0000","scrub_duration":0.001485138,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":500,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:27.142260+0000","last_change":"2024-09-17T08:34:27.142926+0000","last_active":"2024-09-17T08:34:27.142260+0000","last_peered":"2024-09-17T08:34:27.142260+0000","last_clean":"2024-09-17T08:34:27.142260+0000","last_became_active":"2024-09-17T08:33:35.946094+0000","last_became_peered":"2024-09-17T08:33:35.946094+0000","last_unstale":"2024-09-17T08:34:27.142260+0000","last_undegraded":"2024-09-17T08:34:27.142260+0000","last_fullsized":"2024-09-17T08:34:27.142260+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:27.142244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:34:27.142244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:54.682225+0000","scrub_duration":0.001624218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"389'27","reported_seq":76,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941754+0000","last_change":"2024-09-17T08:33:56.761757+0000","last_active":"2024-09-17T08:34:13.941754+0000","last_peered":"2024-09-17T08:34:13.941754+0000","last_clean":"2024-09-17T08:34:13.941754+0000","last_became_active":"2024-09-17T08:32:43.740245+0000","last_became_peered":"2024-09-17T08:32:43.740245+0000","last_unstale":"2024-09-17T08:34:13.941754+0000","last_undegraded":"2024-09-17T08:34:13.941754+0000","last_fullsized":"2024-09-17T08:34:13.941754+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:33:56.761657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.761657+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.222324+0000","scrub_duration":0.0024020830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":335,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.316519+0000","last_change":"2024-09-17T08:34:33.316519+0000","last_active":"2024-09-17T08:34:33.316519+0000","last_peered":"2024-09-17T08:34:33.316519+0000","last_clean":"2024-09-17T08:34:33.316519+0000","last_became_active":"2024-09-17T08:32:32.669069+0000","last_became_peered":"2024-09-17T08:32:32.669069+0000","last_unstale":"2024-09-17T08:34:33.316519+0000","last_undegraded":"2024-09-17T08:34:33.316519+0000","last_fullsized":"2024-09-17T08:34:33.316519+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:34:33.316383+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:34:33.316383+0000","last_clean_scrub_stamp":"2024-09-17T08:34:33.316383+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:41.418917+0000","scrub_duration":0.0021395199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":565,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:09.065411+0000","last_change":"2024-09-17T08:34:09.065411+0000","last_active":"2024-09-17T08:34:09.065411+0000","last_peered":"2024-09-17T08:34:09.065411+0000","last_clean":"2024-09-17T08:34:09.065411+0000","last_became_active":"2024-09-17T08:32:32.656410+0000","last_became_peered":"2024-09-17T08:32:32.656410+0000","last_unstale":"2024-09-17T08:34:09.065411+0000","last_undegraded":"2024-09-17T08:34:09.065411+0000","last_fullsized":"2024-09-17T08:34:09.065411+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:09.065280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:25.622796+0000","last_clean_scrub_stamp":"2024-09-17T08:34:09.065280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:11.510012+0000","scrub_duration":0.001602795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941184+0000","last_change":"2024-09-17T08:32:56.136553+0000","last_active":"2024-09-17T08:34:13.941184+0000","last_peered":"2024-09-17T08:34:13.941184+0000","last_clean":"2024-09-17T08:34:13.941184+0000","last_became_active":"2024-09-17T08:32:43.733334+0000","last_became_peered":"2024-09-17T08:32:43.733334+0000","last_unstale":"2024-09-17T08:34:13.941184+0000","last_undegraded":"2024-09-17T08:34:13.941184+0000","last_fullsized":"2024-09-17T08:34:13.941184+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:56.136470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_clean_scrub_stamp":"2024-09-17T08:32:56.136470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001421093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"468'4","reported_seq":568,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.117199+0000","last_change":"2024-09-17T08:34:18.117320+0000","last_active":"2024-09-17T08:34:18.117199+0000","last_peered":"2024-09-17T08:34:18.117199+0000","last_clean":"2024-09-17T08:34:18.117199+0000","last_became_active":"2024-09-17T08:33:35.946672+0000","last_became_peered":"2024-09-17T08:33:35.946672+0000","last_unstale":"2024-09-17T08:34:18.117199+0000","last_undegraded":"2024-09-17T08:34:18.117199+0000","last_fullsized":"2024-09-17T08:34:18.117199+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'4","last_scrub_stamp":"2024-09-17T08:34:18.117180+0000","last_deep_scrub":"468'4","last_deep_scrub_stamp":"2024-09-17T08:34:18.117180+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.117180+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.877100+0000","scrub_duration":0.0037266669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":147,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621633+0000","last_change":"2024-09-17T08:33:35.007760+0000","last_active":"2024-09-17T08:33:56.621633+0000","last_peered":"2024-09-17T08:33:56.621633+0000","last_clean":"2024-09-17T08:33:56.621633+0000","last_became_active":"2024-09-17T08:33:30.128295+0000","last_became_peered":"2024-09-17T08:33:30.128295+0000","last_unstale":"2024-09-17T08:33:56.621633+0000","last_undegraded":"2024-09-17T08:33:56.621633+0000","last_fullsized":"2024-09-17T08:33:56.621633+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:35.007660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:35.007660+0000","last_clean_scrub_stamp":"2024-09-17T08:33:35.007660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.958982+0000","scrub_duration":0.0014704239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"389'30","reported_seq":631,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:12.632305+0000","last_change":"2024-09-17T08:34:12.632305+0000","last_active":"2024-09-17T08:34:12.632305+0000","last_peered":"2024-09-17T08:34:12.632305+0000","last_clean":"2024-09-17T08:34:12.632305+0000","last_became_active":"2024-09-17T08:32:43.723301+0000","last_became_peered":"2024-09-17T08:32:43.723301+0000","last_unstale":"2024-09-17T08:34:12.632305+0000","last_undegraded":"2024-09-17T08:34:12.632305+0000","last_fullsized":"2024-09-17T08:34:12.632305+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'30","last_scrub_stamp":"2024-09-17T08:34:12.632160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:12.632160+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.397940+0000","scrub_duration":0.002231415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"435'12","reported_seq":315,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.346131+0000","last_change":"2024-09-17T08:34:28.417048+0000","last_active":"2024-09-17T08:34:35.346131+0000","last_peered":"2024-09-17T08:34:35.346131+0000","last_clean":"2024-09-17T08:34:35.346131+0000","last_became_active":"2024-09-17T08:32:32.667694+0000","last_became_peered":"2024-09-17T08:32:32.667694+0000","last_unstale":"2024-09-17T08:34:35.346131+0000","last_undegraded":"2024-09-17T08:34:35.346131+0000","last_fullsized":"2024-09-17T08:34:35.346131+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'12","last_scrub_stamp":"2024-09-17T08:34:28.416906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:34:28.416906+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:30.251662+0000","scrub_duration":0.001784517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":423,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752194+0000","last_change":"2024-09-17T08:33:44.476368+0000","last_active":"2024-09-17T08:34:13.752194+0000","last_peered":"2024-09-17T08:34:13.752194+0000","last_clean":"2024-09-17T08:34:13.752194+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:34:13.752194+0000","last_undegraded":"2024-09-17T08:34:13.752194+0000","last_fullsized":"2024-09-17T08:34:13.752194+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:44.476257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:44.476257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:53.095873+0000","scrub_duration":0.0019968730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941616+0000","last_change":"2024-09-17T08:33:42.744862+0000","last_active":"2024-09-17T08:34:13.941616+0000","last_peered":"2024-09-17T08:34:13.941616+0000","last_clean":"2024-09-17T08:34:13.941616+0000","last_became_active":"2024-09-17T08:32:43.734119+0000","last_became_peered":"2024-09-17T08:32:43.734119+0000","last_unstale":"2024-09-17T08:34:13.941616+0000","last_undegraded":"2024-09-17T08:34:13.941616+0000","last_fullsized":"2024-09-17T08:34:13.941616+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:42.744766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:33:42.744766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:12.195324+0000","scrub_duration":0.0028936869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"389'27","reported_seq":333,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.346213+0000","last_change":"2024-09-17T08:34:08.431596+0000","last_active":"2024-09-17T08:34:35.346213+0000","last_peered":"2024-09-17T08:34:35.346213+0000","last_clean":"2024-09-17T08:34:35.346213+0000","last_became_active":"2024-09-17T08:32:32.668592+0000","last_became_peered":"2024-09-17T08:32:32.668592+0000","last_unstale":"2024-09-17T08:34:35.346213+0000","last_undegraded":"2024-09-17T08:34:35.346213+0000","last_fullsized":"2024-09-17T08:34:35.346213+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:34:08.431471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.431471+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.894819+0000","scrub_duration":0.0022844440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":559,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.784697+0000","last_change":"2024-09-17T08:33:46.606984+0000","last_active":"2024-09-17T08:34:07.784697+0000","last_peered":"2024-09-17T08:34:07.784697+0000","last_clean":"2024-09-17T08:34:07.784697+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:34:07.784697+0000","last_undegraded":"2024-09-17T08:34:07.784697+0000","last_fullsized":"2024-09-17T08:34:07.784697+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:46.606882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:33:46.606882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.424932+0000","scrub_duration":0.0015957689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":535,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748995+0000","last_change":"2024-09-17T08:33:56.230098+0000","last_active":"2024-09-17T08:34:13.748995+0000","last_peered":"2024-09-17T08:34:13.748995+0000","last_clean":"2024-09-17T08:34:13.748995+0000","last_became_active":"2024-09-17T08:32:32.653871+0000","last_became_peered":"2024-09-17T08:32:32.653871+0000","last_unstale":"2024-09-17T08:34:13.748995+0000","last_undegraded":"2024-09-17T08:34:13.748995+0000","last_fullsized":"2024-09-17T08:34:13.748995+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:56.230019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.230019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.952253+0000","scrub_duration":0.001255036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":201,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126723+0000","last_change":"2024-09-17T08:34:18.319335+0000","last_active":"2024-09-17T08:34:33.126723+0000","last_peered":"2024-09-17T08:34:33.126723+0000","last_clean":"2024-09-17T08:34:33.126723+0000","last_became_active":"2024-09-17T08:32:32.666678+0000","last_became_peered":"2024-09-17T08:32:32.666678+0000","last_unstale":"2024-09-17T08:34:33.126723+0000","last_undegraded":"2024-09-17T08:34:33.126723+0000","last_fullsized":"2024-09-17T08:34:33.126723+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:18.319210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.319210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:28.428338+0000","scrub_duration":0.0014244119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"389'46","reported_seq":637,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.005565+0000","last_change":"2024-09-17T08:33:43.626734+0000","last_active":"2024-09-17T08:33:57.005565+0000","last_peered":"2024-09-17T08:33:57.005565+0000","last_clean":"2024-09-17T08:33:57.005565+0000","last_became_active":"2024-09-17T08:33:43.626357+0000","last_became_peered":"2024-09-17T08:33:43.626357+0000","last_unstale":"2024-09-17T08:33:57.005565+0000","last_undegraded":"2024-09-17T08:33:57.005565+0000","last_fullsized":"2024-09-17T08:33:57.005565+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'46","last_scrub_stamp":"2024-09-17T08:33:25.442257+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:33:25.442257+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:27.458260+0000","scrub_duration":0.0069940489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":412,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752285+0000","last_change":"2024-09-17T08:33:09.622678+0000","last_active":"2024-09-17T08:34:13.752285+0000","last_peered":"2024-09-17T08:34:13.752285+0000","last_clean":"2024-09-17T08:34:13.752285+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:34:13.752285+0000","last_undegraded":"2024-09-17T08:34:13.752285+0000","last_fullsized":"2024-09-17T08:34:13.752285+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:09.622583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:20.462452+0000","last_clean_scrub_stamp":"2024-09-17T08:33:09.622583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.139373+0000","scrub_duration":0.0016635459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941689+0000","last_change":"2024-09-17T08:33:22.004704+0000","last_active":"2024-09-17T08:34:13.941689+0000","last_peered":"2024-09-17T08:34:13.941689+0000","last_clean":"2024-09-17T08:34:13.941689+0000","last_became_active":"2024-09-17T08:32:43.727828+0000","last_became_peered":"2024-09-17T08:32:43.727828+0000","last_unstale":"2024-09-17T08:34:13.941689+0000","last_undegraded":"2024-09-17T08:34:13.941689+0000","last_fullsized":"2024-09-17T08:34:13.941689+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:22.004622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:22.004622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001687568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941311+0000","last_change":"2024-09-17T08:33:49.720713+0000","last_active":"2024-09-17T08:34:13.941311+0000","last_peered":"2024-09-17T08:34:13.941311+0000","last_clean":"2024-09-17T08:34:13.941311+0000","last_became_active":"2024-09-17T08:32:43.735242+0000","last_became_peered":"2024-09-17T08:32:43.735242+0000","last_unstale":"2024-09-17T08:34:13.941311+0000","last_undegraded":"2024-09-17T08:34:13.941311+0000","last_fullsized":"2024-09-17T08:34:13.941311+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:49.720631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:49.720631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:13.226529+0000","scrub_duration":0.0045051600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"389'24","reported_seq":65,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941422+0000","last_change":"2024-09-17T08:34:04.759117+0000","last_active":"2024-09-17T08:34:13.941422+0000","last_peered":"2024-09-17T08:34:13.941422+0000","last_clean":"2024-09-17T08:34:13.941422+0000","last_became_active":"2024-09-17T08:32:43.724393+0000","last_became_peered":"2024-09-17T08:32:43.724393+0000","last_unstale":"2024-09-17T08:34:13.941422+0000","last_undegraded":"2024-09-17T08:34:13.941422+0000","last_fullsized":"2024-09-17T08:34:13.941422+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'24","last_scrub_stamp":"2024-09-17T08:34:04.759020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.759020+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.723294+0000","scrub_duration":0.0023702850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":563,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748669+0000","last_change":"2024-09-17T08:33:24.376389+0000","last_active":"2024-09-17T08:34:13.748669+0000","last_peered":"2024-09-17T08:34:13.748669+0000","last_clean":"2024-09-17T08:34:13.748669+0000","last_became_active":"2024-09-17T08:32:43.720912+0000","last_became_peered":"2024-09-17T08:32:43.720912+0000","last_unstale":"2024-09-17T08:34:13.748669+0000","last_undegraded":"2024-09-17T08:34:13.748669+0000","last_fullsized":"2024-09-17T08:34:13.748669+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:24.376228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:31:30.767734+0000","last_clean_scrub_stamp":"2024-09-17T08:33:24.376228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016577860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1538,"num_read_kb":1295,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":8032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1084,"ondisk_log_size":1084,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1134,"num_read_kb":1129,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1430,"ondisk_log_size":1430,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070766,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":776348,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":775166,"kb_avail":92968804,"statfs":{"total":95995035648,"available":95200055296,"internally_reserved":0,"allocated":1200128,"data_stored":849239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":793770443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.71499999999999997,"15min":1.0649999999999999},"min":{"1min":0.35299999999999998,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.73999999999999999,"5min":3.6659999999999999,"15min":17.556999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.81100000000000005,"15min":1.071},"min":{"1min":0.32700000000000001,"5min":0.24199999999999999,"15min":0.22},"max":{"1min":0.83599999999999997,"5min":3.7109999999999999,"15min":18.245999999999999},"last":0.621}]},{"osd":1,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":1.1699999999999999,"15min":1.298},"min":{"1min":0.40000000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.80900000000000005,"5min":21.587,"15min":44.917999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.155,"15min":1.341},"min":{"1min":0.41599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78400000000000003,"5min":21.265000000000001,"15min":44.996000000000002},"last":0.76100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":4.75,"15min":2.2200000000000002},"min":{"1min":0.38500000000000001,"5min":0.27200000000000002,"15min":0.25700000000000001},"max":{"1min":0.75600000000000001,"5min":296.178,"15min":296.178},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":4.5910000000000002,"15min":2.1379999999999999},"min":{"1min":0.45100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.748,"5min":296.27699999999999,"15min":296.27699999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":3.149,"15min":2.3090000000000002},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80500000000000005,"5min":226.595,"15min":226.595},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":3.121,"15min":2.3580000000000001},"min":{"1min":0.42499999999999999,"5min":0.309,"15min":0.27300000000000002},"max":{"1min":0.79700000000000004,"5min":226.65000000000001,"15min":226.65000000000001},"last":0.747}]},{"osd":4,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.68700000000000006,"5min":3.5129999999999999,"15min":4.391},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.56100000000000005,"15min":0.65200000000000002},"min":{"1min":0.36099999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.61199999999999999,"5min":3.4430000000000001,"15min":5.9669999999999996},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.78100000000000003},"min":{"1min":0.36599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68799999999999994,"5min":2.3700000000000001,"15min":34.436},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57999999999999996,"15min":0.83899999999999997},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.26200000000000001},"max":{"1min":0.70099999999999996,"5min":3.4710000000000001,"15min":34.965000000000003},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68500000000000005}]}]},{"osd":6,"up_from":434,"seq":1864015806491,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84384,"kb_used_data":852,"kb_used_omap":4,"kb_used_meta":83515,"kb_avail":93660768,"statfs":{"total":95995035648,"available":95908626432,"internally_reserved":0,"allocated":872448,"data_stored":526946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4441,"internal_metadata":85520039},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.5}]},{"osd":1,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.624},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":41,"seq":176093659448,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":722460,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":721022,"kb_avail":93022692,"statfs":{"total":95995035648,"available":95255236608,"internally_reserved":0,"allocated":1462272,"data_stored":1111228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":738326982},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":2.4430000000000001,"15min":2.5750000000000002},"min":{"1min":0.30099999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.67600000000000005,"5min":162.39500000000001,"15min":162.39500000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":2.4580000000000002,"15min":2.6160000000000001},"min":{"1min":0.35599999999999998,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.64700000000000002,"5min":162.601,"15min":162.601},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.81299999999999994,"15min":1.714},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.64800000000000002,"5min":3.8100000000000001,"15min":66.981999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.92600000000000005,"15min":1.8029999999999999},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.23599999999999999},"max":{"1min":0.68500000000000005,"5min":4.8799999999999999,"15min":66.316000000000003},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.72999999999999998,"15min":1.7969999999999999},"min":{"1min":0.377,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":4.0279999999999996,"15min":131.727},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84399999999999997,"15min":1.962},"min":{"1min":0.36699999999999999,"5min":0.27800000000000002,"15min":0.27200000000000002},"max":{"1min":0.70299999999999996,"5min":4.0090000000000003,"15min":131.50399999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.71899999999999997,"15min":1.6990000000000001},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.25800000000000001},"max":{"1min":0.69999999999999996,"5min":4.4290000000000003,"15min":67.676000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.90700000000000003,"15min":1.7509999999999999},"min":{"1min":0.40200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66900000000000004,"5min":4.1470000000000002,"15min":67.691999999999993},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.58499999999999996,"15min":0.70099999999999996},"min":{"1min":0.25900000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.64700000000000002,"5min":3.6989999999999998,"15min":6.3600000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51600000000000001,"15min":0.65000000000000002},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73599999999999999,"5min":2.085,"15min":6.0899999999999999},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40999999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.93700000000000006,"15min":0.88100000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":25.085000000000001,"15min":25.085000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":23.561,"15min":23.561},"last":0.53800000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953789,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":609508,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":608638,"kb_avail":93135644,"statfs":{"total":95995035648,"available":95370899456,"internally_reserved":0,"allocated":880640,"data_stored":521489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":623245766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.78900000000000003,"15min":0.91400000000000003},"min":{"1min":0.38300000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.88,"5min":4.835,"15min":20.271000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.95099999999999996},"min":{"1min":0.30399999999999999,"5min":0.251,"15min":0.224},"max":{"1min":0.83699999999999997,"5min":4.9240000000000004,"15min":19.798999999999999},"last":0.502}]},{"osd":1,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":2.6269999999999998,"15min":1.671},"min":{"1min":0.36799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85099999999999998,"5min":84.185000000000002,"15min":84.185000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":2.536,"15min":1.581},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.246},"max":{"1min":0.82499999999999996,"5min":84.153999999999996,"15min":84.153999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":2.0009999999999999,"15min":1.2130000000000001},"min":{"1min":0.33100000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77900000000000003,"5min":105.536,"15min":105.536},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":2.0390000000000001,"15min":1.1970000000000001},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":0.88500000000000001,"5min":105.623,"15min":105.623},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.76900000000000002,"15min":1.107},"min":{"1min":0.372,"5min":0.28100000000000003,"15min":0.26300000000000001},"max":{"1min":0.78000000000000003,"5min":4.0810000000000004,"15min":28.138000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.96099999999999997,"15min":1.1759999999999999},"min":{"1min":0.435,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.875,"5min":4.8680000000000003,"15min":28.117000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.71499999999999997},"min":{"1min":0.30199999999999999,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.73999999999999999,"5min":4.2640000000000002,"15min":6.2329999999999997},"last":0.624},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.52600000000000002,"15min":0.66400000000000003},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":2.718,"15min":4.5300000000000002},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65300000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.72599999999999998},"min":{"1min":0.374,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72099999999999997,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.39000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77400000000000002,"5min":3.6869999999999998,"15min":4.6950000000000003},"last":0.52400000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117313,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":812740,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":811262,"kb_avail":92932412,"statfs":{"total":95995035648,"available":95162789888,"internally_reserved":0,"allocated":1499136,"data_stored":1119314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":830732744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":1.052,"15min":2.0499999999999998},"min":{"1min":0.19600000000000001,"5min":0.152,"15min":0.152},"max":{"1min":0.42199999999999999,"5min":8.2620000000000005,"15min":130.40299999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.93500000000000005,"15min":1.7330000000000001},"min":{"1min":0.214,"5min":0.186,"15min":0.17599999999999999},"max":{"1min":0.52900000000000003,"5min":9.4009999999999998,"15min":130.70099999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.3879999999999999,"15min":1.5129999999999999},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.214},"max":{"1min":0.51900000000000002,"5min":20.812999999999999,"15min":71.459999999999994},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":1.345,"15min":1.631},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.192},"max":{"1min":0.49399999999999999,"5min":20.777000000000001,"15min":71.438999999999993},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":1.3169999999999999,"15min":1.4970000000000001},"min":{"1min":0.245,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.495,"5min":21.161000000000001,"15min":60.423999999999999},"last":0.371},{"interface":"front","average":{"1min":0.32400000000000001,"5min":1.351,"15min":1.6459999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":21.292000000000002,"15min":60.368000000000002},"last":0.378}]},{"osd":4,"last update":"Tue Sep 17 08:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.85199999999999998,"15min":1.2130000000000001},"min":{"1min":0.42599999999999999,"5min":0.316,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":6.8479999999999999,"15min":30.722999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.84099999999999997,"15min":1.1359999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27200000000000002},"max":{"1min":0.86699999999999999,"5min":8.7240000000000002,"15min":30.884},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.88600000000000001,"15min":1.111},"min":{"1min":0.35299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":6.9089999999999998,"15min":6.9089999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.86299999999999999,"15min":1.198},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":4.8769999999999998,"15min":9.7289999999999992},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Sep 17 08:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.87,"15min":1.5680000000000001},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":8.6880000000000006,"15min":163.006},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.93500000000000005,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":6.8869999999999996,"15min":162.83699999999999},"last":0.55100000000000005}]}]},{"osd":2,"up_from":227,"seq":974957576364,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":403816,"kb_used_data":1120,"kb_used_omap":4,"kb_used_meta":402683,"kb_avail":93341336,"statfs":{"total":95995035648,"available":95581528064,"internally_reserved":0,"allocated":1146880,"data_stored":784038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":412348067},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.85999999999999999,"15min":1.2190000000000001},"min":{"1min":0.22700000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.54600000000000004,"5min":6.7599999999999998,"15min":9.6389999999999993},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":1.127,"15min":1.3100000000000001},"min":{"1min":0.25700000000000001,"5min":0.192,"15min":0.184},"max":{"1min":0.56499999999999995,"5min":6.2220000000000004,"15min":10.237},"last":0.32200000000000001}]},{"osd":1,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":1.4399999999999999,"15min":1.468},"min":{"1min":0.245,"5min":0.221,"15min":0.19600000000000001},"max":{"1min":0.56100000000000005,"5min":9.2240000000000002,"15min":41.405000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":1.1459999999999999,"15min":1.4510000000000001},"min":{"1min":0.255,"5min":0.217,"15min":0.19500000000000001},"max":{"1min":0.48699999999999999,"5min":8.2759999999999998,"15min":41.25},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.97699999999999998,"15min":1.3959999999999999},"min":{"1min":0.29999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52100000000000002,"5min":5.702,"15min":44.185000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":1.131,"15min":1.5049999999999999},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.54100000000000004,"5min":11.852,"15min":43.994},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.4319999999999999,"15min":1.5840000000000001},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":52.609000000000002,"15min":62.298999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.5149999999999999,"15min":1.536},"min":{"1min":0.40200000000000002,"5min":0.29999999999999999,"15min":0.251},"max":{"1min":0.79300000000000004,"5min":52.570999999999998,"15min":62.469000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.873,"15min":0.97899999999999998},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":6.8529999999999998,"15min":6.8529999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.96999999999999997,"15min":0.95899999999999996},"min":{"1min":0.45500000000000002,"5min":0.33200000000000002,"15min":0.28000000000000003},"max":{"1min":0.73399999999999999,"5min":4.5919999999999996,"15min":4.9770000000000003},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.597,"15min":1.234},"min":{"1min":0.43099999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":0.78700000000000003,"5min":52.055999999999997,"15min":52.055999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.6679999999999999,"15min":1.256},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"max":{"1min":0.73299999999999998,"5min":52.014000000000003,"15min":52.014000000000003},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575179,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":916356,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":915454,"kb_avail":92828796,"statfs":{"total":95995035648,"available":95056687104,"internally_reserved":0,"allocated":909312,"data_stored":529426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":937425350},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":1.0780000000000001,"15min":1.4119999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.155},"max":{"1min":0.621,"5min":7.1609999999999996,"15min":27.670999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.85799999999999998,"15min":1.363},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55000000000000004,"5min":6.5810000000000004,"15min":26.120999999999999},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":1.115,"15min":1.2390000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":6,"15min":26.297999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.37,"5min":1.226,"15min":1.411},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.19900000000000001},"max":{"1min":0.77200000000000002,"5min":8.9900000000000002,"15min":26.602},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":1.5549999999999999,"15min":1.371},"min":{"1min":0.221,"5min":0.221,"15min":0.184},"max":{"1min":0.59399999999999997,"5min":10.470000000000001,"15min":10.470000000000001},"last":0.374},{"interface":"front","average":{"1min":0.40000000000000002,"5min":1.1040000000000001,"15min":1.3220000000000001},"min":{"1min":0.32500000000000001,"5min":0.214,"15min":0.20599999999999999},"max":{"1min":0.65600000000000003,"5min":7.2450000000000001,"15min":22.984999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Sep 17 08:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":2.6219999999999999,"15min":1.7170000000000001},"min":{"1min":0.40699999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79400000000000004,"5min":156.786,"15min":156.786},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":2.8119999999999998,"15min":1.7450000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":156.77699999999999,"15min":156.77699999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":1.216,"15min":1.157},"min":{"1min":0.39500000000000002,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":2.7189999999999999,"5min":9.6769999999999996,"15min":9.6769999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.95199999999999996,"15min":1.1220000000000001},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4470000000000001,"5min":6.2060000000000004,"15min":16.853999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.0049999999999999,"15min":1.387},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.93200000000000005,"5min":4.3920000000000003,"15min":47.442},"last":0.5},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.3009999999999999,"15min":1.448},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35199999999999998},"max":{"1min":2.738,"5min":7.0419999999999998,"15min":47.475000000000001},"last":0.59499999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738701,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":785892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":785022,"kb_avail":92959260,"statfs":{"total":95995035648,"available":95190282240,"internally_reserved":0,"allocated":880640,"data_stored":521445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":803862982},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.996,"15min":1.4399999999999999},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.185},"max":{"1min":0.57999999999999996,"5min":24.738,"15min":41.247},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.3,"15min":1.7},"min":{"1min":0.26600000000000001,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":0.52700000000000002,"5min":26.294,"15min":40.436},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.54,"15min":2.266},"min":{"1min":0.23999999999999999,"5min":0.20000000000000001,"15min":0.17799999999999999},"max":{"1min":0.56000000000000005,"5min":40.359000000000002,"15min":114.771},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":1.3879999999999999,"15min":2.2269999999999999},"min":{"1min":0.26300000000000001,"5min":0.20899999999999999,"15min":0.16700000000000001},"max":{"1min":0.60099999999999998,"5min":40.417000000000002,"15min":114.732},"last":0.36799999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":1.831,"15min":1.7949999999999999},"min":{"1min":0.26100000000000001,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":0.65800000000000003,"5min":44.506,"15min":44.506},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.371,"5min":1.4950000000000001,"15min":1.379},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53600000000000003,"5min":45.286000000000001,"15min":45.286000000000001},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.81100000000000005,"15min":1.3089999999999999},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.26300000000000001},"max":{"1min":0.877,"5min":3.8719999999999999,"15min":54.176000000000002},"last":0.63},{"interface":"front","average":{"1min":0.60899999999999999,"5min":1.012,"15min":1.4019999999999999},"min":{"1min":0.41299999999999998,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.93100000000000005,"5min":5.2300000000000004,"15min":54.097000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.77300000000000002,"15min":1.1140000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.28599999999999998},"max":{"1min":0.84399999999999997,"5min":3.5459999999999998,"15min":39.058},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.75800000000000001,"15min":1.3919999999999999},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.25600000000000001},"max":{"1min":0.95899999999999996,"5min":3.7189999999999999,"15min":54.116},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":1.554,"15min":1.714},"min":{"1min":0.42299999999999999,"5min":0.32300000000000001,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":30.765999999999998,"15min":54.161000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.335,"15min":1.5049999999999999},"min":{"1min":0.50700000000000001,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":30.777999999999999,"15min":53.604999999999997},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:34:36.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T08:34:37.009 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:36 smithi079 ceph-mon[40649]: pgmap v1120: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:37.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:36 smithi079 ceph-mon[40649]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:37.119 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:36 smithi079 ceph-mon[29565]: pgmap v1120: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:37.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:36 smithi079 ceph-mon[29565]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:37.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:36 smithi173 ceph-mon[33187]: pgmap v1120: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:37.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:36 smithi173 ceph-mon[33187]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:38.114 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:37 smithi079 ceph-mon[40649]: 6.1f deep-scrub starts 2024-09-17T08:34:38.114 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:37 smithi079 ceph-mon[40649]: 6.1f deep-scrub ok 2024-09-17T08:34:38.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:37 smithi079 ceph-mon[29565]: 6.1f deep-scrub starts 2024-09-17T08:34:38.114 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:37 smithi079 ceph-mon[29565]: 6.1f deep-scrub ok 2024-09-17T08:34:38.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:37 smithi173 ceph-mon[33187]: 6.1f deep-scrub starts 2024-09-17T08:34:38.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:37 smithi173 ceph-mon[33187]: 6.1f deep-scrub ok 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[29565]: 6.7 scrub starts 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[29565]: 6.7 scrub ok 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[29565]: 2.9 deep-scrub starts 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[29565]: 2.9 deep-scrub ok 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[29565]: pgmap v1121: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[40649]: 6.7 scrub starts 2024-09-17T08:34:39.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[40649]: 6.7 scrub ok 2024-09-17T08:34:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[40649]: 2.9 deep-scrub starts 2024-09-17T08:34:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[40649]: 2.9 deep-scrub ok 2024-09-17T08:34:39.121 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:38 smithi079 ceph-mon[40649]: pgmap v1121: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:34:39.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:38 smithi173 ceph-mon[33187]: 6.7 scrub starts 2024-09-17T08:34:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:38 smithi173 ceph-mon[33187]: 6.7 scrub ok 2024-09-17T08:34:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:38 smithi173 ceph-mon[33187]: 2.9 deep-scrub starts 2024-09-17T08:34:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:38 smithi173 ceph-mon[33187]: 2.9 deep-scrub ok 2024-09-17T08:34:39.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:38 smithi173 ceph-mon[33187]: pgmap v1121: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T08:34:39.239 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-17T08:34:39.241 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-17T08:34:40.013 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[40649]: 6.d scrub starts 2024-09-17T08:34:40.014 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[40649]: 6.d scrub ok 2024-09-17T08:34:40.014 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[40649]: from='client.16302 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:40.014 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[29565]: 6.d scrub starts 2024-09-17T08:34:40.014 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[29565]: 6.d scrub ok 2024-09-17T08:34:40.014 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:39 smithi079 ceph-mon[29565]: from='client.16302 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:40.036 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":1121,"stamp":"2024-09-17T08:34:37.742995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2684,"num_read_kb":2436,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":8551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2634,"ondisk_log_size":2634,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5111536,"kb_used_data":8668,"kb_used_omap":18,"kb_used_meta":5102765,"kb_avail":744849680,"statfs":{"total":767960285184,"available":762726072320,"internally_reserved":0,"allocated":8876032,"data_stored":5979085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18443,"internal_metadata":5225232373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002068"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941384+0000","last_change":"2024-09-17T08:34:02.793177+0000","last_active":"2024-09-17T08:34:13.941384+0000","last_peered":"2024-09-17T08:34:13.941384+0000","last_clean":"2024-09-17T08:34:13.941384+0000","last_became_active":"2024-09-17T08:32:43.740289+0000","last_became_peered":"2024-09-17T08:32:43.740289+0000","last_unstale":"2024-09-17T08:34:13.941384+0000","last_undegraded":"2024-09-17T08:34:13.941384+0000","last_fullsized":"2024-09-17T08:34:13.941384+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:02.793098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:47.923767+0000","last_clean_scrub_stamp":"2024-09-17T08:34:02.793098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:09.068341+0000","scrub_duration":0.001793773,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":545,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621593+0000","last_change":"2024-09-17T08:33:37.625623+0000","last_active":"2024-09-17T08:33:56.621593+0000","last_peered":"2024-09-17T08:33:56.621593+0000","last_clean":"2024-09-17T08:33:56.621593+0000","last_became_active":"2024-09-17T08:09:54.502540+0000","last_became_peered":"2024-09-17T08:09:54.502540+0000","last_unstale":"2024-09-17T08:33:56.621593+0000","last_undegraded":"2024-09-17T08:33:56.621593+0000","last_fullsized":"2024-09-17T08:33:56.621593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:37.625546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:37.625546+0000","last_clean_scrub_stamp":"2024-09-17T08:33:37.625546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.915425+0000","scrub_duration":0.0015847700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"449'46","reported_seq":76,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941951+0000","last_change":"2024-09-17T08:33:20.979714+0000","last_active":"2024-09-17T08:34:13.941951+0000","last_peered":"2024-09-17T08:34:13.941951+0000","last_clean":"2024-09-17T08:34:13.941951+0000","last_became_active":"2024-09-17T08:32:43.735736+0000","last_became_peered":"2024-09-17T08:32:43.735736+0000","last_unstale":"2024-09-17T08:34:13.941951+0000","last_undegraded":"2024-09-17T08:34:13.941951+0000","last_fullsized":"2024-09-17T08:34:13.941951+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"449'46","last_scrub_stamp":"2024-09-17T08:33:20.979618+0000","last_deep_scrub":"449'46","last_deep_scrub_stamp":"2024-09-17T08:33:20.979618+0000","last_clean_scrub_stamp":"2024-09-17T08:33:20.979618+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.412419+0000","scrub_duration":0.0056654280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":563,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748263+0000","last_change":"2024-09-17T08:34:11.211850+0000","last_active":"2024-09-17T08:34:13.748263+0000","last_peered":"2024-09-17T08:34:13.748263+0000","last_clean":"2024-09-17T08:34:13.748263+0000","last_became_active":"2024-09-17T08:32:32.655817+0000","last_became_peered":"2024-09-17T08:32:32.655817+0000","last_unstale":"2024-09-17T08:34:13.748263+0000","last_undegraded":"2024-09-17T08:34:13.748263+0000","last_fullsized":"2024-09-17T08:34:13.748263+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.211748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.211748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.616905+0000","scrub_duration":0.0013268399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.585175+0000","last_change":"2024-09-17T08:34:30.585175+0000","last_active":"2024-09-17T08:34:30.585175+0000","last_peered":"2024-09-17T08:34:30.585175+0000","last_clean":"2024-09-17T08:34:30.585175+0000","last_became_active":"2024-09-17T08:32:43.724367+0000","last_became_peered":"2024-09-17T08:32:43.724367+0000","last_unstale":"2024-09-17T08:34:30.585175+0000","last_undegraded":"2024-09-17T08:34:30.585175+0000","last_fullsized":"2024-09-17T08:34:30.585175+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:30.585076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.591667+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.585076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:58.518150+0000","scrub_duration":0.001590901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"389'25","reported_seq":723,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.274650+0000","last_change":"2024-09-17T08:34:20.274650+0000","last_active":"2024-09-17T08:34:20.274650+0000","last_peered":"2024-09-17T08:34:20.274650+0000","last_clean":"2024-09-17T08:34:20.274650+0000","last_became_active":"2024-09-17T08:32:32.657238+0000","last_became_peered":"2024-09-17T08:32:32.657238+0000","last_unstale":"2024-09-17T08:34:20.274650+0000","last_undegraded":"2024-09-17T08:34:20.274650+0000","last_fullsized":"2024-09-17T08:34:20.274650+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:20.274538+0000","last_deep_scrub":"389'25","last_deep_scrub_stamp":"2024-09-17T08:33:00.607405+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.274538+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:45.475985+0000","scrub_duration":0.002096868,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":412,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751811+0000","last_change":"2024-09-17T08:33:27.421746+0000","last_active":"2024-09-17T08:34:13.751811+0000","last_peered":"2024-09-17T08:34:13.751811+0000","last_clean":"2024-09-17T08:34:13.751811+0000","last_became_active":"2024-09-17T08:15:29.365210+0000","last_became_peered":"2024-09-17T08:15:29.365210+0000","last_unstale":"2024-09-17T08:34:13.751811+0000","last_undegraded":"2024-09-17T08:34:13.751811+0000","last_fullsized":"2024-09-17T08:34:13.751811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:27.421667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:27.421667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.079946+0000","scrub_duration":0.0015319369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":545,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748478+0000","last_change":"2024-09-17T08:33:19.427099+0000","last_active":"2024-09-17T08:34:13.748478+0000","last_peered":"2024-09-17T08:34:13.748478+0000","last_clean":"2024-09-17T08:34:13.748478+0000","last_became_active":"2024-09-17T08:15:29.355689+0000","last_became_peered":"2024-09-17T08:15:29.355689+0000","last_unstale":"2024-09-17T08:34:13.748478+0000","last_undegraded":"2024-09-17T08:34:13.748478+0000","last_fullsized":"2024-09-17T08:34:13.748478+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:19.427014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:28.284981+0000","last_clean_scrub_stamp":"2024-09-17T08:33:19.427014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:46.542638+0000","scrub_duration":0.0015870909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":568,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006250+0000","last_change":"2024-09-17T08:33:56.027534+0000","last_active":"2024-09-17T08:33:57.006250+0000","last_peered":"2024-09-17T08:33:57.006250+0000","last_clean":"2024-09-17T08:33:57.006250+0000","last_became_active":"2024-09-17T08:32:32.662767+0000","last_became_peered":"2024-09-17T08:32:32.662767+0000","last_unstale":"2024-09-17T08:33:57.006250+0000","last_undegraded":"2024-09-17T08:33:57.006250+0000","last_fullsized":"2024-09-17T08:33:57.006250+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:56.027450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:45.384635+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.027450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.835214+0000","scrub_duration":0.001371197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"389'11","reported_seq":609,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:21.248269+0000","last_change":"2024-09-17T08:34:21.248269+0000","last_active":"2024-09-17T08:34:21.248269+0000","last_peered":"2024-09-17T08:34:21.248269+0000","last_clean":"2024-09-17T08:34:21.248269+0000","last_became_active":"2024-09-17T08:32:43.729898+0000","last_became_peered":"2024-09-17T08:32:43.729898+0000","last_unstale":"2024-09-17T08:34:21.248269+0000","last_undegraded":"2024-09-17T08:34:21.248269+0000","last_fullsized":"2024-09-17T08:34:21.248269+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'11","last_scrub_stamp":"2024-09-17T08:34:21.248164+0000","last_deep_scrub":"85'7","last_deep_scrub_stamp":"2024-09-17T08:11:29.578069+0000","last_clean_scrub_stamp":"2024-09-17T08:34:21.248164+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.360498+0000","scrub_duration":0.0019473509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":589,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.162251+0000","last_change":"2024-09-17T08:34:32.162330+0000","last_active":"2024-09-17T08:34:32.162251+0000","last_peered":"2024-09-17T08:34:32.162251+0000","last_clean":"2024-09-17T08:34:32.162251+0000","last_became_active":"2024-09-17T08:15:29.358387+0000","last_became_peered":"2024-09-17T08:15:29.358387+0000","last_unstale":"2024-09-17T08:34:32.162251+0000","last_undegraded":"2024-09-17T08:34:32.162251+0000","last_fullsized":"2024-09-17T08:34:32.162251+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:32.162234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:32.162234+0000","last_clean_scrub_stamp":"2024-09-17T08:34:32.162234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.055336+0000","scrub_duration":0.001400957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":416,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.149569+0000","last_change":"2024-09-17T08:34:18.149654+0000","last_active":"2024-09-17T08:34:18.149569+0000","last_peered":"2024-09-17T08:34:18.149569+0000","last_clean":"2024-09-17T08:34:18.149569+0000","last_became_active":"2024-09-17T08:33:29.919824+0000","last_became_peered":"2024-09-17T08:33:29.919824+0000","last_unstale":"2024-09-17T08:34:18.149569+0000","last_undegraded":"2024-09-17T08:34:18.149569+0000","last_fullsized":"2024-09-17T08:34:18.149569+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:18.149552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.149552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:44.933358+0000","scrub_duration":0.0013843569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":581,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.043536+0000","last_change":"2024-09-17T08:34:13.043536+0000","last_active":"2024-09-17T08:34:13.043536+0000","last_peered":"2024-09-17T08:34:13.043536+0000","last_clean":"2024-09-17T08:34:13.043536+0000","last_became_active":"2024-09-17T08:33:35.947238+0000","last_became_peered":"2024-09-17T08:33:35.947238+0000","last_unstale":"2024-09-17T08:34:13.043536+0000","last_undegraded":"2024-09-17T08:34:13.043536+0000","last_fullsized":"2024-09-17T08:34:13.043536+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:13.043424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:31:43.907526+0000","last_clean_scrub_stamp":"2024-09-17T08:34:13.043424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.360395+0000","scrub_duration":0.001518716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"389'25","reported_seq":607,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748854+0000","last_change":"2024-09-17T08:33:20.460326+0000","last_active":"2024-09-17T08:34:13.748854+0000","last_peered":"2024-09-17T08:34:13.748854+0000","last_clean":"2024-09-17T08:34:13.748854+0000","last_became_active":"2024-09-17T08:32:43.716624+0000","last_became_peered":"2024-09-17T08:32:43.716624+0000","last_unstale":"2024-09-17T08:34:13.748854+0000","last_undegraded":"2024-09-17T08:34:13.748854+0000","last_fullsized":"2024-09-17T08:34:13.748854+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:33:20.460237+0000","last_deep_scrub":"267'19","last_deep_scrub_stamp":"2024-09-17T08:22:36.105550+0000","last_clean_scrub_stamp":"2024-09-17T08:33:20.460237+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:43.172916+0000","scrub_duration":0.0020359200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:00.160240+0000","last_change":"2024-09-17T08:34:00.160240+0000","last_active":"2024-09-17T08:34:00.160240+0000","last_peered":"2024-09-17T08:34:00.160240+0000","last_clean":"2024-09-17T08:34:00.160240+0000","last_became_active":"2024-09-17T08:33:29.926134+0000","last_became_peered":"2024-09-17T08:33:29.926134+0000","last_unstale":"2024-09-17T08:34:00.160240+0000","last_undegraded":"2024-09-17T08:34:00.160240+0000","last_fullsized":"2024-09-17T08:34:00.160240+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:00.160141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:00.160141+0000","last_clean_scrub_stamp":"2024-09-17T08:34:00.160141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:01.181743+0000","scrub_duration":0.0013840759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":556,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.940985+0000","last_change":"2024-09-17T08:34:08.656708+0000","last_active":"2024-09-17T08:34:13.940985+0000","last_peered":"2024-09-17T08:34:13.940985+0000","last_clean":"2024-09-17T08:34:13.940985+0000","last_became_active":"2024-09-17T08:15:29.370344+0000","last_became_peered":"2024-09-17T08:15:29.370344+0000","last_unstale":"2024-09-17T08:34:13.940985+0000","last_undegraded":"2024-09-17T08:34:13.940985+0000","last_fullsized":"2024-09-17T08:34:13.940985+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:08.656623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:08.656623+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.656623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:27.881160+0000","scrub_duration":0.0013164769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"75'130","reported_seq":334,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.525693+0000","last_change":"2024-09-17T08:33:38.946466+0000","last_active":"2024-09-17T08:34:07.525693+0000","last_peered":"2024-09-17T08:34:07.525693+0000","last_clean":"2024-09-17T08:34:07.525693+0000","last_became_active":"2024-09-17T08:33:29.922540+0000","last_became_peered":"2024-09-17T08:33:29.922540+0000","last_unstale":"2024-09-17T08:34:07.525693+0000","last_undegraded":"2024-09-17T08:34:07.525693+0000","last_fullsized":"2024-09-17T08:34:07.525693+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:38.946332+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:04.082189+0000","last_clean_scrub_stamp":"2024-09-17T08:33:38.946332+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.611131+0000","scrub_duration":0.0029773289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"441'53","reported_seq":696,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.126664+0000","last_change":"2024-09-17T08:34:14.126664+0000","last_active":"2024-09-17T08:34:14.126664+0000","last_peered":"2024-09-17T08:34:14.126664+0000","last_clean":"2024-09-17T08:34:14.126664+0000","last_became_active":"2024-09-17T08:20:23.490288+0000","last_became_peered":"2024-09-17T08:20:23.490288+0000","last_unstale":"2024-09-17T08:34:14.126664+0000","last_undegraded":"2024-09-17T08:34:14.126664+0000","last_fullsized":"2024-09-17T08:34:14.126664+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'53","last_scrub_stamp":"2024-09-17T08:34:14.126538+0000","last_deep_scrub":"441'53","last_deep_scrub_stamp":"2024-09-17T08:32:48.696251+0000","last_clean_scrub_stamp":"2024-09-17T08:34:14.126538+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.522869+0000","scrub_duration":0.0029383730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":492,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:15.105153+0000","last_change":"2024-09-17T08:34:15.105241+0000","last_active":"2024-09-17T08:34:15.105153+0000","last_peered":"2024-09-17T08:34:15.105153+0000","last_clean":"2024-09-17T08:34:15.105153+0000","last_became_active":"2024-09-17T08:33:35.943108+0000","last_became_peered":"2024-09-17T08:33:35.943108+0000","last_unstale":"2024-09-17T08:34:15.105153+0000","last_undegraded":"2024-09-17T08:34:15.105153+0000","last_fullsized":"2024-09-17T08:34:15.105153+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:15.105136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:15.105136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.748291+0000","scrub_duration":0.0016169859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":424,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752168+0000","last_change":"2024-09-17T08:33:48.336927+0000","last_active":"2024-09-17T08:34:13.752168+0000","last_peered":"2024-09-17T08:34:13.752168+0000","last_clean":"2024-09-17T08:34:13.752168+0000","last_became_active":"2024-09-17T08:15:29.394748+0000","last_became_peered":"2024-09-17T08:15:29.394748+0000","last_unstale":"2024-09-17T08:34:13.752168+0000","last_undegraded":"2024-09-17T08:34:13.752168+0000","last_fullsized":"2024-09-17T08:34:13.752168+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:48.336829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.336829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:14.362638+0000","scrub_duration":0.001767219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":520,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:34.110166+0000","last_change":"2024-09-17T08:34:34.110166+0000","last_active":"2024-09-17T08:34:34.110166+0000","last_peered":"2024-09-17T08:34:34.110166+0000","last_clean":"2024-09-17T08:34:34.110166+0000","last_became_active":"2024-09-17T08:33:43.628507+0000","last_became_peered":"2024-09-17T08:33:43.628507+0000","last_unstale":"2024-09-17T08:34:34.110166+0000","last_undegraded":"2024-09-17T08:34:34.110166+0000","last_fullsized":"2024-09-17T08:34:34.110166+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:34.110088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:11.020257+0000","last_clean_scrub_stamp":"2024-09-17T08:34:34.110088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:34.928918+0000","scrub_duration":0.0015026880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'130","reported_seq":830,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.005407+0000","last_change":"2024-09-17T08:33:40.996286+0000","last_active":"2024-09-17T08:33:57.005407+0000","last_peered":"2024-09-17T08:33:57.005407+0000","last_clean":"2024-09-17T08:33:57.005407+0000","last_became_active":"2024-09-17T08:33:35.945893+0000","last_became_peered":"2024-09-17T08:33:35.945893+0000","last_unstale":"2024-09-17T08:33:57.005407+0000","last_undegraded":"2024-09-17T08:33:57.005407+0000","last_fullsized":"2024-09-17T08:33:57.005407+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:40.996158+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:40.996158+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:03.023145+0000","scrub_duration":0.0020294979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"389'35","reported_seq":639,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748701+0000","last_change":"2024-09-17T08:33:33.388095+0000","last_active":"2024-09-17T08:34:13.748701+0000","last_peered":"2024-09-17T08:34:13.748701+0000","last_clean":"2024-09-17T08:34:13.748701+0000","last_became_active":"2024-09-17T08:32:43.723164+0000","last_became_peered":"2024-09-17T08:32:43.723164+0000","last_unstale":"2024-09-17T08:34:13.748701+0000","last_undegraded":"2024-09-17T08:34:13.748701+0000","last_fullsized":"2024-09-17T08:34:13.748701+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'35","last_scrub_stamp":"2024-09-17T08:33:33.387988+0000","last_deep_scrub":"389'35","last_deep_scrub_stamp":"2024-09-17T08:31:35.795025+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.387988+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:46.096005+0000","scrub_duration":0.0021778019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751941+0000","last_change":"2024-09-17T08:33:26.451014+0000","last_active":"2024-09-17T08:34:13.751941+0000","last_peered":"2024-09-17T08:34:13.751941+0000","last_clean":"2024-09-17T08:34:13.751941+0000","last_became_active":"2024-09-17T08:32:43.724631+0000","last_became_peered":"2024-09-17T08:32:43.724631+0000","last_unstale":"2024-09-17T08:34:13.751941+0000","last_undegraded":"2024-09-17T08:34:13.751941+0000","last_fullsized":"2024-09-17T08:34:13.751941+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:26.450926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:46.188960+0000","last_clean_scrub_stamp":"2024-09-17T08:33:26.450926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:56.297495+0000","scrub_duration":0.001529709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"461'14","reported_seq":799,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126514+0000","last_change":"2024-09-17T08:33:43.627521+0000","last_active":"2024-09-17T08:34:33.126514+0000","last_peered":"2024-09-17T08:34:33.126514+0000","last_clean":"2024-09-17T08:34:33.126514+0000","last_became_active":"2024-09-17T08:33:43.626800+0000","last_became_peered":"2024-09-17T08:33:43.626800+0000","last_unstale":"2024-09-17T08:34:33.126514+0000","last_undegraded":"2024-09-17T08:34:33.126514+0000","last_fullsized":"2024-09-17T08:34:33.126514+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'11","last_scrub_stamp":"2024-09-17T08:33:33.074745+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-17T08:15:49.311199+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.074745+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.351318+0000","scrub_duration":0.00170889,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":551,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748932+0000","last_change":"2024-09-17T08:33:41.379831+0000","last_active":"2024-09-17T08:34:13.748932+0000","last_peered":"2024-09-17T08:34:13.748932+0000","last_clean":"2024-09-17T08:34:13.748932+0000","last_became_active":"2024-09-17T08:20:23.496680+0000","last_became_peered":"2024-09-17T08:20:23.496680+0000","last_unstale":"2024-09-17T08:34:13.748932+0000","last_undegraded":"2024-09-17T08:34:13.748932+0000","last_fullsized":"2024-09-17T08:34:13.748932+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:41.379728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:48.545686+0000","last_clean_scrub_stamp":"2024-09-17T08:33:41.379728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.381031+0000","scrub_duration":0.001680792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'130","reported_seq":334,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.807175+0000","last_change":"2024-09-17T08:33:39.960080+0000","last_active":"2024-09-17T08:34:07.807175+0000","last_peered":"2024-09-17T08:34:07.807175+0000","last_clean":"2024-09-17T08:34:07.807175+0000","last_became_active":"2024-09-17T08:33:29.924947+0000","last_became_peered":"2024-09-17T08:33:29.924947+0000","last_unstale":"2024-09-17T08:34:07.807175+0000","last_undegraded":"2024-09-17T08:34:07.807175+0000","last_fullsized":"2024-09-17T08:34:07.807175+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:39.959922+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:39.959922+0000","last_clean_scrub_stamp":"2024-09-17T08:33:39.959922+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.413246+0000","scrub_duration":0.0015588970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"390'46","reported_seq":414,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126595+0000","last_change":"2024-09-17T08:34:06.322261+0000","last_active":"2024-09-17T08:34:33.126595+0000","last_peered":"2024-09-17T08:34:33.126595+0000","last_clean":"2024-09-17T08:34:33.126595+0000","last_became_active":"2024-09-17T08:33:43.628298+0000","last_became_peered":"2024-09-17T08:33:43.628298+0000","last_unstale":"2024-09-17T08:34:33.126595+0000","last_undegraded":"2024-09-17T08:34:33.126595+0000","last_fullsized":"2024-09-17T08:34:33.126595+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"390'46","last_scrub_stamp":"2024-09-17T08:34:06.322182+0000","last_deep_scrub":"390'46","last_deep_scrub_stamp":"2024-09-17T08:32:48.046645+0000","last_clean_scrub_stamp":"2024-09-17T08:34:06.322182+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.119548+0000","scrub_duration":0.0024513059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751748+0000","last_change":"2024-09-17T08:33:11.641804+0000","last_active":"2024-09-17T08:34:13.751748+0000","last_peered":"2024-09-17T08:34:13.751748+0000","last_clean":"2024-09-17T08:34:13.751748+0000","last_became_active":"2024-09-17T08:32:43.739708+0000","last_became_peered":"2024-09-17T08:32:43.739708+0000","last_unstale":"2024-09-17T08:34:13.751748+0000","last_undegraded":"2024-09-17T08:34:13.751748+0000","last_fullsized":"2024-09-17T08:34:13.751748+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:11.641709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:11.641709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001805903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"252'9","reported_seq":834,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:28.123872+0000","last_change":"2024-09-17T08:34:12.169517+0000","last_active":"2024-09-17T08:34:28.123872+0000","last_peered":"2024-09-17T08:34:28.123872+0000","last_clean":"2024-09-17T08:34:28.123872+0000","last_became_active":"2024-09-17T08:09:58.507775+0000","last_became_peered":"2024-09-17T08:09:58.507775+0000","last_unstale":"2024-09-17T08:34:28.123872+0000","last_undegraded":"2024-09-17T08:34:28.123872+0000","last_fullsized":"2024-09-17T08:34:28.123872+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'9","last_scrub_stamp":"2024-09-17T08:34:12.169395+0000","last_deep_scrub":"252'9","last_deep_scrub_stamp":"2024-09-17T08:32:26.893190+0000","last_clean_scrub_stamp":"2024-09-17T08:34:12.169395+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.055112+0000","scrub_duration":0.001885255,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":130,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621637+0000","last_change":"2024-09-17T08:33:29.923536+0000","last_active":"2024-09-17T08:33:56.621637+0000","last_peered":"2024-09-17T08:33:56.621637+0000","last_clean":"2024-09-17T08:33:56.621637+0000","last_became_active":"2024-09-17T08:33:29.923207+0000","last_became_peered":"2024-09-17T08:33:29.923207+0000","last_unstale":"2024-09-17T08:33:56.621637+0000","last_undegraded":"2024-09-17T08:33:56.621637+0000","last_fullsized":"2024-09-17T08:33:56.621637+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:18.584274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:18.584274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:21.742281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"389'32","reported_seq":638,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.225988+0000","last_change":"2024-09-17T08:34:19.225988+0000","last_active":"2024-09-17T08:34:19.225988+0000","last_peered":"2024-09-17T08:34:19.225988+0000","last_clean":"2024-09-17T08:34:19.225988+0000","last_became_active":"2024-09-17T08:32:32.650732+0000","last_became_peered":"2024-09-17T08:32:32.650732+0000","last_unstale":"2024-09-17T08:34:19.225988+0000","last_undegraded":"2024-09-17T08:34:19.225988+0000","last_fullsized":"2024-09-17T08:34:19.225988+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'32","last_scrub_stamp":"2024-09-17T08:34:19.225854+0000","last_deep_scrub":"156'16","last_deep_scrub_stamp":"2024-09-17T08:15:44.029355+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.225854+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:43.080057+0000","scrub_duration":0.0022183519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":499,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006461+0000","last_change":"2024-09-17T08:33:55.012426+0000","last_active":"2024-09-17T08:33:57.006461+0000","last_peered":"2024-09-17T08:33:57.006461+0000","last_clean":"2024-09-17T08:33:57.006461+0000","last_became_active":"2024-09-17T08:33:35.942593+0000","last_became_peered":"2024-09-17T08:33:35.942593+0000","last_unstale":"2024-09-17T08:33:57.006461+0000","last_undegraded":"2024-09-17T08:33:57.006461+0000","last_fullsized":"2024-09-17T08:33:57.006461+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:55.012273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:40.836520+0000","last_clean_scrub_stamp":"2024-09-17T08:33:55.012273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.482273+0000","scrub_duration":0.0030888970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":409,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622035+0000","last_change":"2024-09-17T08:33:30.129443+0000","last_active":"2024-09-17T08:33:56.622035+0000","last_peered":"2024-09-17T08:33:56.622035+0000","last_clean":"2024-09-17T08:33:56.622035+0000","last_became_active":"2024-09-17T08:33:30.128925+0000","last_became_peered":"2024-09-17T08:33:30.128925+0000","last_unstale":"2024-09-17T08:33:56.622035+0000","last_undegraded":"2024-09-17T08:33:56.622035+0000","last_fullsized":"2024-09-17T08:33:56.622035+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:02.592944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:30.522581+0000","last_clean_scrub_stamp":"2024-09-17T08:33:02.592944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:27.673186+0000","scrub_duration":0.0089623930000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941933+0000","last_change":"2024-09-17T08:33:37.809677+0000","last_active":"2024-09-17T08:34:13.941933+0000","last_peered":"2024-09-17T08:34:13.941933+0000","last_clean":"2024-09-17T08:34:13.941933+0000","last_became_active":"2024-09-17T08:32:43.729208+0000","last_became_peered":"2024-09-17T08:32:43.729208+0000","last_unstale":"2024-09-17T08:34:13.941933+0000","last_undegraded":"2024-09-17T08:34:13.941933+0000","last_fullsized":"2024-09-17T08:34:13.941933+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:37.809568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:21.692268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:37.809568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.772366+0000","scrub_duration":0.001885627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'130","reported_seq":351,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.942015+0000","last_change":"2024-09-17T08:33:39.785292+0000","last_active":"2024-09-17T08:34:13.942015+0000","last_peered":"2024-09-17T08:34:13.942015+0000","last_clean":"2024-09-17T08:34:13.942015+0000","last_became_active":"2024-09-17T08:32:46.735714+0000","last_became_peered":"2024-09-17T08:32:46.735714+0000","last_unstale":"2024-09-17T08:34:13.942015+0000","last_undegraded":"2024-09-17T08:34:13.942015+0000","last_fullsized":"2024-09-17T08:34:13.942015+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:39.785202+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:28:22.668701+0000","last_clean_scrub_stamp":"2024-09-17T08:33:39.785202+0000","objects_scrubbed":1,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:49.471841+0000","scrub_duration":0.0023755920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"389'49","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.241886+0000","last_change":"2024-09-17T08:34:22.241886+0000","last_active":"2024-09-17T08:34:22.241886+0000","last_peered":"2024-09-17T08:34:22.241886+0000","last_clean":"2024-09-17T08:34:22.241886+0000","last_became_active":"2024-09-17T08:32:32.659219+0000","last_became_peered":"2024-09-17T08:32:32.659219+0000","last_unstale":"2024-09-17T08:34:22.241886+0000","last_undegraded":"2024-09-17T08:34:22.241886+0000","last_fullsized":"2024-09-17T08:34:22.241886+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'49","last_scrub_stamp":"2024-09-17T08:34:22.241752+0000","last_deep_scrub":"349'35","last_deep_scrub_stamp":"2024-09-17T08:27:39.245084+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.241752+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:44.566917+0000","scrub_duration":0.0023798640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":389,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128268+0000","last_change":"2024-09-17T08:33:59.514914+0000","last_active":"2024-09-17T08:34:33.128268+0000","last_peered":"2024-09-17T08:34:33.128268+0000","last_clean":"2024-09-17T08:34:33.128268+0000","last_became_active":"2024-09-17T08:32:32.667307+0000","last_became_peered":"2024-09-17T08:32:32.667307+0000","last_unstale":"2024-09-17T08:34:33.128268+0000","last_undegraded":"2024-09-17T08:34:33.128268+0000","last_fullsized":"2024-09-17T08:34:33.128268+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:59.514789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:24:31.660592+0000","last_clean_scrub_stamp":"2024-09-17T08:33:59.514789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.885022+0000","scrub_duration":0.001575105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"438'13","reported_seq":80,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.808301+0000","last_change":"2024-09-17T08:33:57.801381+0000","last_active":"2024-09-17T08:34:32.808301+0000","last_peered":"2024-09-17T08:34:32.808301+0000","last_clean":"2024-09-17T08:34:32.808301+0000","last_became_active":"2024-09-17T08:32:43.722709+0000","last_became_peered":"2024-09-17T08:32:43.722709+0000","last_unstale":"2024-09-17T08:34:32.808301+0000","last_undegraded":"2024-09-17T08:34:32.808301+0000","last_fullsized":"2024-09-17T08:34:32.808301+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"438'13","last_scrub_stamp":"2024-09-17T08:33:57.801285+0000","last_deep_scrub":"438'13","last_deep_scrub_stamp":"2024-09-17T08:33:57.801285+0000","last_clean_scrub_stamp":"2024-09-17T08:33:57.801285+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:06.984521+0000","scrub_duration":0.002669893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":580,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:10.036971+0000","last_change":"2024-09-17T08:34:10.036971+0000","last_active":"2024-09-17T08:34:10.036971+0000","last_peered":"2024-09-17T08:34:10.036971+0000","last_clean":"2024-09-17T08:34:10.036971+0000","last_became_active":"2024-09-17T08:32:32.663724+0000","last_became_peered":"2024-09-17T08:32:32.663724+0000","last_unstale":"2024-09-17T08:34:10.036971+0000","last_undegraded":"2024-09-17T08:34:10.036971+0000","last_fullsized":"2024-09-17T08:34:10.036971+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.036857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:10.036857+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.036857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.389069+0000","scrub_duration":0.0015775369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'130","reported_seq":476,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748588+0000","last_change":"2024-09-17T08:33:28.447426+0000","last_active":"2024-09-17T08:34:13.748588+0000","last_peered":"2024-09-17T08:34:13.748588+0000","last_clean":"2024-09-17T08:34:13.748588+0000","last_became_active":"2024-09-17T08:32:32.654594+0000","last_became_peered":"2024-09-17T08:32:32.654594+0000","last_unstale":"2024-09-17T08:34:13.748588+0000","last_undegraded":"2024-09-17T08:34:13.748588+0000","last_fullsized":"2024-09-17T08:34:13.748588+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:28.447311+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:31:46.794529+0000","last_clean_scrub_stamp":"2024-09-17T08:33:28.447311+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:47.424437+0000","scrub_duration":0.0015252760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"389'42","reported_seq":71,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941066+0000","last_change":"2024-09-17T08:34:01.843260+0000","last_active":"2024-09-17T08:34:13.941066+0000","last_peered":"2024-09-17T08:34:13.941066+0000","last_clean":"2024-09-17T08:34:13.941066+0000","last_became_active":"2024-09-17T08:32:43.718925+0000","last_became_peered":"2024-09-17T08:32:43.718925+0000","last_unstale":"2024-09-17T08:34:13.941066+0000","last_undegraded":"2024-09-17T08:34:13.941066+0000","last_fullsized":"2024-09-17T08:34:13.941066+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'42","last_scrub_stamp":"2024-09-17T08:34:01.843171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:01.843171+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.113360+0000","scrub_duration":0.0026746589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":439,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:15.137628+0000","last_change":"2024-09-17T08:34:15.137628+0000","last_active":"2024-09-17T08:34:15.137628+0000","last_peered":"2024-09-17T08:34:15.137628+0000","last_clean":"2024-09-17T08:34:15.137628+0000","last_became_active":"2024-09-17T08:32:32.660955+0000","last_became_peered":"2024-09-17T08:32:32.660955+0000","last_unstale":"2024-09-17T08:34:15.137628+0000","last_undegraded":"2024-09-17T08:34:15.137628+0000","last_fullsized":"2024-09-17T08:34:15.137628+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:15.137550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:29:37.364612+0000","last_clean_scrub_stamp":"2024-09-17T08:34:15.137550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:41.429516+0000","scrub_duration":0.001326121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":549,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748263+0000","last_change":"2024-09-17T08:33:36.352045+0000","last_active":"2024-09-17T08:34:13.748263+0000","last_peered":"2024-09-17T08:34:13.748263+0000","last_clean":"2024-09-17T08:34:13.748263+0000","last_became_active":"2024-09-17T08:32:43.732117+0000","last_became_peered":"2024-09-17T08:32:43.732117+0000","last_unstale":"2024-09-17T08:34:13.748263+0000","last_undegraded":"2024-09-17T08:34:13.748263+0000","last_fullsized":"2024-09-17T08:34:13.748263+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:36.351945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:43.596489+0000","last_clean_scrub_stamp":"2024-09-17T08:33:36.351945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:48.985504+0000","scrub_duration":0.0020477490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":577,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748773+0000","last_change":"2024-09-17T08:33:30.387806+0000","last_active":"2024-09-17T08:34:13.748773+0000","last_peered":"2024-09-17T08:34:13.748773+0000","last_clean":"2024-09-17T08:34:13.748773+0000","last_became_active":"2024-09-17T08:32:43.720906+0000","last_became_peered":"2024-09-17T08:32:43.720906+0000","last_unstale":"2024-09-17T08:34:13.748773+0000","last_undegraded":"2024-09-17T08:34:13.748773+0000","last_fullsized":"2024-09-17T08:34:13.748773+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:30.387707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:30.387707+0000","last_clean_scrub_stamp":"2024-09-17T08:33:30.387707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.786564+0000","scrub_duration":0.001561303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"75'130","reported_seq":475,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127677+0000","last_change":"2024-09-17T08:33:12.831461+0000","last_active":"2024-09-17T08:34:33.127677+0000","last_peered":"2024-09-17T08:34:33.127677+0000","last_clean":"2024-09-17T08:34:33.127677+0000","last_became_active":"2024-09-17T08:32:46.734867+0000","last_became_peered":"2024-09-17T08:32:46.734867+0000","last_unstale":"2024-09-17T08:34:33.127677+0000","last_undegraded":"2024-09-17T08:34:33.127677+0000","last_fullsized":"2024-09-17T08:34:33.127677+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:12.831216+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:31:33.537488+0000","last_clean_scrub_stamp":"2024-09-17T08:33:12.831216+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.830657+0000","scrub_duration":0.001778567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"389'39","reported_seq":609,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622023+0000","last_change":"2024-09-17T08:33:31.734577+0000","last_active":"2024-09-17T08:33:56.622023+0000","last_peered":"2024-09-17T08:33:56.622023+0000","last_clean":"2024-09-17T08:33:56.622023+0000","last_became_active":"2024-09-17T08:09:57.042009+0000","last_became_peered":"2024-09-17T08:09:57.042009+0000","last_unstale":"2024-09-17T08:33:56.622023+0000","last_undegraded":"2024-09-17T08:33:56.622023+0000","last_fullsized":"2024-09-17T08:33:56.622023+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'39","last_scrub_stamp":"2024-09-17T08:33:31.734487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.734487+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:47.409141+0000","scrub_duration":0.0030547149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"328'2","reported_seq":562,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:58.066945+0000","last_change":"2024-09-17T08:33:58.066945+0000","last_active":"2024-09-17T08:33:58.066945+0000","last_peered":"2024-09-17T08:33:58.066945+0000","last_clean":"2024-09-17T08:33:58.066945+0000","last_became_active":"2024-09-17T08:09:54.506529+0000","last_became_peered":"2024-09-17T08:09:54.506529+0000","last_unstale":"2024-09-17T08:33:58.066945+0000","last_undegraded":"2024-09-17T08:33:58.066945+0000","last_fullsized":"2024-09-17T08:33:58.066945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'2","last_scrub_stamp":"2024-09-17T08:33:58.066822+0000","last_deep_scrub":"328'2","last_deep_scrub_stamp":"2024-09-17T08:26:27.789796+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.066822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:03.198571+0000","scrub_duration":0.0019489920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":190,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:10.282195+0000","last_change":"2024-09-17T08:34:10.282195+0000","last_active":"2024-09-17T08:34:10.282195+0000","last_peered":"2024-09-17T08:34:10.282195+0000","last_clean":"2024-09-17T08:34:10.282195+0000","last_became_active":"2024-09-17T08:33:29.925585+0000","last_became_peered":"2024-09-17T08:33:29.925585+0000","last_unstale":"2024-09-17T08:34:10.282195+0000","last_undegraded":"2024-09-17T08:34:10.282195+0000","last_fullsized":"2024-09-17T08:34:10.282195+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.282097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:44.468114+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.282097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.953681+0000","scrub_duration":0.0016866299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":455,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350406+0000","last_change":"2024-09-17T08:33:58.275497+0000","last_active":"2024-09-17T08:34:19.350406+0000","last_peered":"2024-09-17T08:34:19.350406+0000","last_clean":"2024-09-17T08:34:19.350406+0000","last_became_active":"2024-09-17T08:32:32.670334+0000","last_became_peered":"2024-09-17T08:32:32.670334+0000","last_unstale":"2024-09-17T08:34:19.350406+0000","last_undegraded":"2024-09-17T08:34:19.350406+0000","last_fullsized":"2024-09-17T08:34:19.350406+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:58.275424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.275424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.713250+0000","scrub_duration":0.0013653510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"75'130","reported_seq":471,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128192+0000","last_change":"2024-09-17T08:34:16.249901+0000","last_active":"2024-09-17T08:34:33.128192+0000","last_peered":"2024-09-17T08:34:33.128192+0000","last_clean":"2024-09-17T08:34:33.128192+0000","last_became_active":"2024-09-17T08:32:46.733976+0000","last_became_peered":"2024-09-17T08:32:46.733976+0000","last_unstale":"2024-09-17T08:34:33.128192+0000","last_undegraded":"2024-09-17T08:34:33.128192+0000","last_fullsized":"2024-09-17T08:34:33.128192+0000","mapping_epoch":440,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":441,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:34:16.249788+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:34:16.249788+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.912187+0000","scrub_duration":0.001733671,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"389'35","reported_seq":472,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350443+0000","last_change":"2024-09-17T08:33:49.301865+0000","last_active":"2024-09-17T08:34:19.350443+0000","last_peered":"2024-09-17T08:34:19.350443+0000","last_clean":"2024-09-17T08:34:19.350443+0000","last_became_active":"2024-09-17T08:15:29.387848+0000","last_became_peered":"2024-09-17T08:15:29.387848+0000","last_unstale":"2024-09-17T08:34:19.350443+0000","last_undegraded":"2024-09-17T08:34:19.350443+0000","last_fullsized":"2024-09-17T08:34:19.350443+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'35","last_scrub_stamp":"2024-09-17T08:33:49.301712+0000","last_deep_scrub":"220'25","last_deep_scrub_stamp":"2024-09-17T08:29:30.429348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:49.301712+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.681674+0000","scrub_duration":0.00257275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":425,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.068278+0000","last_change":"2024-09-17T08:34:20.068278+0000","last_active":"2024-09-17T08:34:20.068278+0000","last_peered":"2024-09-17T08:34:20.068278+0000","last_clean":"2024-09-17T08:34:20.068278+0000","last_became_active":"2024-09-17T08:15:29.393959+0000","last_became_peered":"2024-09-17T08:15:29.393959+0000","last_unstale":"2024-09-17T08:34:20.068278+0000","last_undegraded":"2024-09-17T08:34:20.068278+0000","last_fullsized":"2024-09-17T08:34:20.068278+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:20.068140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:19:44.159424+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.068140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.774839+0000","scrub_duration":0.001743334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"435'11","reported_seq":862,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122164+0000","last_change":"2024-09-17T08:33:47.576848+0000","last_active":"2024-09-17T08:34:23.122164+0000","last_peered":"2024-09-17T08:34:23.122164+0000","last_clean":"2024-09-17T08:34:23.122164+0000","last_became_active":"2024-09-17T08:32:32.664207+0000","last_became_peered":"2024-09-17T08:32:32.664207+0000","last_unstale":"2024-09-17T08:34:23.122164+0000","last_undegraded":"2024-09-17T08:34:23.122164+0000","last_fullsized":"2024-09-17T08:34:23.122164+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'11","last_scrub_stamp":"2024-09-17T08:33:47.576751+0000","last_deep_scrub":"213'9","last_deep_scrub_stamp":"2024-09-17T08:23:28.852455+0000","last_clean_scrub_stamp":"2024-09-17T08:33:47.576751+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.314296+0000","scrub_duration":0.0021314350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":574,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:26.757471+0000","last_change":"2024-09-17T08:34:26.757543+0000","last_active":"2024-09-17T08:34:26.757471+0000","last_peered":"2024-09-17T08:34:26.757471+0000","last_clean":"2024-09-17T08:34:26.757471+0000","last_became_active":"2024-09-17T08:32:43.720606+0000","last_became_peered":"2024-09-17T08:32:43.720606+0000","last_unstale":"2024-09-17T08:34:26.757471+0000","last_undegraded":"2024-09-17T08:34:26.757471+0000","last_fullsized":"2024-09-17T08:34:26.757471+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.757459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.757459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:54.097535+0000","scrub_duration":0.001314274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'130","reported_seq":321,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621316+0000","last_change":"2024-09-17T08:33:35.993275+0000","last_active":"2024-09-17T08:33:56.621316+0000","last_peered":"2024-09-17T08:33:56.621316+0000","last_clean":"2024-09-17T08:33:56.621316+0000","last_became_active":"2024-09-17T08:33:29.923602+0000","last_became_peered":"2024-09-17T08:33:29.923602+0000","last_unstale":"2024-09-17T08:33:56.621316+0000","last_undegraded":"2024-09-17T08:33:56.621316+0000","last_fullsized":"2024-09-17T08:33:56.621316+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:35.993184+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:33:35.993184+0000","last_clean_scrub_stamp":"2024-09-17T08:33:35.993184+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:37.358223+0000","scrub_duration":0.001483306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"467'70","reported_seq":822,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.809299+0000","last_change":"2024-09-17T08:34:08.184942+0000","last_active":"2024-09-17T08:34:32.809299+0000","last_peered":"2024-09-17T08:34:32.809299+0000","last_clean":"2024-09-17T08:34:32.809299+0000","last_became_active":"2024-09-17T08:32:43.736276+0000","last_became_peered":"2024-09-17T08:32:43.736276+0000","last_unstale":"2024-09-17T08:34:32.809299+0000","last_undegraded":"2024-09-17T08:34:32.809299+0000","last_fullsized":"2024-09-17T08:34:32.809299+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'70","last_scrub_stamp":"2024-09-17T08:34:08.184827+0000","last_deep_scrub":"467'70","last_deep_scrub_stamp":"2024-09-17T08:34:08.184827+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.184827+0000","objects_scrubbed":10,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.992157+0000","scrub_duration":0.0068099370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":411,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:04.230356+0000","last_change":"2024-09-17T08:34:04.230356+0000","last_active":"2024-09-17T08:34:04.230356+0000","last_peered":"2024-09-17T08:34:04.230356+0000","last_clean":"2024-09-17T08:34:04.230356+0000","last_became_active":"2024-09-17T08:33:29.924321+0000","last_became_peered":"2024-09-17T08:33:29.924321+0000","last_unstale":"2024-09-17T08:34:04.230356+0000","last_undegraded":"2024-09-17T08:34:04.230356+0000","last_fullsized":"2024-09-17T08:34:04.230356+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:04.230259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:53.067191+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.230259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.324120+0000","scrub_duration":0.00140515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":559,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621912+0000","last_change":"2024-09-17T08:33:34.711717+0000","last_active":"2024-09-17T08:33:56.621912+0000","last_peered":"2024-09-17T08:33:56.621912+0000","last_clean":"2024-09-17T08:33:56.621912+0000","last_became_active":"2024-09-17T08:15:29.379535+0000","last_became_peered":"2024-09-17T08:15:29.379535+0000","last_unstale":"2024-09-17T08:33:56.621912+0000","last_undegraded":"2024-09-17T08:33:56.621912+0000","last_fullsized":"2024-09-17T08:33:56.621912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:34.711633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:22.920405+0000","last_clean_scrub_stamp":"2024-09-17T08:33:34.711633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:38.556910+0000","scrub_duration":0.001465062,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941865+0000","last_change":"2024-09-17T08:34:11.793143+0000","last_active":"2024-09-17T08:34:13.941865+0000","last_peered":"2024-09-17T08:34:13.941865+0000","last_clean":"2024-09-17T08:34:13.941865+0000","last_became_active":"2024-09-17T08:32:43.727553+0000","last_became_peered":"2024-09-17T08:32:43.727553+0000","last_unstale":"2024-09-17T08:34:13.941865+0000","last_undegraded":"2024-09-17T08:34:13.941865+0000","last_fullsized":"2024-09-17T08:34:13.941865+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.793062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:23:32.936595+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.793062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:25.899589+0000","scrub_duration":0.001569097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'130","reported_seq":781,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.005407+0000","last_change":"2024-09-17T08:33:43.900682+0000","last_active":"2024-09-17T08:33:57.005407+0000","last_peered":"2024-09-17T08:33:57.005407+0000","last_clean":"2024-09-17T08:33:57.005407+0000","last_became_active":"2024-09-17T08:33:35.942576+0000","last_became_peered":"2024-09-17T08:33:35.942576+0000","last_unstale":"2024-09-17T08:33:57.005407+0000","last_undegraded":"2024-09-17T08:33:57.005407+0000","last_fullsized":"2024-09-17T08:33:57.005407+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:43.900574+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:43.900574+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:54.476276+0000","scrub_duration":0.001917321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"389'21","reported_seq":91,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:16.094444+0000","last_change":"2024-09-17T08:34:16.094444+0000","last_active":"2024-09-17T08:34:16.094444+0000","last_peered":"2024-09-17T08:34:16.094444+0000","last_clean":"2024-09-17T08:34:16.094444+0000","last_became_active":"2024-09-17T08:31:50.001044+0000","last_became_peered":"2024-09-17T08:31:50.001044+0000","last_unstale":"2024-09-17T08:34:16.094444+0000","last_undegraded":"2024-09-17T08:34:16.094444+0000","last_fullsized":"2024-09-17T08:34:16.094444+0000","mapping_epoch":423,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":424,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'21","last_scrub_stamp":"2024-09-17T08:34:16.094348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:16.094348+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.549686+0000","scrub_duration":0.0024298119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"441'6","reported_seq":1600,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.864908+0000","last_change":"2024-09-17T08:34:17.075627+0000","last_active":"2024-09-17T08:34:32.864908+0000","last_peered":"2024-09-17T08:34:32.864908+0000","last_clean":"2024-09-17T08:34:32.864908+0000","last_became_active":"2024-09-17T08:32:32.661051+0000","last_became_peered":"2024-09-17T08:32:32.661051+0000","last_unstale":"2024-09-17T08:34:32.864908+0000","last_undegraded":"2024-09-17T08:34:32.864908+0000","last_fullsized":"2024-09-17T08:34:32.864908+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'6","last_scrub_stamp":"2024-09-17T08:34:17.075546+0000","last_deep_scrub":"441'6","last_deep_scrub_stamp":"2024-09-17T08:32:48.504881+0000","last_clean_scrub_stamp":"2024-09-17T08:34:17.075546+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:42.865178+0000","scrub_duration":0.0016728159999999999,"objects_trimmed":0,"snaptrim_duration":0.0026546989999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1134,"num_read_kb":1129,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":485,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.125777+0000","last_change":"2024-09-17T08:33:51.083094+0000","last_active":"2024-09-17T08:34:33.125777+0000","last_peered":"2024-09-17T08:34:33.125777+0000","last_clean":"2024-09-17T08:34:33.125777+0000","last_became_active":"2024-09-17T08:33:43.736595+0000","last_became_peered":"2024-09-17T08:33:43.736595+0000","last_unstale":"2024-09-17T08:34:33.125777+0000","last_undegraded":"2024-09-17T08:34:33.125777+0000","last_fullsized":"2024-09-17T08:34:33.125777+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:51.083004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:23.081888+0000","last_clean_scrub_stamp":"2024-09-17T08:33:51.083004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.527861+0000","scrub_duration":0.001394564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":551,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126634+0000","last_change":"2024-09-17T08:34:03.266126+0000","last_active":"2024-09-17T08:34:33.126634+0000","last_peered":"2024-09-17T08:34:33.126634+0000","last_clean":"2024-09-17T08:34:33.126634+0000","last_became_active":"2024-09-17T08:33:43.627052+0000","last_became_peered":"2024-09-17T08:33:43.627052+0000","last_unstale":"2024-09-17T08:34:33.126634+0000","last_undegraded":"2024-09-17T08:34:33.126634+0000","last_fullsized":"2024-09-17T08:34:33.126634+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.265988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:18:32.635696+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.265988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.762258+0000","scrub_duration":0.0017509660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":501,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.116086+0000","last_change":"2024-09-17T08:34:19.116175+0000","last_active":"2024-09-17T08:34:19.116086+0000","last_peered":"2024-09-17T08:34:19.116086+0000","last_clean":"2024-09-17T08:34:19.116086+0000","last_became_active":"2024-09-17T08:33:35.944536+0000","last_became_peered":"2024-09-17T08:33:35.944536+0000","last_unstale":"2024-09-17T08:34:19.116086+0000","last_undegraded":"2024-09-17T08:34:19.116086+0000","last_fullsized":"2024-09-17T08:34:19.116086+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:19.116070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:34.623625+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.116070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:32.674800+0000","scrub_duration":0.0015923300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'130","reported_seq":477,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.751852+0000","last_change":"2024-09-17T08:33:31.394511+0000","last_active":"2024-09-17T08:34:13.751852+0000","last_peered":"2024-09-17T08:34:13.751852+0000","last_clean":"2024-09-17T08:34:13.751852+0000","last_became_active":"2024-09-17T08:32:32.664164+0000","last_became_peered":"2024-09-17T08:32:32.664164+0000","last_unstale":"2024-09-17T08:34:13.751852+0000","last_undegraded":"2024-09-17T08:34:13.751852+0000","last_fullsized":"2024-09-17T08:34:13.751852+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:33:31.394422+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.394422+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:53.574905+0000","scrub_duration":0.00180649,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"389'42","reported_seq":679,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.695355+0000","last_change":"2024-09-17T08:34:32.695426+0000","last_active":"2024-09-17T08:34:32.695355+0000","last_peered":"2024-09-17T08:34:32.695355+0000","last_clean":"2024-09-17T08:34:32.695355+0000","last_became_active":"2024-09-17T08:20:23.490027+0000","last_became_peered":"2024-09-17T08:20:23.490027+0000","last_unstale":"2024-09-17T08:34:32.695355+0000","last_undegraded":"2024-09-17T08:34:32.695355+0000","last_fullsized":"2024-09-17T08:34:32.695355+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'42","last_scrub_stamp":"2024-09-17T08:34:32.695342+0000","last_deep_scrub":"389'42","last_deep_scrub_stamp":"2024-09-17T08:33:29.760243+0000","last_clean_scrub_stamp":"2024-09-17T08:34:32.695342+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:53.701113+0000","scrub_duration":0.0019870500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'2","reported_seq":589,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.827367+0000","last_change":"2024-09-17T08:34:22.827478+0000","last_active":"2024-09-17T08:34:22.827367+0000","last_peered":"2024-09-17T08:34:22.827367+0000","last_clean":"2024-09-17T08:34:22.827367+0000","last_became_active":"2024-09-17T08:32:43.738598+0000","last_became_peered":"2024-09-17T08:32:43.738598+0000","last_unstale":"2024-09-17T08:34:22.827367+0000","last_undegraded":"2024-09-17T08:34:22.827367+0000","last_fullsized":"2024-09-17T08:34:22.827367+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-17T08:34:22.827339+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-17T08:16:50.181067+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.827339+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:35.148205+0000","scrub_duration":0.001894073,"objects_trimmed":0,"snaptrim_duration":0.00058547700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":511,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:57.006496+0000","last_change":"2024-09-17T08:33:46.954112+0000","last_active":"2024-09-17T08:33:57.006496+0000","last_peered":"2024-09-17T08:33:57.006496+0000","last_clean":"2024-09-17T08:33:57.006496+0000","last_became_active":"2024-09-17T08:33:35.942772+0000","last_became_peered":"2024-09-17T08:33:35.942772+0000","last_unstale":"2024-09-17T08:33:57.006496+0000","last_undegraded":"2024-09-17T08:33:57.006496+0000","last_fullsized":"2024-09-17T08:33:57.006496+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:46.953992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:35.399515+0000","last_clean_scrub_stamp":"2024-09-17T08:33:46.953992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:06.776876+0000","scrub_duration":0.0025532979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622050+0000","last_change":"2024-09-17T08:33:31.027945+0000","last_active":"2024-09-17T08:33:56.622050+0000","last_peered":"2024-09-17T08:33:56.622050+0000","last_clean":"2024-09-17T08:33:56.622050+0000","last_became_active":"2024-09-17T08:33:29.923055+0000","last_became_peered":"2024-09-17T08:33:29.923055+0000","last_unstale":"2024-09-17T08:33:56.622050+0000","last_undegraded":"2024-09-17T08:33:56.622050+0000","last_fullsized":"2024-09-17T08:33:56.622050+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:31.027838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:31.027838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.760376+0000","scrub_duration":0.0016732629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":458,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:19.350509+0000","last_change":"2024-09-17T08:34:11.209317+0000","last_active":"2024-09-17T08:34:19.350509+0000","last_peered":"2024-09-17T08:34:19.350509+0000","last_clean":"2024-09-17T08:34:19.350509+0000","last_became_active":"2024-09-17T08:32:43.735221+0000","last_became_peered":"2024-09-17T08:32:43.735221+0000","last_unstale":"2024-09-17T08:34:19.350509+0000","last_undegraded":"2024-09-17T08:34:19.350509+0000","last_fullsized":"2024-09-17T08:34:19.350509+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:11.209241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:11.209241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:40.277382+0000","scrub_duration":0.0017053770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"389'45","reported_seq":377,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.345333+0000","last_change":"2024-09-17T08:33:58.554487+0000","last_active":"2024-09-17T08:34:35.345333+0000","last_peered":"2024-09-17T08:34:35.345333+0000","last_clean":"2024-09-17T08:34:35.345333+0000","last_became_active":"2024-09-17T08:32:32.655755+0000","last_became_peered":"2024-09-17T08:32:32.655755+0000","last_unstale":"2024-09-17T08:34:35.345333+0000","last_undegraded":"2024-09-17T08:34:35.345333+0000","last_fullsized":"2024-09-17T08:34:35.345333+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'45","last_scrub_stamp":"2024-09-17T08:33:58.554370+0000","last_deep_scrub":"389'45","last_deep_scrub_stamp":"2024-09-17T08:33:58.554370+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.554370+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:19.186909+0000","scrub_duration":0.0026170529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"419'3","reported_seq":57,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941222+0000","last_change":"2024-09-17T08:33:36.855970+0000","last_active":"2024-09-17T08:34:13.941222+0000","last_peered":"2024-09-17T08:34:13.941222+0000","last_clean":"2024-09-17T08:34:13.941222+0000","last_became_active":"2024-09-17T08:32:43.727064+0000","last_became_peered":"2024-09-17T08:32:43.727064+0000","last_unstale":"2024-09-17T08:34:13.941222+0000","last_undegraded":"2024-09-17T08:34:13.941222+0000","last_fullsized":"2024-09-17T08:34:13.941222+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"419'3","last_scrub_stamp":"2024-09-17T08:33:36.855865+0000","last_deep_scrub":"419'3","last_deep_scrub_stamp":"2024-09-17T08:31:37.564007+0000","last_clean_scrub_stamp":"2024-09-17T08:33:36.855865+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:51.886060+0000","scrub_duration":0.0019571559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.253240+0000","last_change":"2024-09-17T08:34:07.253240+0000","last_active":"2024-09-17T08:34:07.253240+0000","last_peered":"2024-09-17T08:34:07.253240+0000","last_clean":"2024-09-17T08:34:07.253240+0000","last_became_active":"2024-09-17T08:33:29.928307+0000","last_became_peered":"2024-09-17T08:33:29.928307+0000","last_unstale":"2024-09-17T08:34:07.253240+0000","last_undegraded":"2024-09-17T08:34:07.253240+0000","last_fullsized":"2024-09-17T08:34:07.253240+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:07.253146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:16:43.234376+0000","last_clean_scrub_stamp":"2024-09-17T08:34:07.253146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.900418+0000","scrub_duration":0.0015987600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'130","reported_seq":765,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:25.189082+0000","last_change":"2024-09-17T08:34:25.189180+0000","last_active":"2024-09-17T08:34:25.189082+0000","last_peered":"2024-09-17T08:34:25.189082+0000","last_clean":"2024-09-17T08:34:25.189082+0000","last_became_active":"2024-09-17T08:33:35.944526+0000","last_became_peered":"2024-09-17T08:33:35.944526+0000","last_unstale":"2024-09-17T08:34:25.189082+0000","last_undegraded":"2024-09-17T08:34:25.189082+0000","last_fullsized":"2024-09-17T08:34:25.189082+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'130","last_scrub_stamp":"2024-09-17T08:34:25.189065+0000","last_deep_scrub":"75'130","last_deep_scrub_stamp":"2024-09-17T08:24:24.654371+0000","last_clean_scrub_stamp":"2024-09-17T08:34:25.189065+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:28.139259+0000","scrub_duration":0.001807061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":568,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:21.180010+0000","last_change":"2024-09-17T08:34:21.180115+0000","last_active":"2024-09-17T08:34:21.180010+0000","last_peered":"2024-09-17T08:34:21.180010+0000","last_clean":"2024-09-17T08:34:21.180010+0000","last_became_active":"2024-09-17T08:33:35.946777+0000","last_became_peered":"2024-09-17T08:33:35.946777+0000","last_unstale":"2024-09-17T08:34:21.180010+0000","last_undegraded":"2024-09-17T08:34:21.180010+0000","last_fullsized":"2024-09-17T08:34:21.180010+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:21.179992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:21.179992+0000","last_clean_scrub_stamp":"2024-09-17T08:34:21.179992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.445761+0000","scrub_duration":0.0016641130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":130,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.622114+0000","last_change":"2024-09-17T08:33:30.129157+0000","last_active":"2024-09-17T08:33:56.622114+0000","last_peered":"2024-09-17T08:33:56.622114+0000","last_clean":"2024-09-17T08:33:56.622114+0000","last_became_active":"2024-09-17T08:33:30.128228+0000","last_became_peered":"2024-09-17T08:33:30.128228+0000","last_unstale":"2024-09-17T08:33:56.622114+0000","last_undegraded":"2024-09-17T08:33:56.622114+0000","last_fullsized":"2024-09-17T08:33:56.622114+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:15.007821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:15.007821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:21.142102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"390'51","reported_seq":427,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127037+0000","last_change":"2024-09-17T08:33:11.877859+0000","last_active":"2024-09-17T08:34:33.127037+0000","last_peered":"2024-09-17T08:34:33.127037+0000","last_clean":"2024-09-17T08:34:33.127037+0000","last_became_active":"2024-09-17T08:32:32.665974+0000","last_became_peered":"2024-09-17T08:32:32.665974+0000","last_unstale":"2024-09-17T08:34:33.127037+0000","last_undegraded":"2024-09-17T08:34:33.127037+0000","last_fullsized":"2024-09-17T08:34:33.127037+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"390'51","last_scrub_stamp":"2024-09-17T08:33:11.877729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:11.877729+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:38.447825+0000","scrub_duration":0.0027582140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":464,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126517+0000","last_change":"2024-09-17T08:34:30.201519+0000","last_active":"2024-09-17T08:34:33.126517+0000","last_peered":"2024-09-17T08:34:33.126517+0000","last_clean":"2024-09-17T08:34:33.126517+0000","last_became_active":"2024-09-17T08:33:43.737691+0000","last_became_peered":"2024-09-17T08:33:43.737691+0000","last_unstale":"2024-09-17T08:34:33.126517+0000","last_undegraded":"2024-09-17T08:34:33.126517+0000","last_fullsized":"2024-09-17T08:34:33.126517+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:30.201439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:30.201439+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.201439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:57.677602+0000","scrub_duration":0.0018739329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":548,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.125613+0000","last_change":"2024-09-17T08:34:19.315448+0000","last_active":"2024-09-17T08:34:33.125613+0000","last_peered":"2024-09-17T08:34:33.125613+0000","last_clean":"2024-09-17T08:34:33.125613+0000","last_became_active":"2024-09-17T08:20:23.485722+0000","last_became_peered":"2024-09-17T08:20:23.485722+0000","last_unstale":"2024-09-17T08:34:33.125613+0000","last_undegraded":"2024-09-17T08:34:33.125613+0000","last_fullsized":"2024-09-17T08:34:33.125613+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:19.315374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:28.198833+0000","last_clean_scrub_stamp":"2024-09-17T08:34:19.315374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.174069+0000","scrub_duration":0.0016805679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":494,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:17.130432+0000","last_change":"2024-09-17T08:34:17.130527+0000","last_active":"2024-09-17T08:34:17.130432+0000","last_peered":"2024-09-17T08:34:17.130432+0000","last_clean":"2024-09-17T08:34:17.130432+0000","last_became_active":"2024-09-17T08:33:35.945813+0000","last_became_peered":"2024-09-17T08:33:35.945813+0000","last_unstale":"2024-09-17T08:34:17.130432+0000","last_undegraded":"2024-09-17T08:34:17.130432+0000","last_fullsized":"2024-09-17T08:34:17.130432+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:17.130400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:17.130400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.878747+0000","scrub_duration":0.001414242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"468'25","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.781037+0000","last_change":"2024-09-17T08:34:23.781128+0000","last_active":"2024-09-17T08:34:23.781037+0000","last_peered":"2024-09-17T08:34:23.781037+0000","last_clean":"2024-09-17T08:34:23.781037+0000","last_became_active":"2024-09-17T08:15:29.379731+0000","last_became_peered":"2024-09-17T08:15:29.379731+0000","last_unstale":"2024-09-17T08:34:23.781037+0000","last_undegraded":"2024-09-17T08:34:23.781037+0000","last_fullsized":"2024-09-17T08:34:23.781037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'25","last_scrub_stamp":"2024-09-17T08:34:23.781020+0000","last_deep_scrub":"468'25","last_deep_scrub_stamp":"2024-09-17T08:34:23.781020+0000","last_clean_scrub_stamp":"2024-09-17T08:34:23.781020+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.965648+0000","scrub_duration":0.0047644130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":442,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752158+0000","last_change":"2024-09-17T08:33:32.444569+0000","last_active":"2024-09-17T08:34:13.752158+0000","last_peered":"2024-09-17T08:34:13.752158+0000","last_clean":"2024-09-17T08:34:13.752158+0000","last_became_active":"2024-09-17T08:32:43.726028+0000","last_became_peered":"2024-09-17T08:32:43.726028+0000","last_unstale":"2024-09-17T08:34:13.752158+0000","last_undegraded":"2024-09-17T08:34:13.752158+0000","last_fullsized":"2024-09-17T08:34:13.752158+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:32.444495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:40.994687+0000","last_clean_scrub_stamp":"2024-09-17T08:33:32.444495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:45.215832+0000","scrub_duration":0.0015980600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":521,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126185+0000","last_change":"2024-09-17T08:34:23.288287+0000","last_active":"2024-09-17T08:34:33.126185+0000","last_peered":"2024-09-17T08:34:33.126185+0000","last_clean":"2024-09-17T08:34:33.126185+0000","last_became_active":"2024-09-17T08:33:43.629024+0000","last_became_peered":"2024-09-17T08:33:43.629024+0000","last_unstale":"2024-09-17T08:34:33.126185+0000","last_undegraded":"2024-09-17T08:34:33.126185+0000","last_fullsized":"2024-09-17T08:34:33.126185+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:23.288196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:23.288196+0000","last_clean_scrub_stamp":"2024-09-17T08:34:23.288196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:40.271669+0000","scrub_duration":0.0015586490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":554,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.647087+0000","last_change":"2024-09-17T08:34:13.647087+0000","last_active":"2024-09-17T08:34:13.647087+0000","last_peered":"2024-09-17T08:34:13.647087+0000","last_clean":"2024-09-17T08:34:13.647087+0000","last_became_active":"2024-09-17T08:20:23.488202+0000","last_became_peered":"2024-09-17T08:20:23.488202+0000","last_unstale":"2024-09-17T08:34:13.647087+0000","last_undegraded":"2024-09-17T08:34:13.647087+0000","last_fullsized":"2024-09-17T08:34:13.647087+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:13.646989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:34:13.646989+0000","last_clean_scrub_stamp":"2024-09-17T08:34:13.646989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:29.988115+0000","scrub_duration":0.0015587999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"389'46","reported_seq":562,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.219371+0000","last_change":"2024-09-17T08:34:30.219459+0000","last_active":"2024-09-17T08:34:30.219371+0000","last_peered":"2024-09-17T08:34:30.219371+0000","last_clean":"2024-09-17T08:34:30.219371+0000","last_became_active":"2024-09-17T08:33:29.926228+0000","last_became_peered":"2024-09-17T08:33:29.926228+0000","last_unstale":"2024-09-17T08:34:30.219371+0000","last_undegraded":"2024-09-17T08:34:30.219371+0000","last_fullsized":"2024-09-17T08:34:30.219371+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'46","last_scrub_stamp":"2024-09-17T08:34:30.219352+0000","last_deep_scrub":"389'46","last_deep_scrub_stamp":"2024-09-17T08:34:30.219352+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.219352+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:59.962987+0000","scrub_duration":0.0026729029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":509,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126895+0000","last_change":"2024-09-17T08:33:43.627938+0000","last_active":"2024-09-17T08:34:33.126895+0000","last_peered":"2024-09-17T08:34:33.126895+0000","last_clean":"2024-09-17T08:34:33.126895+0000","last_became_active":"2024-09-17T08:33:43.627449+0000","last_became_peered":"2024-09-17T08:33:43.627449+0000","last_unstale":"2024-09-17T08:34:33.126895+0000","last_undegraded":"2024-09-17T08:34:33.126895+0000","last_fullsized":"2024-09-17T08:34:33.126895+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:16.966496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:16.966496+0000","last_clean_scrub_stamp":"2024-09-17T08:33:16.966496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:42.809931+0000","scrub_duration":0.0035556149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"435'11","reported_seq":872,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122522+0000","last_change":"2024-09-17T08:34:01.124122+0000","last_active":"2024-09-17T08:34:23.122522+0000","last_peered":"2024-09-17T08:34:23.122522+0000","last_clean":"2024-09-17T08:34:23.122522+0000","last_became_active":"2024-09-17T08:32:32.655178+0000","last_became_peered":"2024-09-17T08:32:32.655178+0000","last_unstale":"2024-09-17T08:34:23.122522+0000","last_undegraded":"2024-09-17T08:34:23.122522+0000","last_fullsized":"2024-09-17T08:34:23.122522+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'11","last_scrub_stamp":"2024-09-17T08:34:01.124010+0000","last_deep_scrub":"267'9","last_deep_scrub_stamp":"2024-09-17T08:28:39.372822+0000","last_clean_scrub_stamp":"2024-09-17T08:34:01.124010+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:16.952157+0000","scrub_duration":0.0018368589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":266,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:05.245370+0000","last_change":"2024-09-17T08:34:05.245370+0000","last_active":"2024-09-17T08:34:05.245370+0000","last_peered":"2024-09-17T08:34:05.245370+0000","last_clean":"2024-09-17T08:34:05.245370+0000","last_became_active":"2024-09-17T08:33:29.925368+0000","last_became_peered":"2024-09-17T08:33:29.925368+0000","last_unstale":"2024-09-17T08:34:05.245370+0000","last_undegraded":"2024-09-17T08:34:05.245370+0000","last_fullsized":"2024-09-17T08:34:05.245370+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:05.245271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:15:43.069319+0000","last_clean_scrub_stamp":"2024-09-17T08:34:05.245271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:33.409968+0000","scrub_duration":0.0013971459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"467'29","reported_seq":679,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748311+0000","last_change":"2024-09-17T08:33:58.242429+0000","last_active":"2024-09-17T08:34:13.748311+0000","last_peered":"2024-09-17T08:34:13.748311+0000","last_clean":"2024-09-17T08:34:13.748311+0000","last_became_active":"2024-09-17T08:32:43.732127+0000","last_became_peered":"2024-09-17T08:32:43.732127+0000","last_unstale":"2024-09-17T08:34:13.748311+0000","last_undegraded":"2024-09-17T08:34:13.748311+0000","last_fullsized":"2024-09-17T08:34:13.748311+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'29","last_scrub_stamp":"2024-09-17T08:33:58.242288+0000","last_deep_scrub":"467'29","last_deep_scrub_stamp":"2024-09-17T08:33:58.242288+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.242288+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.757095+0000","scrub_duration":0.0047194519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"444'4","reported_seq":569,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:27.344832+0000","last_change":"2024-09-17T08:34:27.344832+0000","last_active":"2024-09-17T08:34:27.344832+0000","last_peered":"2024-09-17T08:34:27.344832+0000","last_clean":"2024-09-17T08:34:27.344832+0000","last_became_active":"2024-09-17T08:29:02.550936+0000","last_became_peered":"2024-09-17T08:29:02.550936+0000","last_unstale":"2024-09-17T08:34:27.344832+0000","last_undegraded":"2024-09-17T08:34:27.344832+0000","last_fullsized":"2024-09-17T08:34:27.344832+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"444'4","last_scrub_stamp":"2024-09-17T08:34:27.344696+0000","last_deep_scrub":"444'4","last_deep_scrub_stamp":"2024-09-17T08:33:04.659998+0000","last_clean_scrub_stamp":"2024-09-17T08:34:27.344696+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:57.338812+0000","scrub_duration":0.001830786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":489,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.125565+0000","last_change":"2024-09-17T08:33:48.044948+0000","last_active":"2024-09-17T08:34:33.125565+0000","last_peered":"2024-09-17T08:34:33.125565+0000","last_clean":"2024-09-17T08:34:33.125565+0000","last_became_active":"2024-09-17T08:33:43.736688+0000","last_became_peered":"2024-09-17T08:33:43.736688+0000","last_unstale":"2024-09-17T08:34:33.125565+0000","last_undegraded":"2024-09-17T08:34:33.125565+0000","last_fullsized":"2024-09-17T08:34:33.125565+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:48.044860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:17:43.435490+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.044860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.103510+0000","scrub_duration":0.0013201689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941349+0000","last_change":"2024-09-17T08:34:03.743212+0000","last_active":"2024-09-17T08:34:13.941349+0000","last_peered":"2024-09-17T08:34:13.941349+0000","last_clean":"2024-09-17T08:34:13.941349+0000","last_became_active":"2024-09-17T08:32:43.739607+0000","last_became_peered":"2024-09-17T08:32:43.739607+0000","last_unstale":"2024-09-17T08:34:13.941349+0000","last_undegraded":"2024-09-17T08:34:13.941349+0000","last_fullsized":"2024-09-17T08:34:13.941349+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.743082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.743082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.330450+0000","scrub_duration":0.001381841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"389'25","reported_seq":618,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.731731+0000","last_change":"2024-09-17T08:34:18.731815+0000","last_active":"2024-09-17T08:34:18.731731+0000","last_peered":"2024-09-17T08:34:18.731731+0000","last_clean":"2024-09-17T08:34:18.731731+0000","last_became_active":"2024-09-17T08:32:43.730052+0000","last_became_peered":"2024-09-17T08:32:43.730052+0000","last_unstale":"2024-09-17T08:34:18.731731+0000","last_undegraded":"2024-09-17T08:34:18.731731+0000","last_fullsized":"2024-09-17T08:34:18.731731+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:18.731717+0000","last_deep_scrub":"220'19","last_deep_scrub_stamp":"2024-09-17T08:25:33.032722+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.731717+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:32.200082+0000","scrub_duration":0.00231562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":503,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.125693+0000","last_change":"2024-09-17T08:33:43.737320+0000","last_active":"2024-09-17T08:34:33.125693+0000","last_peered":"2024-09-17T08:34:33.125693+0000","last_clean":"2024-09-17T08:34:33.125693+0000","last_became_active":"2024-09-17T08:33:43.737045+0000","last_became_peered":"2024-09-17T08:33:43.737045+0000","last_unstale":"2024-09-17T08:34:33.125693+0000","last_undegraded":"2024-09-17T08:34:33.125693+0000","last_fullsized":"2024-09-17T08:34:33.125693+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:22.520435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:22.520435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:50.272245+0000","scrub_duration":0.025124259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941649+0000","last_change":"2024-09-17T08:33:18.963625+0000","last_active":"2024-09-17T08:34:13.941649+0000","last_peered":"2024-09-17T08:34:13.941649+0000","last_clean":"2024-09-17T08:34:13.941649+0000","last_became_active":"2024-09-17T08:32:43.722547+0000","last_became_peered":"2024-09-17T08:32:43.722547+0000","last_unstale":"2024-09-17T08:34:13.941649+0000","last_undegraded":"2024-09-17T08:34:13.941649+0000","last_fullsized":"2024-09-17T08:34:13.941649+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:18.963447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:18.963447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017451299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":476,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126289+0000","last_change":"2024-09-17T08:34:10.294438+0000","last_active":"2024-09-17T08:34:33.126289+0000","last_peered":"2024-09-17T08:34:33.126289+0000","last_clean":"2024-09-17T08:34:33.126289+0000","last_became_active":"2024-09-17T08:33:43.628666+0000","last_became_peered":"2024-09-17T08:33:43.628666+0000","last_unstale":"2024-09-17T08:34:33.126289+0000","last_undegraded":"2024-09-17T08:34:33.126289+0000","last_fullsized":"2024-09-17T08:34:33.126289+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:10.294355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:30.595456+0000","last_clean_scrub_stamp":"2024-09-17T08:34:10.294355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.035808+0000","scrub_duration":0.0014631900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":330,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.127027+0000","last_change":"2024-09-17T08:34:26.354336+0000","last_active":"2024-09-17T08:34:33.127027+0000","last_peered":"2024-09-17T08:34:33.127027+0000","last_clean":"2024-09-17T08:34:33.127027+0000","last_became_active":"2024-09-17T08:32:32.658304+0000","last_became_peered":"2024-09-17T08:32:32.658304+0000","last_unstale":"2024-09-17T08:34:33.127027+0000","last_undegraded":"2024-09-17T08:34:33.127027+0000","last_fullsized":"2024-09-17T08:34:33.127027+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.354233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.354233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:37.372385+0000","scrub_duration":0.001699737,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"420'21","reported_seq":61,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941575+0000","last_change":"2024-09-17T08:33:48.668393+0000","last_active":"2024-09-17T08:34:13.941575+0000","last_peered":"2024-09-17T08:34:13.941575+0000","last_clean":"2024-09-17T08:34:13.941575+0000","last_became_active":"2024-09-17T08:32:43.725971+0000","last_became_peered":"2024-09-17T08:32:43.725971+0000","last_unstale":"2024-09-17T08:34:13.941575+0000","last_undegraded":"2024-09-17T08:34:13.941575+0000","last_fullsized":"2024-09-17T08:34:13.941575+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"420'21","last_scrub_stamp":"2024-09-17T08:33:48.668286+0000","last_deep_scrub":"420'21","last_deep_scrub_stamp":"2024-09-17T08:31:40.137904+0000","last_clean_scrub_stamp":"2024-09-17T08:33:48.668286+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:52.935284+0000","scrub_duration":0.001802606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":143,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621952+0000","last_change":"2024-09-17T08:33:29.921338+0000","last_active":"2024-09-17T08:33:56.621952+0000","last_peered":"2024-09-17T08:33:56.621952+0000","last_clean":"2024-09-17T08:33:56.621952+0000","last_became_active":"2024-09-17T08:33:29.919398+0000","last_became_peered":"2024-09-17T08:33:29.919398+0000","last_unstale":"2024-09-17T08:33:56.621952+0000","last_undegraded":"2024-09-17T08:33:56.621952+0000","last_fullsized":"2024-09-17T08:33:56.621952+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:58.139916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:58.139916+0000","last_clean_scrub_stamp":"2024-09-17T08:32:58.139916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:20.150445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":574,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:08.043793+0000","last_change":"2024-09-17T08:34:07.080340+0000","last_active":"2024-09-17T08:34:08.043793+0000","last_peered":"2024-09-17T08:34:08.043793+0000","last_clean":"2024-09-17T08:34:08.043793+0000","last_became_active":"2024-09-17T08:32:32.651944+0000","last_became_peered":"2024-09-17T08:32:32.651944+0000","last_unstale":"2024-09-17T08:34:08.043793+0000","last_undegraded":"2024-09-17T08:34:08.043793+0000","last_fullsized":"2024-09-17T08:34:08.043793+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:07.080233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:07.080233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:21.237210+0000","scrub_duration":0.00163532,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"389'30","reported_seq":65,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941975+0000","last_change":"2024-09-17T08:33:50.746646+0000","last_active":"2024-09-17T08:34:13.941975+0000","last_peered":"2024-09-17T08:34:13.941975+0000","last_clean":"2024-09-17T08:34:13.941975+0000","last_became_active":"2024-09-17T08:32:43.727080+0000","last_became_peered":"2024-09-17T08:32:43.727080+0000","last_unstale":"2024-09-17T08:34:13.941975+0000","last_undegraded":"2024-09-17T08:34:13.941975+0000","last_fullsized":"2024-09-17T08:34:13.941975+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'30","last_scrub_stamp":"2024-09-17T08:33:50.746545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:50.746545+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:13.481858+0000","scrub_duration":0.001928164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":200,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:26.126564+0000","last_change":"2024-09-17T08:34:26.126647+0000","last_active":"2024-09-17T08:34:26.126564+0000","last_peered":"2024-09-17T08:34:26.126564+0000","last_clean":"2024-09-17T08:34:26.126564+0000","last_became_active":"2024-09-17T08:33:29.923325+0000","last_became_peered":"2024-09-17T08:33:29.923325+0000","last_unstale":"2024-09-17T08:34:26.126564+0000","last_undegraded":"2024-09-17T08:34:26.126564+0000","last_fullsized":"2024-09-17T08:34:26.126564+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:26.126549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:20:31.890809+0000","last_clean_scrub_stamp":"2024-09-17T08:34:26.126549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:52.507737+0000","scrub_duration":0.0015897789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"455'13","reported_seq":167,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:23.122543+0000","last_change":"2024-09-17T08:33:58.156020+0000","last_active":"2024-09-17T08:34:23.122543+0000","last_peered":"2024-09-17T08:34:23.122543+0000","last_clean":"2024-09-17T08:34:23.122543+0000","last_became_active":"2024-09-17T08:33:29.921687+0000","last_became_peered":"2024-09-17T08:33:29.921687+0000","last_unstale":"2024-09-17T08:34:23.122543+0000","last_undegraded":"2024-09-17T08:34:23.122543+0000","last_fullsized":"2024-09-17T08:34:23.122543+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"455'13","last_scrub_stamp":"2024-09-17T08:33:58.155909+0000","last_deep_scrub":"201'12","last_deep_scrub_stamp":"2024-09-17T08:29:34.411246+0000","last_clean_scrub_stamp":"2024-09-17T08:33:58.155909+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:17.174516+0000","scrub_duration":0.001838604,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":556,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:11.972638+0000","last_change":"2024-09-17T08:33:50.561826+0000","last_active":"2024-09-17T08:34:11.972638+0000","last_peered":"2024-09-17T08:34:11.972638+0000","last_clean":"2024-09-17T08:34:11.972638+0000","last_became_active":"2024-09-17T08:32:43.719215+0000","last_became_peered":"2024-09-17T08:32:43.719215+0000","last_unstale":"2024-09-17T08:34:11.972638+0000","last_undegraded":"2024-09-17T08:34:11.972638+0000","last_fullsized":"2024-09-17T08:34:11.972638+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:50.561717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:50.561717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:09.215291+0000","scrub_duration":0.0019649379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"389'25","reported_seq":440,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:22.126546+0000","last_change":"2024-09-17T08:34:22.126546+0000","last_active":"2024-09-17T08:34:22.126546+0000","last_peered":"2024-09-17T08:34:22.126546+0000","last_clean":"2024-09-17T08:34:22.126546+0000","last_became_active":"2024-09-17T08:33:29.926041+0000","last_became_peered":"2024-09-17T08:33:29.926041+0000","last_unstale":"2024-09-17T08:34:22.126546+0000","last_undegraded":"2024-09-17T08:34:22.126546+0000","last_fullsized":"2024-09-17T08:34:22.126546+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'25","last_scrub_stamp":"2024-09-17T08:34:22.126408+0000","last_deep_scrub":"389'25","last_deep_scrub_stamp":"2024-09-17T08:33:05.055099+0000","last_clean_scrub_stamp":"2024-09-17T08:34:22.126408+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:47.692473+0000","scrub_duration":0.0022084919999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":562,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:03.545529+0000","last_change":"2024-09-17T08:34:03.545529+0000","last_active":"2024-09-17T08:34:03.545529+0000","last_peered":"2024-09-17T08:34:03.545529+0000","last_clean":"2024-09-17T08:34:03.545529+0000","last_became_active":"2024-09-17T08:32:43.730244+0000","last_became_peered":"2024-09-17T08:32:43.730244+0000","last_unstale":"2024-09-17T08:34:03.545529+0000","last_undegraded":"2024-09-17T08:34:03.545529+0000","last_fullsized":"2024-09-17T08:34:03.545529+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:03.545422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:22:26.033887+0000","last_clean_scrub_stamp":"2024-09-17T08:34:03.545422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:04.748468+0000","scrub_duration":0.001371415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"438'14","reported_seq":80,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:32.809819+0000","last_change":"2024-09-17T08:33:28.955404+0000","last_active":"2024-09-17T08:34:32.809819+0000","last_peered":"2024-09-17T08:34:32.809819+0000","last_clean":"2024-09-17T08:34:32.809819+0000","last_became_active":"2024-09-17T08:32:43.729362+0000","last_became_peered":"2024-09-17T08:32:43.729362+0000","last_unstale":"2024-09-17T08:34:32.809819+0000","last_undegraded":"2024-09-17T08:34:32.809819+0000","last_fullsized":"2024-09-17T08:34:32.809819+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"438'14","last_scrub_stamp":"2024-09-17T08:33:28.955300+0000","last_deep_scrub":"88'9","last_deep_scrub_stamp":"2024-09-17T08:11:49.005295+0000","last_clean_scrub_stamp":"2024-09-17T08:33:28.955300+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:39.293174+0000","scrub_duration":0.0021009459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":533,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.749126+0000","last_change":"2024-09-17T08:33:27.443109+0000","last_active":"2024-09-17T08:34:13.749126+0000","last_peered":"2024-09-17T08:34:13.749126+0000","last_clean":"2024-09-17T08:34:13.749126+0000","last_became_active":"2024-09-17T08:20:23.482006+0000","last_became_peered":"2024-09-17T08:20:23.482006+0000","last_unstale":"2024-09-17T08:34:13.749126+0000","last_undegraded":"2024-09-17T08:34:13.749126+0000","last_fullsized":"2024-09-17T08:34:13.749126+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:27.443000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:28:38.263303+0000","last_clean_scrub_stamp":"2024-09-17T08:33:27.443000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:51.341041+0000","scrub_duration":0.001620898,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"389'21","reported_seq":469,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.349850+0000","last_change":"2024-09-17T08:33:07.573106+0000","last_active":"2024-09-17T08:34:14.349850+0000","last_peered":"2024-09-17T08:34:14.349850+0000","last_clean":"2024-09-17T08:34:14.349850+0000","last_became_active":"2024-09-17T08:32:32.671197+0000","last_became_peered":"2024-09-17T08:32:32.671197+0000","last_unstale":"2024-09-17T08:34:14.349850+0000","last_undegraded":"2024-09-17T08:34:14.349850+0000","last_fullsized":"2024-09-17T08:34:14.349850+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'21","last_scrub_stamp":"2024-09-17T08:33:07.572956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:07.572956+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026949299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":325,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.128506+0000","last_change":"2024-09-17T08:33:40.661731+0000","last_active":"2024-09-17T08:34:33.128506+0000","last_peered":"2024-09-17T08:34:33.128506+0000","last_clean":"2024-09-17T08:34:33.128506+0000","last_became_active":"2024-09-17T08:32:43.715260+0000","last_became_peered":"2024-09-17T08:32:43.715260+0000","last_unstale":"2024-09-17T08:34:33.128506+0000","last_undegraded":"2024-09-17T08:34:33.128506+0000","last_fullsized":"2024-09-17T08:34:33.128506+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:40.661630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:34.199418+0000","last_clean_scrub_stamp":"2024-09-17T08:33:40.661630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.017334+0000","scrub_duration":0.001546813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":64,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941256+0000","last_change":"2024-09-17T08:34:05.804038+0000","last_active":"2024-09-17T08:34:13.941256+0000","last_peered":"2024-09-17T08:34:13.941256+0000","last_clean":"2024-09-17T08:34:13.941256+0000","last_became_active":"2024-09-17T08:32:43.735363+0000","last_became_peered":"2024-09-17T08:32:43.735363+0000","last_unstale":"2024-09-17T08:34:13.941256+0000","last_undegraded":"2024-09-17T08:34:13.941256+0000","last_fullsized":"2024-09-17T08:34:13.941256+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:05.803956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:32:53.069071+0000","last_clean_scrub_stamp":"2024-09-17T08:34:05.803956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:08.364498+0000","scrub_duration":0.0011965089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":556,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:20.793270+0000","last_change":"2024-09-17T08:34:20.793335+0000","last_active":"2024-09-17T08:34:20.793270+0000","last_peered":"2024-09-17T08:34:20.793270+0000","last_clean":"2024-09-17T08:34:20.793270+0000","last_became_active":"2024-09-17T08:15:29.372723+0000","last_became_peered":"2024-09-17T08:15:29.372723+0000","last_unstale":"2024-09-17T08:34:20.793270+0000","last_undegraded":"2024-09-17T08:34:20.793270+0000","last_fullsized":"2024-09-17T08:34:20.793270+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:20.793258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:34:20.793258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:48.060833+0000","scrub_duration":0.00185181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"389'27","reported_seq":634,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748598+0000","last_change":"2024-09-17T08:34:04.330881+0000","last_active":"2024-09-17T08:34:13.748598+0000","last_peered":"2024-09-17T08:34:13.748598+0000","last_clean":"2024-09-17T08:34:13.748598+0000","last_became_active":"2024-09-17T08:32:32.651582+0000","last_became_peered":"2024-09-17T08:32:32.651582+0000","last_unstale":"2024-09-17T08:34:13.748598+0000","last_undegraded":"2024-09-17T08:34:13.748598+0000","last_fullsized":"2024-09-17T08:34:13.748598+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:34:04.330744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.330744+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.852415+0000","scrub_duration":0.002077307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":423,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:31.211527+0000","last_change":"2024-09-17T08:34:31.211527+0000","last_active":"2024-09-17T08:34:31.211527+0000","last_peered":"2024-09-17T08:34:31.211527+0000","last_clean":"2024-09-17T08:34:31.211527+0000","last_became_active":"2024-09-17T08:15:29.367807+0000","last_became_peered":"2024-09-17T08:15:29.367807+0000","last_unstale":"2024-09-17T08:34:31.211527+0000","last_undegraded":"2024-09-17T08:34:31.211527+0000","last_fullsized":"2024-09-17T08:34:31.211527+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:31.211431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:34:31.211431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:38.017157+0000","scrub_duration":0.0017236840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":431,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:14.349874+0000","last_change":"2024-09-17T08:33:33.427392+0000","last_active":"2024-09-17T08:34:14.349874+0000","last_peered":"2024-09-17T08:34:14.349874+0000","last_clean":"2024-09-17T08:34:14.349874+0000","last_became_active":"2024-09-17T08:32:43.725035+0000","last_became_peered":"2024-09-17T08:32:43.725035+0000","last_unstale":"2024-09-17T08:34:14.349874+0000","last_undegraded":"2024-09-17T08:34:14.349874+0000","last_fullsized":"2024-09-17T08:34:14.349874+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:33.427308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:33:33.427308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:54.408162+0000","scrub_duration":0.001853267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941599+0000","last_change":"2024-09-17T08:33:25.988653+0000","last_active":"2024-09-17T08:34:13.941599+0000","last_peered":"2024-09-17T08:34:13.941599+0000","last_clean":"2024-09-17T08:34:13.941599+0000","last_became_active":"2024-09-17T08:32:43.734497+0000","last_became_peered":"2024-09-17T08:32:43.734497+0000","last_unstale":"2024-09-17T08:34:13.941599+0000","last_undegraded":"2024-09-17T08:34:13.941599+0000","last_fullsized":"2024-09-17T08:34:13.941599+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:25.988536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:25.988536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021364700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"468'14","reported_seq":280,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:30.284798+0000","last_change":"2024-09-17T08:34:30.284798+0000","last_active":"2024-09-17T08:34:30.284798+0000","last_peered":"2024-09-17T08:34:30.284798+0000","last_clean":"2024-09-17T08:34:30.284798+0000","last_became_active":"2024-09-17T08:32:32.656521+0000","last_became_peered":"2024-09-17T08:32:32.656521+0000","last_unstale":"2024-09-17T08:34:30.284798+0000","last_undegraded":"2024-09-17T08:34:30.284798+0000","last_fullsized":"2024-09-17T08:34:30.284798+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'14","last_scrub_stamp":"2024-09-17T08:34:30.284687+0000","last_deep_scrub":"468'14","last_deep_scrub_stamp":"2024-09-17T08:34:30.284687+0000","last_clean_scrub_stamp":"2024-09-17T08:34:30.284687+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:45.057295+0000","scrub_duration":0.0032137440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":233,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.943339+0000","last_change":"2024-09-17T08:33:29.925087+0000","last_active":"2024-09-17T08:34:13.943339+0000","last_peered":"2024-09-17T08:34:13.943339+0000","last_clean":"2024-09-17T08:34:13.943339+0000","last_became_active":"2024-09-17T08:33:29.924785+0000","last_became_peered":"2024-09-17T08:33:29.924785+0000","last_unstale":"2024-09-17T08:34:13.943339+0000","last_undegraded":"2024-09-17T08:34:13.943339+0000","last_fullsized":"2024-09-17T08:34:13.943339+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:03.997048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:03.997048+0000","last_clean_scrub_stamp":"2024-09-17T08:33:03.997048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.058333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":311,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:34.306676+0000","last_change":"2024-09-17T08:34:34.306676+0000","last_active":"2024-09-17T08:34:34.306676+0000","last_peered":"2024-09-17T08:34:34.306676+0000","last_clean":"2024-09-17T08:34:34.306676+0000","last_became_active":"2024-09-17T08:32:32.664391+0000","last_became_peered":"2024-09-17T08:32:32.664391+0000","last_unstale":"2024-09-17T08:34:34.306676+0000","last_undegraded":"2024-09-17T08:34:34.306676+0000","last_fullsized":"2024-09-17T08:34:34.306676+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:34.306553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:29.113668+0000","last_clean_scrub_stamp":"2024-09-17T08:34:34.306553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:49.216675+0000","scrub_duration":0.001485138,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":500,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:27.142260+0000","last_change":"2024-09-17T08:34:27.142926+0000","last_active":"2024-09-17T08:34:27.142260+0000","last_peered":"2024-09-17T08:34:27.142260+0000","last_clean":"2024-09-17T08:34:27.142260+0000","last_became_active":"2024-09-17T08:33:35.946094+0000","last_became_peered":"2024-09-17T08:33:35.946094+0000","last_unstale":"2024-09-17T08:34:27.142260+0000","last_undegraded":"2024-09-17T08:34:27.142260+0000","last_fullsized":"2024-09-17T08:34:27.142260+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:27.142244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:25:26.873378+0000","last_clean_scrub_stamp":"2024-09-17T08:34:27.142244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:54.682225+0000","scrub_duration":0.001624218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"389'27","reported_seq":76,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941754+0000","last_change":"2024-09-17T08:33:56.761757+0000","last_active":"2024-09-17T08:34:13.941754+0000","last_peered":"2024-09-17T08:34:13.941754+0000","last_clean":"2024-09-17T08:34:13.941754+0000","last_became_active":"2024-09-17T08:32:43.740245+0000","last_became_peered":"2024-09-17T08:32:43.740245+0000","last_unstale":"2024-09-17T08:34:13.941754+0000","last_undegraded":"2024-09-17T08:34:13.941754+0000","last_fullsized":"2024-09-17T08:34:13.941754+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:33:56.761657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.761657+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:07.222324+0000","scrub_duration":0.0024020830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'2","reported_seq":335,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.316519+0000","last_change":"2024-09-17T08:34:33.316519+0000","last_active":"2024-09-17T08:34:33.316519+0000","last_peered":"2024-09-17T08:34:33.316519+0000","last_clean":"2024-09-17T08:34:33.316519+0000","last_became_active":"2024-09-17T08:32:32.669069+0000","last_became_peered":"2024-09-17T08:32:32.669069+0000","last_unstale":"2024-09-17T08:34:33.316519+0000","last_undegraded":"2024-09-17T08:34:33.316519+0000","last_fullsized":"2024-09-17T08:34:33.316519+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-17T08:34:33.316383+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-17T08:34:33.316383+0000","last_clean_scrub_stamp":"2024-09-17T08:34:33.316383+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:41.418917+0000","scrub_duration":0.0021395199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":565,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:09.065411+0000","last_change":"2024-09-17T08:34:09.065411+0000","last_active":"2024-09-17T08:34:09.065411+0000","last_peered":"2024-09-17T08:34:09.065411+0000","last_clean":"2024-09-17T08:34:09.065411+0000","last_became_active":"2024-09-17T08:32:32.656410+0000","last_became_peered":"2024-09-17T08:32:32.656410+0000","last_unstale":"2024-09-17T08:34:09.065411+0000","last_undegraded":"2024-09-17T08:34:09.065411+0000","last_fullsized":"2024-09-17T08:34:09.065411+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:09.065280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:25.622796+0000","last_clean_scrub_stamp":"2024-09-17T08:34:09.065280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:11.510012+0000","scrub_duration":0.001602795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941184+0000","last_change":"2024-09-17T08:32:56.136553+0000","last_active":"2024-09-17T08:34:13.941184+0000","last_peered":"2024-09-17T08:34:13.941184+0000","last_clean":"2024-09-17T08:34:13.941184+0000","last_became_active":"2024-09-17T08:32:43.733334+0000","last_became_peered":"2024-09-17T08:32:43.733334+0000","last_unstale":"2024-09-17T08:34:13.941184+0000","last_undegraded":"2024-09-17T08:34:13.941184+0000","last_fullsized":"2024-09-17T08:34:13.941184+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:32:56.136470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:27:29.641570+0000","last_clean_scrub_stamp":"2024-09-17T08:32:56.136470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001421093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"468'4","reported_seq":568,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:18.117199+0000","last_change":"2024-09-17T08:34:18.117320+0000","last_active":"2024-09-17T08:34:18.117199+0000","last_peered":"2024-09-17T08:34:18.117199+0000","last_clean":"2024-09-17T08:34:18.117199+0000","last_became_active":"2024-09-17T08:33:35.946672+0000","last_became_peered":"2024-09-17T08:33:35.946672+0000","last_unstale":"2024-09-17T08:34:18.117199+0000","last_undegraded":"2024-09-17T08:34:18.117199+0000","last_fullsized":"2024-09-17T08:34:18.117199+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"468'4","last_scrub_stamp":"2024-09-17T08:34:18.117180+0000","last_deep_scrub":"468'4","last_deep_scrub_stamp":"2024-09-17T08:34:18.117180+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.117180+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:26.877100+0000","scrub_duration":0.0037266669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":147,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:33:56.621633+0000","last_change":"2024-09-17T08:33:35.007760+0000","last_active":"2024-09-17T08:33:56.621633+0000","last_peered":"2024-09-17T08:33:56.621633+0000","last_clean":"2024-09-17T08:33:56.621633+0000","last_became_active":"2024-09-17T08:33:30.128295+0000","last_became_peered":"2024-09-17T08:33:30.128295+0000","last_unstale":"2024-09-17T08:33:56.621633+0000","last_undegraded":"2024-09-17T08:33:56.621633+0000","last_fullsized":"2024-09-17T08:33:56.621633+0000","mapping_epoch":453,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":454,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:35.007660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:33:35.007660+0000","last_clean_scrub_stamp":"2024-09-17T08:33:35.007660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:57.958982+0000","scrub_duration":0.0014704239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"389'30","reported_seq":631,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:12.632305+0000","last_change":"2024-09-17T08:34:12.632305+0000","last_active":"2024-09-17T08:34:12.632305+0000","last_peered":"2024-09-17T08:34:12.632305+0000","last_clean":"2024-09-17T08:34:12.632305+0000","last_became_active":"2024-09-17T08:32:43.723301+0000","last_became_peered":"2024-09-17T08:32:43.723301+0000","last_unstale":"2024-09-17T08:34:12.632305+0000","last_undegraded":"2024-09-17T08:34:12.632305+0000","last_fullsized":"2024-09-17T08:34:12.632305+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'30","last_scrub_stamp":"2024-09-17T08:34:12.632160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:12.632160+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.397940+0000","scrub_duration":0.002231415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"435'12","reported_seq":315,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.346131+0000","last_change":"2024-09-17T08:34:28.417048+0000","last_active":"2024-09-17T08:34:35.346131+0000","last_peered":"2024-09-17T08:34:35.346131+0000","last_clean":"2024-09-17T08:34:35.346131+0000","last_became_active":"2024-09-17T08:32:32.667694+0000","last_became_peered":"2024-09-17T08:32:32.667694+0000","last_unstale":"2024-09-17T08:34:35.346131+0000","last_undegraded":"2024-09-17T08:34:35.346131+0000","last_fullsized":"2024-09-17T08:34:35.346131+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"435'12","last_scrub_stamp":"2024-09-17T08:34:28.416906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:57.490227+0000","last_clean_scrub_stamp":"2024-09-17T08:34:28.416906+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:30.251662+0000","scrub_duration":0.001784517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":423,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752194+0000","last_change":"2024-09-17T08:33:44.476368+0000","last_active":"2024-09-17T08:34:13.752194+0000","last_peered":"2024-09-17T08:34:13.752194+0000","last_clean":"2024-09-17T08:34:13.752194+0000","last_became_active":"2024-09-17T08:15:29.382753+0000","last_became_peered":"2024-09-17T08:15:29.382753+0000","last_unstale":"2024-09-17T08:34:13.752194+0000","last_undegraded":"2024-09-17T08:34:13.752194+0000","last_fullsized":"2024-09-17T08:34:13.752194+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:44.476257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:44.476257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:53.095873+0000","scrub_duration":0.0019968730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941616+0000","last_change":"2024-09-17T08:33:42.744862+0000","last_active":"2024-09-17T08:34:13.941616+0000","last_peered":"2024-09-17T08:34:13.941616+0000","last_clean":"2024-09-17T08:34:13.941616+0000","last_became_active":"2024-09-17T08:32:43.734119+0000","last_became_peered":"2024-09-17T08:32:43.734119+0000","last_unstale":"2024-09-17T08:34:13.941616+0000","last_undegraded":"2024-09-17T08:34:13.941616+0000","last_fullsized":"2024-09-17T08:34:13.941616+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:42.744766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:31.941534+0000","last_clean_scrub_stamp":"2024-09-17T08:33:42.744766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:12.195324+0000","scrub_duration":0.0028936869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"389'27","reported_seq":333,"reported_epoch":470,"state":"active+clean","last_fresh":"2024-09-17T08:34:35.346213+0000","last_change":"2024-09-17T08:34:08.431596+0000","last_active":"2024-09-17T08:34:35.346213+0000","last_peered":"2024-09-17T08:34:35.346213+0000","last_clean":"2024-09-17T08:34:35.346213+0000","last_became_active":"2024-09-17T08:32:32.668592+0000","last_became_peered":"2024-09-17T08:32:32.668592+0000","last_unstale":"2024-09-17T08:34:35.346213+0000","last_undegraded":"2024-09-17T08:34:35.346213+0000","last_fullsized":"2024-09-17T08:34:35.346213+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'27","last_scrub_stamp":"2024-09-17T08:34:08.431471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:08.431471+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:18.894819+0000","scrub_duration":0.0022844440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":559,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-17T08:34:07.784697+0000","last_change":"2024-09-17T08:33:46.606984+0000","last_active":"2024-09-17T08:34:07.784697+0000","last_peered":"2024-09-17T08:34:07.784697+0000","last_clean":"2024-09-17T08:34:07.784697+0000","last_became_active":"2024-09-17T08:20:23.488231+0000","last_became_peered":"2024-09-17T08:20:23.488231+0000","last_unstale":"2024-09-17T08:34:07.784697+0000","last_undegraded":"2024-09-17T08:34:07.784697+0000","last_fullsized":"2024-09-17T08:34:07.784697+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:46.606882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:14:54.789314+0000","last_clean_scrub_stamp":"2024-09-17T08:33:46.606882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:10.424932+0000","scrub_duration":0.0015957689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":535,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748995+0000","last_change":"2024-09-17T08:33:56.230098+0000","last_active":"2024-09-17T08:34:13.748995+0000","last_peered":"2024-09-17T08:34:13.748995+0000","last_clean":"2024-09-17T08:34:13.748995+0000","last_became_active":"2024-09-17T08:32:32.653871+0000","last_became_peered":"2024-09-17T08:32:32.653871+0000","last_unstale":"2024-09-17T08:34:13.748995+0000","last_undegraded":"2024-09-17T08:34:13.748995+0000","last_fullsized":"2024-09-17T08:34:13.748995+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:56.230019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:11:36.624764+0000","last_clean_scrub_stamp":"2024-09-17T08:33:56.230019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:22.952253+0000","scrub_duration":0.001255036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":201,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126723+0000","last_change":"2024-09-17T08:34:18.319335+0000","last_active":"2024-09-17T08:34:33.126723+0000","last_peered":"2024-09-17T08:34:33.126723+0000","last_clean":"2024-09-17T08:34:33.126723+0000","last_became_active":"2024-09-17T08:32:32.666678+0000","last_became_peered":"2024-09-17T08:32:32.666678+0000","last_unstale":"2024-09-17T08:34:33.126723+0000","last_undegraded":"2024-09-17T08:34:33.126723+0000","last_fullsized":"2024-09-17T08:34:33.126723+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:34:18.319210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:21:24.999967+0000","last_clean_scrub_stamp":"2024-09-17T08:34:18.319210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:28.428338+0000","scrub_duration":0.0014244119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"389'46","reported_seq":650,"reported_epoch":469,"state":"active+clean","last_fresh":"2024-09-17T08:34:33.126365+0000","last_change":"2024-09-17T08:34:32.130748+0000","last_active":"2024-09-17T08:34:33.126365+0000","last_peered":"2024-09-17T08:34:33.126365+0000","last_clean":"2024-09-17T08:34:33.126365+0000","last_became_active":"2024-09-17T08:33:43.626357+0000","last_became_peered":"2024-09-17T08:33:43.626357+0000","last_unstale":"2024-09-17T08:34:33.126365+0000","last_undegraded":"2024-09-17T08:34:33.126365+0000","last_fullsized":"2024-09-17T08:34:33.126365+0000","mapping_epoch":460,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'46","last_scrub_stamp":"2024-09-17T08:34:32.130651+0000","last_deep_scrub":"220'34","last_deep_scrub_stamp":"2024-09-17T08:24:23.759435+0000","last_clean_scrub_stamp":"2024-09-17T08:34:32.130651+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:47.730913+0000","scrub_duration":0.0022343549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":412,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.752285+0000","last_change":"2024-09-17T08:33:09.622678+0000","last_active":"2024-09-17T08:34:13.752285+0000","last_peered":"2024-09-17T08:34:13.752285+0000","last_clean":"2024-09-17T08:34:13.752285+0000","last_became_active":"2024-09-17T08:15:29.386082+0000","last_became_peered":"2024-09-17T08:15:29.386082+0000","last_unstale":"2024-09-17T08:34:13.752285+0000","last_undegraded":"2024-09-17T08:34:13.752285+0000","last_fullsized":"2024-09-17T08:34:13.752285+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:09.622583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:30:20.462452+0000","last_clean_scrub_stamp":"2024-09-17T08:33:09.622583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:36.139373+0000","scrub_duration":0.0016635459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941689+0000","last_change":"2024-09-17T08:33:22.004704+0000","last_active":"2024-09-17T08:34:13.941689+0000","last_peered":"2024-09-17T08:34:13.941689+0000","last_clean":"2024-09-17T08:34:13.941689+0000","last_became_active":"2024-09-17T08:32:43.727828+0000","last_became_peered":"2024-09-17T08:32:43.727828+0000","last_unstale":"2024-09-17T08:34:13.941689+0000","last_undegraded":"2024-09-17T08:34:13.941689+0000","last_fullsized":"2024-09-17T08:34:13.941689+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:22.004622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:59.494268+0000","last_clean_scrub_stamp":"2024-09-17T08:33:22.004622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001687568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":53,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941311+0000","last_change":"2024-09-17T08:33:49.720713+0000","last_active":"2024-09-17T08:34:13.941311+0000","last_peered":"2024-09-17T08:34:13.941311+0000","last_clean":"2024-09-17T08:34:13.941311+0000","last_became_active":"2024-09-17T08:32:43.735242+0000","last_became_peered":"2024-09-17T08:32:43.735242+0000","last_unstale":"2024-09-17T08:34:13.941311+0000","last_undegraded":"2024-09-17T08:34:13.941311+0000","last_fullsized":"2024-09-17T08:34:13.941311+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:49.720631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:53.481832+0000","last_clean_scrub_stamp":"2024-09-17T08:33:49.720631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:13.226529+0000","scrub_duration":0.0045051600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"389'24","reported_seq":65,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.941422+0000","last_change":"2024-09-17T08:34:04.759117+0000","last_active":"2024-09-17T08:34:13.941422+0000","last_peered":"2024-09-17T08:34:13.941422+0000","last_clean":"2024-09-17T08:34:13.941422+0000","last_became_active":"2024-09-17T08:32:43.724393+0000","last_became_peered":"2024-09-17T08:32:43.724393+0000","last_unstale":"2024-09-17T08:34:13.941422+0000","last_undegraded":"2024-09-17T08:34:13.941422+0000","last_fullsized":"2024-09-17T08:34:13.941422+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"389'24","last_scrub_stamp":"2024-09-17T08:34:04.759020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:09:55.487348+0000","last_clean_scrub_stamp":"2024-09-17T08:34:04.759020+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:24.723294+0000","scrub_duration":0.0023702850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":563,"reported_epoch":468,"state":"active+clean","last_fresh":"2024-09-17T08:34:13.748669+0000","last_change":"2024-09-17T08:33:24.376389+0000","last_active":"2024-09-17T08:34:13.748669+0000","last_peered":"2024-09-17T08:34:13.748669+0000","last_clean":"2024-09-17T08:34:13.748669+0000","last_became_active":"2024-09-17T08:32:43.720912+0000","last_became_peered":"2024-09-17T08:32:43.720912+0000","last_unstale":"2024-09-17T08:34:13.748669+0000","last_undegraded":"2024-09-17T08:34:13.748669+0000","last_fullsized":"2024-09-17T08:34:13.748669+0000","mapping_epoch":437,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":438,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T08:33:24.376228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T08:31:30.767734+0000","last_clean_scrub_stamp":"2024-09-17T08:33:24.376228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016577860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1538,"num_read_kb":1295,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":8032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1084,"ondisk_log_size":1084,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1134,"num_read_kb":1129,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1430,"ondisk_log_size":1430,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":59,"seq":253403070766,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":776348,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":775166,"kb_avail":92968804,"statfs":{"total":95995035648,"available":95200055296,"internally_reserved":0,"allocated":1200128,"data_stored":849239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":793770443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.71499999999999997,"15min":1.0649999999999999},"min":{"1min":0.35299999999999998,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.73999999999999999,"5min":3.6659999999999999,"15min":17.556999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.81100000000000005,"15min":1.071},"min":{"1min":0.32700000000000001,"5min":0.24199999999999999,"15min":0.22},"max":{"1min":0.83599999999999997,"5min":3.7109999999999999,"15min":18.245999999999999},"last":0.621}]},{"osd":1,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":1.1699999999999999,"15min":1.298},"min":{"1min":0.40000000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.80900000000000005,"5min":21.587,"15min":44.917999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.155,"15min":1.341},"min":{"1min":0.41599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78400000000000003,"5min":21.265000000000001,"15min":44.996000000000002},"last":0.76100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":4.75,"15min":2.2200000000000002},"min":{"1min":0.38500000000000001,"5min":0.27200000000000002,"15min":0.25700000000000001},"max":{"1min":0.75600000000000001,"5min":296.178,"15min":296.178},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":4.5910000000000002,"15min":2.1379999999999999},"min":{"1min":0.45100000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.748,"5min":296.27699999999999,"15min":296.27699999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":3.149,"15min":2.3090000000000002},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80500000000000005,"5min":226.595,"15min":226.595},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":3.121,"15min":2.3580000000000001},"min":{"1min":0.42499999999999999,"5min":0.309,"15min":0.27300000000000002},"max":{"1min":0.79700000000000004,"5min":226.65000000000001,"15min":226.65000000000001},"last":0.747}]},{"osd":4,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.50600000000000001,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.23999999999999999,"15min":0.224},"max":{"1min":0.68700000000000006,"5min":3.5129999999999999,"15min":4.391},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.56100000000000005,"15min":0.65200000000000002},"min":{"1min":0.36099999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.61199999999999999,"5min":3.4430000000000001,"15min":5.9669999999999996},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.78100000000000003},"min":{"1min":0.36599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68799999999999994,"5min":2.3700000000000001,"15min":34.436},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57999999999999996,"15min":0.83899999999999997},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.26200000000000001},"max":{"1min":0.70099999999999996,"5min":3.4710000000000001,"15min":34.965000000000003},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68500000000000005}]}]},{"osd":6,"up_from":434,"seq":1864015806491,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84384,"kb_used_data":852,"kb_used_omap":4,"kb_used_meta":83515,"kb_avail":93660768,"statfs":{"total":95995035648,"available":95908626432,"internally_reserved":0,"allocated":872448,"data_stored":526946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4441,"internal_metadata":85520039},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.5}]},{"osd":1,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.624},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Sep 17 08:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.63600000000000001}]}]},{"osd":5,"up_from":41,"seq":176093659448,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":722460,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":721022,"kb_avail":93022692,"statfs":{"total":95995035648,"available":95255236608,"internally_reserved":0,"allocated":1462272,"data_stored":1111228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":738326982},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":2.4430000000000001,"15min":2.5750000000000002},"min":{"1min":0.30099999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.67600000000000005,"5min":162.39500000000001,"15min":162.39500000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":2.4580000000000002,"15min":2.6160000000000001},"min":{"1min":0.35599999999999998,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.64700000000000002,"5min":162.601,"15min":162.601},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.81299999999999994,"15min":1.714},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.64800000000000002,"5min":3.8100000000000001,"15min":66.981999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.92600000000000005,"15min":1.8029999999999999},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.23599999999999999},"max":{"1min":0.68500000000000005,"5min":4.8799999999999999,"15min":66.316000000000003},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Sep 17 08:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.72999999999999998,"15min":1.7969999999999999},"min":{"1min":0.377,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":4.0279999999999996,"15min":131.727},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84399999999999997,"15min":1.962},"min":{"1min":0.36699999999999999,"5min":0.27800000000000002,"15min":0.27200000000000002},"max":{"1min":0.70299999999999996,"5min":4.0090000000000003,"15min":131.50399999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.71899999999999997,"15min":1.6990000000000001},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.25800000000000001},"max":{"1min":0.69999999999999996,"5min":4.4290000000000003,"15min":67.676000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.90700000000000003,"15min":1.7509999999999999},"min":{"1min":0.40200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66900000000000004,"5min":4.1470000000000002,"15min":67.691999999999993},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.58499999999999996,"15min":0.70099999999999996},"min":{"1min":0.25900000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.64700000000000002,"5min":3.6989999999999998,"15min":6.3600000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51600000000000001,"15min":0.65000000000000002},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73599999999999999,"5min":2.085,"15min":6.0899999999999999},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40999999999999998}]},{"osd":7,"last update":"Tue Sep 17 08:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.93700000000000006,"15min":0.88100000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":25.085000000000001,"15min":25.085000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":23.561,"15min":23.561},"last":0.53800000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953789,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":609508,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":608638,"kb_avail":93135644,"statfs":{"total":95995035648,"available":95370899456,"internally_reserved":0,"allocated":880640,"data_stored":521489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":623245766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.78900000000000003,"15min":0.91400000000000003},"min":{"1min":0.38300000000000001,"5min":0.221,"15min":0.21099999999999999},"max":{"1min":0.88,"5min":4.835,"15min":20.271000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.95099999999999996},"min":{"1min":0.30399999999999999,"5min":0.251,"15min":0.224},"max":{"1min":0.83699999999999997,"5min":4.9240000000000004,"15min":19.798999999999999},"last":0.502}]},{"osd":1,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":2.6269999999999998,"15min":1.671},"min":{"1min":0.36799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85099999999999998,"5min":84.185000000000002,"15min":84.185000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":2.536,"15min":1.581},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.246},"max":{"1min":0.82499999999999996,"5min":84.153999999999996,"15min":84.153999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Sep 17 08:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":2.0009999999999999,"15min":1.2130000000000001},"min":{"1min":0.33100000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77900000000000003,"5min":105.536,"15min":105.536},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":2.0390000000000001,"15min":1.1970000000000001},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":0.88500000000000001,"5min":105.623,"15min":105.623},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Sep 17 08:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.76900000000000002,"15min":1.107},"min":{"1min":0.372,"5min":0.28100000000000003,"15min":0.26300000000000001},"max":{"1min":0.78000000000000003,"5min":4.0810000000000004,"15min":28.138000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.96099999999999997,"15min":1.1759999999999999},"min":{"1min":0.435,"5min":0.28199999999999997,"15min":0.26700000000000002},"max":{"1min":0.875,"5min":4.8680000000000003,"15min":28.117000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Sep 17 08:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.71499999999999997},"min":{"1min":0.30199999999999999,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.73999999999999999,"5min":4.2640000000000002,"15min":6.2329999999999997},"last":0.624},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.52600000000000002,"15min":0.66400000000000003},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":2.718,"15min":4.5300000000000002},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65300000000000002}]},{"osd":7,"last update":"Tue Sep 17 08:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.72599999999999998},"min":{"1min":0.374,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72099999999999997,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.39000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77400000000000002,"5min":3.6869999999999998,"15min":4.6950000000000003},"last":0.52400000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117313,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":812740,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":811262,"kb_avail":92932412,"statfs":{"total":95995035648,"available":95162789888,"internally_reserved":0,"allocated":1499136,"data_stored":1119314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":830732744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":1.052,"15min":2.0499999999999998},"min":{"1min":0.19600000000000001,"5min":0.152,"15min":0.152},"max":{"1min":0.42199999999999999,"5min":8.2620000000000005,"15min":130.40299999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.93500000000000005,"15min":1.7330000000000001},"min":{"1min":0.214,"5min":0.186,"15min":0.17599999999999999},"max":{"1min":0.52900000000000003,"5min":9.4009999999999998,"15min":130.70099999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Tue Sep 17 08:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.3879999999999999,"15min":1.5129999999999999},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.214},"max":{"1min":0.51900000000000002,"5min":20.812999999999999,"15min":71.459999999999994},"last":0.309},{"interface":"front","average":{"1min":0.34100000000000003,"5min":1.345,"15min":1.631},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.192},"max":{"1min":0.49399999999999999,"5min":20.777000000000001,"15min":71.438999999999993},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":1.3169999999999999,"15min":1.4970000000000001},"min":{"1min":0.245,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.495,"5min":21.161000000000001,"15min":60.423999999999999},"last":0.371},{"interface":"front","average":{"1min":0.32400000000000001,"5min":1.351,"15min":1.6459999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":21.292000000000002,"15min":60.368000000000002},"last":0.378}]},{"osd":4,"last update":"Tue Sep 17 08:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.85199999999999998,"15min":1.2130000000000001},"min":{"1min":0.42599999999999999,"5min":0.316,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":6.8479999999999999,"15min":30.722999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.84099999999999997,"15min":1.1359999999999999},"min":{"1min":0.33100000000000002,"5min":0.27700000000000002,"15min":0.27200000000000002},"max":{"1min":0.86699999999999999,"5min":8.7240000000000002,"15min":30.884},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.88600000000000001,"15min":1.111},"min":{"1min":0.35299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":6.9089999999999998,"15min":6.9089999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.86299999999999999,"15min":1.198},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":4.8769999999999998,"15min":9.7289999999999992},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Sep 17 08:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.87,"15min":1.5680000000000001},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":8.6880000000000006,"15min":163.006},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.93500000000000005,"15min":1.75},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":6.8869999999999996,"15min":162.83699999999999},"last":0.55100000000000005}]}]},{"osd":2,"up_from":227,"seq":974957576365,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":403848,"kb_used_data":1144,"kb_used_omap":4,"kb_used_meta":402683,"kb_avail":93341304,"statfs":{"total":95995035648,"available":95581495296,"internally_reserved":0,"allocated":1171456,"data_stored":799998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":412348067},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.85999999999999999,"15min":1.2190000000000001},"min":{"1min":0.22700000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.54600000000000004,"5min":6.7599999999999998,"15min":9.6389999999999993},"last":0.307},{"interface":"front","average":{"1min":0.35899999999999999,"5min":1.127,"15min":1.3100000000000001},"min":{"1min":0.25700000000000001,"5min":0.192,"15min":0.184},"max":{"1min":0.56499999999999995,"5min":6.2220000000000004,"15min":10.237},"last":0.42299999999999999}]},{"osd":1,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":1.4399999999999999,"15min":1.468},"min":{"1min":0.245,"5min":0.221,"15min":0.19600000000000001},"max":{"1min":0.56100000000000005,"5min":9.2240000000000002,"15min":41.405000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":1.1459999999999999,"15min":1.4510000000000001},"min":{"1min":0.255,"5min":0.217,"15min":0.19500000000000001},"max":{"1min":0.48699999999999999,"5min":8.2759999999999998,"15min":41.25},"last":0.36799999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.97699999999999998,"15min":1.3959999999999999},"min":{"1min":0.29999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52100000000000002,"5min":5.702,"15min":44.185000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":1.131,"15min":1.5049999999999999},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.54100000000000004,"5min":11.852,"15min":43.994},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.4319999999999999,"15min":1.5840000000000001},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":52.609000000000002,"15min":62.298999999999999},"last":0.749},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.5149999999999999,"15min":1.536},"min":{"1min":0.40200000000000002,"5min":0.29999999999999999,"15min":0.251},"max":{"1min":0.79300000000000004,"5min":52.570999999999998,"15min":62.469000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.873,"15min":0.97899999999999998},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.27200000000000002},"max":{"1min":0.74299999999999999,"5min":6.8529999999999998,"15min":6.8529999999999998},"last":0.623},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.96999999999999997,"15min":0.95899999999999996},"min":{"1min":0.45500000000000002,"5min":0.33200000000000002,"15min":0.28000000000000003},"max":{"1min":0.73399999999999999,"5min":4.5919999999999996,"15min":4.9770000000000003},"last":0.78200000000000003}]},{"osd":6,"last update":"Tue Sep 17 08:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.597,"15min":1.234},"min":{"1min":0.43099999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":0.78700000000000003,"5min":52.055999999999997,"15min":52.055999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.6679999999999999,"15min":1.256},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"max":{"1min":0.73299999999999998,"5min":52.014000000000003,"15min":52.014000000000003},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575179,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":916356,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":915454,"kb_avail":92828796,"statfs":{"total":95995035648,"available":95056687104,"internally_reserved":0,"allocated":909312,"data_stored":529426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":937425350},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 08:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":1.0780000000000001,"15min":1.4119999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.155},"max":{"1min":0.621,"5min":7.1609999999999996,"15min":27.670999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.85799999999999998,"15min":1.363},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.55000000000000004,"5min":6.5810000000000004,"15min":26.120999999999999},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Sep 17 08:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":1.115,"15min":1.2390000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":6,"15min":26.297999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.37,"5min":1.226,"15min":1.411},"min":{"1min":0.23699999999999999,"5min":0.22900000000000001,"15min":0.19900000000000001},"max":{"1min":0.77200000000000002,"5min":8.9900000000000002,"15min":26.602},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Sep 17 08:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":1.5549999999999999,"15min":1.371},"min":{"1min":0.221,"5min":0.221,"15min":0.184},"max":{"1min":0.59399999999999997,"5min":10.470000000000001,"15min":10.470000000000001},"last":0.374},{"interface":"front","average":{"1min":0.40000000000000002,"5min":1.1040000000000001,"15min":1.3220000000000001},"min":{"1min":0.32500000000000001,"5min":0.214,"15min":0.20599999999999999},"max":{"1min":0.65600000000000003,"5min":7.2450000000000001,"15min":22.984999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Sep 17 08:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":2.6219999999999999,"15min":1.7170000000000001},"min":{"1min":0.40699999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79400000000000004,"5min":156.786,"15min":156.786},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":2.8119999999999998,"15min":1.7450000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":156.77699999999999,"15min":156.77699999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":1.216,"15min":1.157},"min":{"1min":0.39500000000000002,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":2.7189999999999999,"5min":9.6769999999999996,"15min":9.6769999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.95199999999999996,"15min":1.1220000000000001},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4470000000000001,"5min":6.2060000000000004,"15min":16.853999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue Sep 17 08:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Sep 17 08:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.0049999999999999,"15min":1.387},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.93200000000000005,"5min":4.3920000000000003,"15min":47.442},"last":0.5},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.3009999999999999,"15min":1.448},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35199999999999998},"max":{"1min":2.738,"5min":7.0419999999999998,"15min":47.475000000000001},"last":0.59499999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738701,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":785892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":785022,"kb_avail":92959260,"statfs":{"total":95995035648,"available":95190282240,"internally_reserved":0,"allocated":880640,"data_stored":521445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":803862982},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.996,"15min":1.4399999999999999},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.185},"max":{"1min":0.57999999999999996,"5min":24.738,"15min":41.247},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.3,"15min":1.7},"min":{"1min":0.26600000000000001,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":0.52700000000000002,"5min":26.294,"15min":40.436},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Sep 17 08:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.54,"15min":2.266},"min":{"1min":0.23999999999999999,"5min":0.20000000000000001,"15min":0.17799999999999999},"max":{"1min":0.56000000000000005,"5min":40.359000000000002,"15min":114.771},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":1.3879999999999999,"15min":2.2269999999999999},"min":{"1min":0.26300000000000001,"5min":0.20899999999999999,"15min":0.16700000000000001},"max":{"1min":0.60099999999999998,"5min":40.417000000000002,"15min":114.732},"last":0.36799999999999999}]},{"osd":3,"last update":"Tue Sep 17 08:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":1.831,"15min":1.7949999999999999},"min":{"1min":0.26100000000000001,"5min":0.20799999999999999,"15min":0.20399999999999999},"max":{"1min":0.65800000000000003,"5min":44.506,"15min":44.506},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.371,"5min":1.4950000000000001,"15min":1.379},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53600000000000003,"5min":45.286000000000001,"15min":45.286000000000001},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Sep 17 08:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.81100000000000005,"15min":1.3089999999999999},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.26300000000000001},"max":{"1min":0.877,"5min":3.8719999999999999,"15min":54.176000000000002},"last":0.63},{"interface":"front","average":{"1min":0.60899999999999999,"5min":1.012,"15min":1.4019999999999999},"min":{"1min":0.41299999999999998,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.93100000000000005,"5min":5.2300000000000004,"15min":54.097000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Sep 17 08:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.77300000000000002,"15min":1.1140000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.28599999999999998},"max":{"1min":0.84399999999999997,"5min":3.5459999999999998,"15min":39.058},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.75800000000000001,"15min":1.3919999999999999},"min":{"1min":0.41799999999999998,"5min":0.317,"15min":0.25600000000000001},"max":{"1min":0.95899999999999996,"5min":3.7189999999999999,"15min":54.116},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Sep 17 08:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Sep 17 08:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":1.554,"15min":1.714},"min":{"1min":0.42299999999999999,"5min":0.32300000000000001,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":30.765999999999998,"15min":54.161000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.335,"15min":1.5049999999999999},"min":{"1min":0.50700000000000001,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":30.777999999999999,"15min":53.604999999999997},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T08:34:40.044 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-17T08:34:40.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T08:34:40.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:39 smithi173 ceph-mon[33187]: 6.d scrub starts 2024-09-17T08:34:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:39 smithi173 ceph-mon[33187]: 6.d scrub ok 2024-09-17T08:34:40.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:39 smithi173 ceph-mon[33187]: from='client.16302 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T08:34:40.242 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[40649]: 1.6 scrub starts 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[40649]: 1.6 scrub ok 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[40649]: pgmap v1122: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[29565]: 1.6 scrub starts 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[29565]: 1.6 scrub ok 2024-09-17T08:34:41.078 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:40 smithi079 ceph-mon[29565]: pgmap v1122: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:40 smithi173 ceph-mon[33187]: 1.6 scrub starts 2024-09-17T08:34:41.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:40 smithi173 ceph-mon[33187]: 1.6 scrub ok 2024-09-17T08:34:41.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:40 smithi173 ceph-mon[33187]: pgmap v1122: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T08:34:42.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[40649]: 5.f scrub starts 2024-09-17T08:34:42.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[40649]: 5.f scrub ok 2024-09-17T08:34:42.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[40649]: 4.12 scrub starts 2024-09-17T08:34:42.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[40649]: 4.12 scrub ok 2024-09-17T08:34:42.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[29565]: 5.f scrub starts 2024-09-17T08:34:42.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[29565]: 5.f scrub ok 2024-09-17T08:34:42.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[29565]: 4.12 scrub starts 2024-09-17T08:34:42.101 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:41 smithi079 ceph-mon[29565]: 4.12 scrub ok 2024-09-17T08:34:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:41 smithi173 ceph-mon[33187]: 5.f scrub starts 2024-09-17T08:34:42.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:41 smithi173 ceph-mon[33187]: 5.f scrub ok 2024-09-17T08:34:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:41 smithi173 ceph-mon[33187]: 4.12 scrub starts 2024-09-17T08:34:42.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:41 smithi173 ceph-mon[33187]: 4.12 scrub ok 2024-09-17T08:34:42.369 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE Bus STOPPING 2024-09-17T08:34:42.845 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:42.845 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE Bus STOPPED 2024-09-17T08:34:42.845 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE Bus STARTING 2024-09-17T08:34:42.845 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE Serving on http://:::9283 2024-09-17T08:34:42.846 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:42] ENGINE Bus STARTED 2024-09-17T08:34:43.018 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T08:34:43.027 INFO:tasks.cephadm:Teardown begin 2024-09-17T08:34:43.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:34:43.054 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:34:43.082 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T08:34:43.082 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:34:43.109 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T08:34:43.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: ::ffff:172.21.15.173 - - [17/Sep/2024:08:34:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T08:34:43.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 6.2 scrub starts 2024-09-17T08:34:43.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 6.2 scrub ok 2024-09-17T08:34:43.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 4.a scrub starts 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 4.a scrub ok 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 6.13 scrub starts 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 6.13 scrub ok 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: pgmap v1123: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 5.5 scrub starts 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: 5.5 scrub ok 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[40649]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 6.2 scrub starts 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 6.2 scrub ok 2024-09-17T08:34:43.120 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 4.a scrub starts 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 4.a scrub ok 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 6.13 scrub starts 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 6.13 scrub ok 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: pgmap v1123: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 5.5 scrub starts 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: 5.5 scrub ok 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-17T08:34:43.121 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:42 smithi079 ceph-mon[29565]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:43.153 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T08:34:43.153 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T08:34:43.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a 2024-09-17T08:34:43.223 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 6.2 scrub starts 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 6.2 scrub ok 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 4.a scrub starts 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 4.a scrub ok 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 6.13 scrub starts 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 6.13 scrub ok 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: pgmap v1123: 142 pgs: 142 active+clean; 582 KiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 5.5 scrub starts 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: 5.5 scrub ok 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: from='client.? 172.21.15.79:0/4269073261' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-17T08:34:43.224 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:42 smithi173 ceph-mon[33187]: from='mgr.14562 172.21.15.79:0/1387355940' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T08:34:43.461 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:43 smithi079 systemd[1]: Stopping Ceph mon.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:34:43.460+0000 7f02f1ace640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a[29542]: 2024-09-17T08:34:43.460+0000 7f02f1ace640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T08:34:43.869 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:43 smithi079 podman[235704]: 2024-09-17 08:34:43.685929514 +0000 UTC m=+0.351908656 container died d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE Bus STOPPING 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE Bus STOPPED 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE Bus STARTING 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE Serving on http://:::9283 2024-09-17T08:34:43.870 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:43 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:43] ENGINE Bus STARTED 2024-09-17T08:34:43.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:34:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug there is no tcmu-runner data available 2024-09-17T08:34:44.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:44 smithi079 podman[235704]: 2024-09-17 08:34:44.034069 +0000 UTC m=+0.700048139 container cleanup d2cb1692318c1aa9d7f9dbb5ef387794b576117dd237857f7b9e1e174980aede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_CLEAN=True) 2024-09-17T08:34:44.343 INFO:journalctl@ceph.mon.a.smithi079.stdout:Sep 17 08:34:44 smithi079 bash[235704]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-a 2024-09-17T08:34:44.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T08:34:44.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:44.509 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T08:34:44.509 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-17T08:34:44.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.c 2024-09-17T08:34:44.619 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:44 smithi079 systemd[1]: Stopping Ceph mon.c for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:45.119 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:44 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:44] ENGINE Bus STOPPING 2024-09-17T08:34:45.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:44 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c[40626]: 2024-09-17T08:34:44.831+0000 7f8022071640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:45.119 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:44 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c[40626]: 2024-09-17T08:34:44.831+0000 7f8022071640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:34:45.120 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:44 smithi079 podman[235848]: 2024-09-17 08:34:44.89332254 +0000 UTC m=+0.161741988 container died 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-17T08:34:45.414 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:45 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T08:34:45.414 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:45 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:45] ENGINE Bus STOPPED 2024-09-17T08:34:45.414 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:45 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:45] ENGINE Bus STARTING 2024-09-17T08:34:45.414 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:45 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:45] ENGINE Serving on http://:::9283 2024-09-17T08:34:45.414 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:45 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: [17/Sep/2024:08:34:45] ENGINE Bus STARTED 2024-09-17T08:34:45.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:45 smithi079 podman[235848]: 2024-09-17 08:34:45.414129821 +0000 UTC m=+0.682549314 container cleanup 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b) 2024-09-17T08:34:45.795 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:45 smithi079 bash[235848]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c 2024-09-17T08:34:45.795 INFO:journalctl@ceph.mon.c.smithi079.stdout:Sep 17 08:34:45 smithi079 podman[235864]: 2024-09-17 08:34:45.648914592 +0000 UTC m=+0.736278613 container remove 8a46d2b87fbdc8ebeaf6727441107559040274089dad23f34762535efc09cfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T08:34:45.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T08:34:45.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:45.834 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-17T08:34:45.835 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-17T08:34:45.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.b 2024-09-17T08:34:46.184 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:45 smithi173 systemd[1]: Stopping Ceph mon.b for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b[33164]: 2024-09-17T08:34:46.678+0000 7f57692e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:46.973 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b[33164]: 2024-09-17T08:34:46.678+0000 7f57692e0640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T08:34:46.974 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:46 smithi173 podman[169296]: 2024-09-17 08:34:46.923958824 +0000 UTC m=+0.386040075 container died de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:34:47.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 podman[169296]: 2024-09-17 08:34:47.344841446 +0000 UTC m=+0.806922678 container cleanup de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-17T08:34:47.474 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 bash[169296]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b 2024-09-17T08:34:47.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T08:34:47.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 podman[169313]: 2024-09-17 08:34:47.604986264 +0000 UTC m=+0.681516535 container remove de918ff78315a57f999e1d46a1f3875d0a94d21314b16ee1df545144edf7d713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:34:47.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-17T08:34:47.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 systemd[1]: Stopped Ceph mon.b for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:34:47.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Sep 17 08:34:47 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mon.b.service: Consumed 21.690s CPU time. 2024-09-17T08:34:47.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:47.869 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-17T08:34:47.869 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T08:34:47.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y 2024-09-17T08:34:48.170 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:47 smithi079 systemd[1]: Stopping Ceph mgr.y for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:48.502 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:34:48.169+0000 7fc1efe19640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:48.503 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:48 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y[29820]: 2024-09-17T08:34:48.169+0000 7fc1efe19640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-17T08:34:48.503 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:48 smithi079 podman[235993]: 2024-09-17 08:34:48.200746823 +0000 UTC m=+0.144113314 container died 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-17T08:34:48.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:48 smithi079 podman[235993]: 2024-09-17 08:34:48.701944744 +0000 UTC m=+0.645311226 container cleanup 57fa2f233e16a84cd8276513d1df0c583238172691ad2f1f2bfb4f143030c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-17T08:34:48.869 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Sep 17 08:34:48 smithi079 bash[235993]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-y 2024-09-17T08:34:49.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T08:34:49.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:49.203 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T08:34:49.203 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T08:34:49.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.x 2024-09-17T08:34:49.473 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:34:49 smithi173 systemd[1]: Stopping Ceph mgr.x for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:49.843 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:34:49 smithi173 podman[169429]: 2024-09-17 08:34:49.558087911 +0000 UTC m=+0.139372068 container died bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-17T08:34:50.223 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:34:49 smithi173 podman[169429]: 2024-09-17 08:34:49.941396146 +0000 UTC m=+0.522680314 container cleanup bccadcdc20c8bab4bca3920c83ddd68691ad48baf50935a1ee0ec6c8f0d6c33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:34:50.223 INFO:journalctl@ceph.mgr.x.smithi173.stdout:Sep 17 08:34:49 smithi173 bash[169429]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-mgr-x 2024-09-17T08:34:50.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T08:34:50.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:50.504 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T08:34:50.504 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T08:34:50.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.0 2024-09-17T08:34:50.825 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:50 smithi079 systemd[1]: Stopping Ceph osd.0 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:51.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:50 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0[44871]: 2024-09-17T08:34:50.823+0000 7f11571a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:51.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:50 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0[44871]: 2024-09-17T08:34:50.823+0000 7f11571a7640 -1 osd.0 470 *** Got signal Terminated *** 2024-09-17T08:34:51.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:50 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0[44871]: 2024-09-17T08:34:50.823+0000 7f11571a7640 -1 osd.0 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:34:56.665 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:56 smithi079 podman[236122]: 2024-09-17 08:34:56.387176105 +0000 UTC m=+5.695203094 container died 56eccc23d4d7020a09871ca222ad5be3f7b44a7967d735ea32b335371a6128c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T08:34:57.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:56 smithi079 podman[236122]: 2024-09-17 08:34:56.856171173 +0000 UTC m=+6.164198151 container cleanup 56eccc23d4d7020a09871ca222ad5be3f7b44a7967d735ea32b335371a6128c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b) 2024-09-17T08:34:57.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:56 smithi079 bash[236122]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0 2024-09-17T08:34:57.472 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236141]: 2024-09-17 08:34:57.145693181 +0000 UTC m=+0.753150190 container remove 56eccc23d4d7020a09871ca222ad5be3f7b44a7967d735ea32b335371a6128c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T08:34:57.778 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236238]: 2024-09-17 08:34:57.375751643 +0000 UTC m=+0.015822286 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:34:57.778 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236238]: 2024-09-17 08:34:57.517055267 +0000 UTC m=+0.157125910 container create 5dc6485e5a9e38726b72209cffc2be38c5d087002d6b857f620c820d8a5f239d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-17T08:34:58.119 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236238]: 2024-09-17 08:34:57.778351065 +0000 UTC m=+0.418421708 container init 5dc6485e5a9e38726b72209cffc2be38c5d087002d6b857f620c820d8a5f239d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-17T08:34:58.120 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236238]: 2024-09-17 08:34:57.783724777 +0000 UTC m=+0.423795418 container start 5dc6485e5a9e38726b72209cffc2be38c5d087002d6b857f620c820d8a5f239d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-17T08:34:58.120 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:57 smithi079 podman[236238]: 2024-09-17 08:34:57.886510517 +0000 UTC m=+0.526581158 container attach 5dc6485e5a9e38726b72209cffc2be38c5d087002d6b857f620c820d8a5f239d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-17T08:34:58.120 INFO:journalctl@ceph.osd.0.smithi079.stdout:Sep 17 08:34:58 smithi079 podman[236238]: 2024-09-17 08:34:58.007257344 +0000 UTC m=+0.647327985 container died 5dc6485e5a9e38726b72209cffc2be38c5d087002d6b857f620c820d8a5f239d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-17T08:34:58.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T08:34:58.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:34:58.829 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T08:34:58.829 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T08:34:58.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.1 2024-09-17T08:34:59.119 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:34:58 smithi079 systemd[1]: Stopping Ceph osd.1 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:34:59.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:34:59 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1[51274]: 2024-09-17T08:34:59.172+0000 7f9236e4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:34:59.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:34:59 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1[51274]: 2024-09-17T08:34:59.172+0000 7f9236e4b640 -1 osd.1 470 *** Got signal Terminated *** 2024-09-17T08:34:59.620 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:34:59 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1[51274]: 2024-09-17T08:34:59.172+0000 7f9236e4b640 -1 osd.1 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:35:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:35:02.625Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:35:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:35:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:35:02.628Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:35:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:35:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:35:02.628Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:35:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:35:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:35:02.629Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:35:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:35:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:35:02.629Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:35:05.010 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:04 smithi079 podman[236366]: 2024-09-17 08:35:04.740690651 +0000 UTC m=+5.693294562 container died cb4abd5a7dc65268b47733d531489381ba5181ea6f5f767b45f7a307ed704fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:05.352 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:05 smithi079 podman[236366]: 2024-09-17 08:35:05.18461014 +0000 UTC m=+6.137214040 container cleanup cb4abd5a7dc65268b47733d531489381ba5181ea6f5f767b45f7a307ed704fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:35:05.352 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:05 smithi079 bash[236366]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1 2024-09-17T08:35:05.619 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:05 smithi079 podman[236387]: 2024-09-17 08:35:05.352507317 +0000 UTC m=+0.606051668 container remove cb4abd5a7dc65268b47733d531489381ba5181ea6f5f767b45f7a307ed704fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:06.064 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:05 smithi079 podman[236483]: 2024-09-17 08:35:05.685570732 +0000 UTC m=+0.015815329 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:06.064 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:05 smithi079 podman[236483]: 2024-09-17 08:35:05.803386337 +0000 UTC m=+0.133630961 container create 360aff8e0a6beeb5b1c2fba23e64db5546f9de1c02e01e011de50fb574a29305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:06.064 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:06 smithi079 podman[236483]: 2024-09-17 08:35:06.06421658 +0000 UTC m=+0.394461173 container init 360aff8e0a6beeb5b1c2fba23e64db5546f9de1c02e01e011de50fb574a29305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b) 2024-09-17T08:35:06.369 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:06 smithi079 podman[236483]: 2024-09-17 08:35:06.06926691 +0000 UTC m=+0.399511495 container start 360aff8e0a6beeb5b1c2fba23e64db5546f9de1c02e01e011de50fb574a29305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T08:35:06.370 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:06 smithi079 podman[236483]: 2024-09-17 08:35:06.147345821 +0000 UTC m=+0.477590403 container attach 360aff8e0a6beeb5b1c2fba23e64db5546f9de1c02e01e011de50fb574a29305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=reef-19e2a2b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-17T08:35:06.370 INFO:journalctl@ceph.osd.1.smithi079.stdout:Sep 17 08:35:06 smithi079 podman[236483]: 2024-09-17 08:35:06.261363175 +0000 UTC m=+0.591607759 container died 360aff8e0a6beeb5b1c2fba23e64db5546f9de1c02e01e011de50fb574a29305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-17T08:35:07.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T08:35:07.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:07.061 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T08:35:07.061 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T08:35:07.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2 2024-09-17T08:35:07.369 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:07 smithi079 systemd[1]: Stopping Ceph osd.2 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:07.869 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:35:07.380+0000 7fd5e973e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:07.870 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:35:07.380+0000 7fd5e973e640 -1 osd.2 470 *** Got signal Terminated *** 2024-09-17T08:35:07.870 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:07 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2[143185]: 2024-09-17T08:35:07.380+0000 7fd5e973e640 -1 osd.2 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:13.163 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:12 smithi079 podman[236612]: 2024-09-17 08:35:12.892898661 +0000 UTC m=+5.615162788 container died 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-17T08:35:13.537 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:13 smithi079 podman[236612]: 2024-09-17 08:35:13.319654732 +0000 UTC m=+6.041918852 container cleanup 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-17T08:35:13.538 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:13 smithi079 bash[236612]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2 2024-09-17T08:35:13.869 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:13 smithi079 podman[236633]: 2024-09-17 08:35:13.537731344 +0000 UTC m=+0.642234066 container remove 143873d049c361cbba35616334c1c7a5b58be5c89874faaaebd144cca8d4fb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b) 2024-09-17T08:35:14.224 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:13 smithi079 podman[236726]: 2024-09-17 08:35:13.83791273 +0000 UTC m=+0.015769507 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:14.224 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:13 smithi079 podman[236726]: 2024-09-17 08:35:13.988610723 +0000 UTC m=+0.166467490 container create d14f421a43f5d572d37e1601e340c6bbf055533c2c51b2ff13c1135debb6b55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:35:14.224 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:14 smithi079 podman[236726]: 2024-09-17 08:35:14.224484897 +0000 UTC m=+0.402341666 container init d14f421a43f5d572d37e1601e340c6bbf055533c2c51b2ff13c1135debb6b55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True) 2024-09-17T08:35:14.619 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:14 smithi079 podman[236726]: 2024-09-17 08:35:14.229585508 +0000 UTC m=+0.407442273 container start d14f421a43f5d572d37e1601e340c6bbf055533c2c51b2ff13c1135debb6b55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:14.619 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:14 smithi079 podman[236726]: 2024-09-17 08:35:14.316038116 +0000 UTC m=+0.493894888 container attach d14f421a43f5d572d37e1601e340c6bbf055533c2c51b2ff13c1135debb6b55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:35:14.620 INFO:journalctl@ceph.osd.2.smithi079.stdout:Sep 17 08:35:14 smithi079 podman[236726]: 2024-09-17 08:35:14.416896681 +0000 UTC m=+0.594753456 container died d14f421a43f5d572d37e1601e340c6bbf055533c2c51b2ff13c1135debb6b55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-17T08:35:15.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T08:35:15.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:15.054 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T08:35:15.054 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-17T08:35:15.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.3 2024-09-17T08:35:15.369 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:15 smithi079 systemd[1]: Stopping Ceph osd.3 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:15.869 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:15 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3[63405]: 2024-09-17T08:35:15.394+0000 7f47c47be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:15.869 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:15 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3[63405]: 2024-09-17T08:35:15.394+0000 7f47c47be640 -1 osd.3 470 *** Got signal Terminated *** 2024-09-17T08:35:15.870 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:15 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3[63405]: 2024-09-17T08:35:15.394+0000 7f47c47be640 -1 osd.3 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:17.474 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:17 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:17.143+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:18.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:18 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:18.124+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:19.391 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:19 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:19.140+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:19.646 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:19 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:19.489+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:20.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:20 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:20.135+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:20.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:19 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:19.972+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:20.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:20 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:20.503+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:21.265 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:20 smithi079 podman[236854]: 2024-09-17 08:35:20.933505597 +0000 UTC m=+5.654179832 container died 73e963eeb616d75587047fdd8900e94a80629abb6ac23456645d180d4bba8e20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:21.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:21 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:21.087+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:21.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:21 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:21.021+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:21.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:21 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:21.094+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:21.619 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:21 smithi079 podman[236854]: 2024-09-17 08:35:21.421557112 +0000 UTC m=+6.142231333 container cleanup 73e963eeb616d75587047fdd8900e94a80629abb6ac23456645d180d4bba8e20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-17T08:35:21.619 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:21 smithi079 bash[236854]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3 2024-09-17T08:35:21.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:21 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:21.538+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:22.027 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:21 smithi079 podman[236874]: 2024-09-17 08:35:21.673231876 +0000 UTC m=+0.743545044 container remove 73e963eeb616d75587047fdd8900e94a80629abb6ac23456645d180d4bba8e20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:35:22.334 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:21.929956964 +0000 UTC m=+0.015930833 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:22.334 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:22.032177787 +0000 UTC m=+0.118151650 container create f2eb94b66b40d108382149825b5d90d4e1c3b840a90193235462f41e8111957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b) 2024-09-17T08:35:22.334 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:22.25123678 +0000 UTC m=+0.337210649 container init f2eb94b66b40d108382149825b5d90d4e1c3b840a90193235462f41e8111957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-17T08:35:22.335 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:22.256459109 +0000 UTC m=+0.342432972 container start f2eb94b66b40d108382149825b5d90d4e1c3b840a90193235462f41e8111957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-17T08:35:22.335 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:22.334394148 +0000 UTC m=+0.420368018 container attach f2eb94b66b40d108382149825b5d90d4e1c3b840a90193235462f41e8111957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-17T08:35:22.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:22.106+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:22.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:22.103+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:22.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:22.042+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:22.619 INFO:journalctl@ceph.osd.3.smithi079.stdout:Sep 17 08:35:22 smithi079 podman[236969]: 2024-09-17 08:35:22.443788078 +0000 UTC m=+0.529761938 container died f2eb94b66b40d108382149825b5d90d4e1c3b840a90193235462f41e8111957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T08:35:22.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:22.569+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:23.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.3.service' 2024-09-17T08:35:23.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:23.045 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-17T08:35:23.045 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-17T08:35:23.045 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.4 2024-09-17T08:35:23.355 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:23.077+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:23.355 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:23 smithi173 systemd[1]: Stopping Ceph osd.4 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:23.355 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:23.147+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:23.356 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:22 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:22.999+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:23.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:23.538+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:23.724 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:23.354+0000 7f79c8793640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:23.724 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:23.354+0000 7f79c8793640 -1 osd.4 470 *** Got signal Terminated *** 2024-09-17T08:35:23.724 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:23 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:23.354+0000 7f79c8793640 -1 osd.4 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:24.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:24 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:24.123+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:24.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:24 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:24.046+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:24.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:24 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:24.027+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:24.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:24 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:24.507+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:25.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:25 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:25.158+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:25.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:25 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:25.049+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:25.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:25 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:25.067+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:25.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:25 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:25.529+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:26.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:26.103+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:26.474 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:26.151+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:26.474 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:26.151+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:00.747682+0000 front 2024-09-17T08:35:00.747820+0000 (oldest deadline 2024-09-17T08:35:26.047648+0000) 2024-09-17T08:35:26.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:26.050+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:26.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:26.050+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:26.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:26.538+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:26.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:26 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:26.538+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:27.473 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:27.152+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:27.474 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:27.152+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:00.747682+0000 front 2024-09-17T08:35:00.747820+0000 (oldest deadline 2024-09-17T08:35:26.047648+0000) 2024-09-17T08:35:27.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:27.116+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:27.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:27.059+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:27.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:27.059+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:27.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:27.501+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:27.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:27 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:27.501+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:28.463 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:28.092+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:28.464 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:28.122+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:50.846098+0000 front 2024-09-17T08:34:50.846024+0000 (oldest deadline 2024-09-17T08:35:16.746132+0000) 2024-09-17T08:35:28.464 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4[38479]: 2024-09-17T08:35:28.122+0000 7f79c4da7640 -1 osd.4 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:00.747682+0000 front 2024-09-17T08:35:00.747820+0000 (oldest deadline 2024-09-17T08:35:26.047648+0000) 2024-09-17T08:35:28.464 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:28.037+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:28.464 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:28.037+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:28.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:28.462+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:28.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:28 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:28.462+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:29.178 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:29.061+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:29.179 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:29.061+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:29.179 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:29.080+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:29.179 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:29.080+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:29.179 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:28 smithi173 podman[169587]: 2024-09-17 08:35:28.891990978 +0000 UTC m=+5.638528790 container died 1298e215c180d69e845e77678c7ddff33958eeca42e3f2f05ccd5d3e87006b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-17T08:35:29.452 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:29 smithi173 podman[169587]: 2024-09-17 08:35:29.377579843 +0000 UTC m=+6.124117654 container cleanup 1298e215c180d69e845e77678c7ddff33958eeca42e3f2f05ccd5d3e87006b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-17T08:35:29.452 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:29 smithi173 bash[169587]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4 2024-09-17T08:35:29.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:29.450+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:29.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:29 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:29.450+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:29.724 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:29 smithi173 podman[169609]: 2024-09-17 08:35:29.688161956 +0000 UTC m=+0.791710507 container remove 1298e215c180d69e845e77678c7ddff33958eeca42e3f2f05ccd5d3e87006b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b) 2024-09-17T08:35:30.374 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:30.071+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:30.374 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:30.071+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:30.044+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:30.045+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:30.038969614 +0000 UTC m=+0.112260885 container create 4d247b20c4c8be4687e197da3c39502939c99d8fb04ce36208ca3dba092c20c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:29.943058821 +0000 UTC m=+0.016350103 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:30.26654303 +0000 UTC m=+0.339834305 container init 4d247b20c4c8be4687e197da3c39502939c99d8fb04ce36208ca3dba092c20c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:30.271436859 +0000 UTC m=+0.344728133 container start 4d247b20c4c8be4687e197da3c39502939c99d8fb04ce36208ca3dba092c20c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:35:30.375 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:30.374688573 +0000 UTC m=+0.447979854 container attach 4d247b20c4c8be4687e197da3c39502939c99d8fb04ce36208ca3dba092c20c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:35:30.676 INFO:journalctl@ceph.osd.4.smithi173.stdout:Sep 17 08:35:30 smithi173 podman[169701]: 2024-09-17 08:35:30.460230992 +0000 UTC m=+0.533522269 container died 4d247b20c4c8be4687e197da3c39502939c99d8fb04ce36208ca3dba092c20c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:30.676 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:30.472+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:30.676 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:30 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:30.472+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:31.082 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.4.service' 2024-09-17T08:35:31.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:31.121 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-17T08:35:31.121 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-17T08:35:31.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.5 2024-09-17T08:35:31.433 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:31.025+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:31.025+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 systemd[1]: Stopping Ceph osd.5 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:31.066+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:31.066+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:31.432+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:31.434 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:31.432+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:31.723 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:31.465+0000 7f60a1d24640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:31.724 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:31.465+0000 7f60a1d24640 -1 osd.5 470 *** Got signal Terminated *** 2024-09-17T08:35:31.724 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:31 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:31.465+0000 7f60a1d24640 -1 osd.5 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:32.447 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:32.005+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:32.447 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:32.005+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:32.448 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:32.056+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:32.448 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:32.056+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:32.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:32.446+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:32.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:32 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:32.446+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:33.473 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:33.018+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:33.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:33.018+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:33.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:33.044+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:33.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:33.044+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:33.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:33.476+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:33.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:33 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:33.476+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:34.473 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:34.002+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:34.473 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:34.002+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:34.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:34.019+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:34.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:34.019+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:34.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:34.509+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:34.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:34 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:34.509+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:35.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:35.009+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:35.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:35.009+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:35.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:34.999+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:35.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:34.999+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:35.474 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:34.999+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:10.889890+0000 front 2024-09-17T08:35:10.889934+0000 (oldest deadline 2024-09-17T08:35:34.989821+0000) 2024-09-17T08:35:35.960 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:35.546+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:35.960 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:35.546+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:35.960 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:35.546+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:36.223 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:36.020+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:36.224 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:36.020+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:36.224 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:35.959+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:54.488005+0000 front 2024-09-17T08:34:54.487900+0000 (oldest deadline 2024-09-17T08:35:20.387856+0000) 2024-09-17T08:35:36.224 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:35.959+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:02.089132+0000 front 2024-09-17T08:35:02.088955+0000 (oldest deadline 2024-09-17T08:35:25.588983+0000) 2024-09-17T08:35:36.224 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:35 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5[44544]: 2024-09-17T08:35:35.959+0000 7f609a330640 -1 osd.5 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:10.889890+0000 front 2024-09-17T08:35:10.889934+0000 (oldest deadline 2024-09-17T08:35:34.989821+0000) 2024-09-17T08:35:36.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:36.500+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:37.069 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:36.500+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:37.070 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:36 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:36.500+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:37.265 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:36 smithi173 podman[169831]: 2024-09-17 08:35:36.990302191 +0000 UTC m=+5.634930228 container died 70c1da61f0fdd9ba7edc9660db2761f6a6451975e5fbdd959e52f09e3129c6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:37.266 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:37.054+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:37.266 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:37.054+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:37.640 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:37 smithi173 podman[169831]: 2024-09-17 08:35:37.372591893 +0000 UTC m=+6.017219924 container cleanup 70c1da61f0fdd9ba7edc9660db2761f6a6451975e5fbdd959e52f09e3129c6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T08:35:37.640 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:37 smithi173 bash[169831]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5 2024-09-17T08:35:37.640 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:37.477+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:37.640 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:37.477+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:37.640 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:37 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:37.477+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:37.974 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:37 smithi173 podman[169851]: 2024-09-17 08:35:37.640387312 +0000 UTC m=+0.646195888 container remove 70c1da61f0fdd9ba7edc9660db2761f6a6451975e5fbdd959e52f09e3129c6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b) 2024-09-17T08:35:38.335 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:38.039+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:38.335 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:38.039+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:38.335 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:38.039+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:38.335 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:37 smithi173 podman[169946]: 2024-09-17 08:35:37.884840583 +0000 UTC m=+0.015775062 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:38.336 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:37 smithi173 podman[169946]: 2024-09-17 08:35:37.999506912 +0000 UTC m=+0.130441383 container create c283f631e6f3aa254c81ef00ab660af3cd508816f291dc406738eb9504ee55fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-19e2a2b, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-17T08:35:38.336 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:38 smithi173 podman[169946]: 2024-09-17 08:35:38.335628288 +0000 UTC m=+0.466562760 container init c283f631e6f3aa254c81ef00ab660af3cd508816f291dc406738eb9504ee55fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T08:35:38.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:38.469+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:38.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:38.469+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:38.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:38 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:38.469+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:38.724 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:38 smithi173 podman[169946]: 2024-09-17 08:35:38.341474992 +0000 UTC m=+0.472409469 container start c283f631e6f3aa254c81ef00ab660af3cd508816f291dc406738eb9504ee55fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2) 2024-09-17T08:35:38.724 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:38 smithi173 podman[169946]: 2024-09-17 08:35:38.427387576 +0000 UTC m=+0.558322060 container attach c283f631e6f3aa254c81ef00ab660af3cd508816f291dc406738eb9504ee55fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:35:38.725 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 17 08:35:38 smithi173 podman[169946]: 2024-09-17 08:35:38.524205062 +0000 UTC m=+0.655139530 container died c283f631e6f3aa254c81ef00ab660af3cd508816f291dc406738eb9504ee55fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-17T08:35:39.113 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:39.079+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:39.114 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:39.079+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:39.114 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:39.079+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:39.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.5.service' 2024-09-17T08:35:39.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:39.188 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-17T08:35:39.188 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-17T08:35:39.188 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6 2024-09-17T08:35:39.452 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 systemd[1]: Stopping Ceph osd.6 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:39.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.451+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:39.723 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.451+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:39.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.451+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:39.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.502+0000 7f73e4be7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:39.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.502+0000 7f73e4be7640 -1 osd.6 470 *** Got signal Terminated *** 2024-09-17T08:35:39.724 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:39 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:39.502+0000 7f73e4be7640 -1 osd.6 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:40.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:40.122+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:40.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:40.122+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:40.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:40.122+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:40.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:40.480+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:40.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:40.480+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:40.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:40 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:40.480+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:41.477 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:41.087+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:41.477 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:41.087+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:41.477 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:41.087+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:41.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:41.490+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:41.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:41.490+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:41.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:41 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:41.490+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:42.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:42.108+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:42.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:42.108+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:42.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:42.108+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:42.973 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:42.522+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:42.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:42.522+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:42.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:42 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:42.522+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:43.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:43.134+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:43.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:43.134+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:43.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:43.134+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:43.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:43.134+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:43.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:43.535+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:55.768454+0000 front 2024-09-17T08:34:55.768407+0000 (oldest deadline 2024-09-17T08:35:18.668276+0000) 2024-09-17T08:35:43.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:43.535+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:34:58.668700+0000 front 2024-09-17T08:34:58.668772+0000 (oldest deadline 2024-09-17T08:35:26.269274+0000) 2024-09-17T08:35:43.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:43.535+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:12.170335+0000 front 2024-09-17T08:35:12.170272+0000 (oldest deadline 2024-09-17T08:35:34.970591+0000) 2024-09-17T08:35:43.974 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:43 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6[165943]: 2024-09-17T08:35:43.535+0000 7f73dd1f3640 -1 osd.6 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.271673+0000 front 2024-09-17T08:35:17.271649+0000 (oldest deadline 2024-09-17T08:35:42.571606+0000) 2024-09-17T08:35:44.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:44 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:44.183+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:44.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:44 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:44.183+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:44.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:44 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:44.183+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:44.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:44 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:44.183+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:45.223 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:44 smithi173 podman[170074]: 2024-09-17 08:35:44.962122778 +0000 UTC m=+5.553301264 container died 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:45.224 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:45 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:45.162+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:45.224 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:45 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:45.162+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:45.224 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:45 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:45.162+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:45.224 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:45 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:45.162+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:45.609 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:45 smithi173 podman[170074]: 2024-09-17 08:35:45.316629668 +0000 UTC m=+5.907808142 container cleanup 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-17T08:35:45.609 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:45 smithi173 bash[170074]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6 2024-09-17T08:35:45.935 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:45 smithi173 podman[170094]: 2024-09-17 08:35:45.609688071 +0000 UTC m=+0.644775661 container remove 5d4ad251e83e13f0805bf2ae0a9e8bd1d3d61f4ecd61109c489b7bf6d3978348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:46.195 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:46.165+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:46.165+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:46.165+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:46 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:46.165+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:45 smithi173 podman[170187]: 2024-09-17 08:35:45.935622907 +0000 UTC m=+0.107356715 container create 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:45 smithi173 podman[170187]: 2024-09-17 08:35:45.844124206 +0000 UTC m=+0.015857969 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 podman[170187]: 2024-09-17 08:35:46.113467112 +0000 UTC m=+0.285200857 container init 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 podman[170187]: 2024-09-17 08:35:46.118537768 +0000 UTC m=+0.290271517 container start 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-17T08:35:46.196 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 podman[170187]: 2024-09-17 08:35:46.196019887 +0000 UTC m=+0.367753635 container attach 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-17T08:35:46.473 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 podman[170187]: 2024-09-17 08:35:46.305417997 +0000 UTC m=+0.477151757 container died 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-17T08:35:46.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service' 2024-09-17T08:35:46.844 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 podman[170230]: 2024-09-17 08:35:46.799277634 +0000 UTC m=+0.487352825 container remove 95c556481659a20c5d56e390541373167dd4948b1f09e83158a9f60620729a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-17T08:35:46.845 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-17T08:35:46.845 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 systemd[1]: Stopped Ceph osd.6 for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:35:46.845 INFO:journalctl@ceph.osd.6.smithi173.stdout:Sep 17 08:35:46 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.6.service: Consumed 4.693s CPU time. 2024-09-17T08:35:46.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:46.860 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-17T08:35:46.860 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-17T08:35:46.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.7 2024-09-17T08:35:47.138 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:46 smithi173 systemd[1]: Stopping Ceph osd.7 for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:47.179 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.137+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:47.179 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.137+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:47.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.137+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:47.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.137+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:47.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.195+0000 7f9e546c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:47.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.195+0000 7f9e546c5640 -1 osd.7 470 *** Got signal Terminated *** 2024-09-17T08:35:47.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:47 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:47.195+0000 7f9e546c5640 -1 osd.7 470 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T08:35:48.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:48 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:48.142+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:48.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:48 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:48.142+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:48.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:48 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:48.142+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:48.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:48 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:48.142+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:49.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:49 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:49.133+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:49.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:49 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:49.133+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:49.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:49 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:49.133+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:49.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:49 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:49.133+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:50.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:50 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:50.094+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:50.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:50 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:50.094+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:50.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:50 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:50.094+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:50.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:50 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:50.094+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:51.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:51 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:51.111+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:51.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:51 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:51.111+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:51.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:51 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:51.111+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:51.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:51 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:51.111+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:52.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:52.106+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6806 osd.0 since back 2024-09-17T08:34:53.026714+0000 front 2024-09-17T08:34:53.026863+0000 (oldest deadline 2024-09-17T08:35:19.427183+0000) 2024-09-17T08:35:52.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:52.106+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6814 osd.1 since back 2024-09-17T08:35:03.428743+0000 front 2024-09-17T08:35:03.428829+0000 (oldest deadline 2024-09-17T08:35:28.728680+0000) 2024-09-17T08:35:52.473 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:52.106+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6822 osd.2 since back 2024-09-17T08:35:11.629963+0000 front 2024-09-17T08:35:11.629897+0000 (oldest deadline 2024-09-17T08:35:37.529969+0000) 2024-09-17T08:35:52.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:52.106+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.79:6830 osd.3 since back 2024-09-17T08:35:17.530430+0000 front 2024-09-17T08:35:17.530457+0000 (oldest deadline 2024-09-17T08:35:42.830394+0000) 2024-09-17T08:35:52.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7[56958]: 2024-09-17T08:35:52.106+0000 7f9e4ccd1640 -1 osd.7 470 heartbeat_check: no reply from 172.21.15.173:6804 osd.4 since back 2024-09-17T08:35:27.531180+0000 front 2024-09-17T08:35:27.531245+0000 (oldest deadline 2024-09-17T08:35:51.631237+0000) 2024-09-17T08:35:53.127 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:52 smithi173 podman[170314]: 2024-09-17 08:35:52.800775424 +0000 UTC m=+5.722816635 container died 9ae2cb0c9a4bb0006d9a97742b85a0c2c0bf13d89b0a50896be059c0705e58a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-17T08:35:53.436 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:53 smithi173 podman[170314]: 2024-09-17 08:35:53.210515394 +0000 UTC m=+6.132556592 container cleanup 9ae2cb0c9a4bb0006d9a97742b85a0c2c0bf13d89b0a50896be059c0705e58a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T08:35:53.436 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:53 smithi173 bash[170314]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7 2024-09-17T08:35:53.723 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:53 smithi173 podman[170346]: 2024-09-17 08:35:53.436604111 +0000 UTC m=+0.620233490 container remove 9ae2cb0c9a4bb0006d9a97742b85a0c2c0bf13d89b0a50896be059c0705e58a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-17T08:35:54.039 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:53 smithi173 podman[170439]: 2024-09-17 08:35:53.671037972 +0000 UTC m=+0.015741263 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T08:35:54.039 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:53 smithi173 podman[170439]: 2024-09-17 08:35:53.78738692 +0000 UTC m=+0.132090238 container create 6ed7c98cbaa5418fd080419bfce0342d22285e050d9ac6992b868f05bd68ebee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T08:35:54.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:54 smithi173 podman[170439]: 2024-09-17 08:35:54.039776992 +0000 UTC m=+0.384480277 container init 6ed7c98cbaa5418fd080419bfce0342d22285e050d9ac6992b868f05bd68ebee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T08:35:54.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:54 smithi173 podman[170439]: 2024-09-17 08:35:54.045241148 +0000 UTC m=+0.389944425 container start 6ed7c98cbaa5418fd080419bfce0342d22285e050d9ac6992b868f05bd68ebee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:35:54.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:54 smithi173 podman[170439]: 2024-09-17 08:35:54.131120454 +0000 UTC m=+0.475823744 container attach 6ed7c98cbaa5418fd080419bfce0342d22285e050d9ac6992b868f05bd68ebee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-17T08:35:54.474 INFO:journalctl@ceph.osd.7.smithi173.stdout:Sep 17 08:35:54 smithi173 podman[170439]: 2024-09-17 08:35:54.232621461 +0000 UTC m=+0.577324741 container died 6ed7c98cbaa5418fd080419bfce0342d22285e050d9ac6992b868f05bd68ebee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-17T08:35:54.867 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@osd.7.service' 2024-09-17T08:35:54.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:35:54.906 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-17T08:35:54.906 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T08:35:54.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T08:35:55.211 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:35:54 smithi079 systemd[1]: Stopping Ceph rgw.foo.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:35:55.619 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:35:55 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a[68798]: 2024-09-17T08:35:55.210+0000 7fc5c67e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T08:35:55.619 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:35:55 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a[68798]: 2024-09-17T08:35:55.210+0000 7fc5ca04b880 -1 shutting down 2024-09-17T08:36:02.973 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:02.626Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:36:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:02.628Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:36:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:02.628Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:36:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:02.628Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:36:02.974 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:02 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:02.629Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.79:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.79:8765: connect: connection refused" 2024-09-17T08:36:05.509 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 bash[237096]: time="2024-09-17T08:36:05Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T08:36:05.509 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 podman[237096]: 2024-09-17 08:36:05.228869771 +0000 UTC m=+10.130912163 container died 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-17T08:36:05.841 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 podman[237096]: 2024-09-17 08:36:05.673808983 +0000 UTC m=+10.575851417 container cleanup 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-17T08:36:05.842 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 bash[237096]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T08:36:06.102 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 podman[237120]: 2024-09-17 08:36:05.841696613 +0000 UTC m=+0.610090160 container remove 73d02485ce4d22a9f6a796a3e496846213e43355bf7d69db4b200a976507e391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-rgw-foo-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T08:36:06.102 INFO:journalctl@ceph.rgw.foo.a.smithi079.stdout:Sep 17 08:36:05 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T08:36:06.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T08:36:06.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:36:06.141 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T08:36:06.141 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T08:36:06.141 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-66e8930a-74cb-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T08:36:06.430 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 systemd[1]: Stopping Ceph prometheus.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:06.721 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:36:06.721 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T08:36:06.721 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T08:36:06.721 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T08:36:06.721 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.430Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.431Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.431Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a[68290]: ts=2024-09-17T08:36:06.431Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T08:36:06.722 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 podman[170565]: 2024-09-17 08:36:06.441318794 +0000 UTC m=+0.102707470 container died 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:06.973 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 podman[170565]: 2024-09-17 08:36:06.828280805 +0000 UTC m=+0.489669467 container cleanup 13d98da550b73c318e4887e707d5b21a74cfb9888f9827cc7350badcc92f01e8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:06.973 INFO:journalctl@ceph.prometheus.a.smithi173.stdout:Sep 17 08:36:06 smithi173 bash[170565]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T08:36:07.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66e8930a-74cb-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T08:36:07.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T08:36:07.228 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T08:36:07.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:36:07.380 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:36:09.056 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:08 smithi079 systemd[1]: Stopping Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:09.369 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a[79867]: ts=2024-09-17T08:36:09.055Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T08:36:09.369 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 podman[237557]: 2024-09-17 08:36:09.117141933 +0000 UTC m=+0.165213735 container died 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:09.755 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 podman[237557]: 2024-09-17 08:36:09.537588566 +0000 UTC m=+0.585660367 container cleanup 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:09.755 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 bash[237557]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T08:36:10.034 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 podman[237574]: 2024-09-17 08:36:09.755453701 +0000 UTC m=+0.638350054 container remove 72b7d24119ee6361d3bec78c9d014b5138e0f9655ad133f8c649b484223088ce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:10.034 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:09 smithi079 podman[237574]: 2024-09-17 08:36:09.831335456 +0000 UTC m=+0.714231767 volume remove 66e6d1332bfad7f857ee05c2db3ce8d20117ccb1148591158d1d39c76834fbbd 2024-09-17T08:36:10.369 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:10 smithi079 systemd[1]: Stopping Ceph node-exporter.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:10.370 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:10 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T08:36:10.370 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:10 smithi079 systemd[1]: Stopped Ceph alertmanager.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:36:10.370 INFO:journalctl@ceph.alertmanager.a.smithi079.stdout:Sep 17 08:36:10 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.753s CPU time. 2024-09-17T08:36:10.866 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:10 smithi079 podman[237688]: 2024-09-17 08:36:10.488402175 +0000 UTC m=+0.114867694 container died 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:10.866 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:10 smithi079 podman[237688]: 2024-09-17 08:36:10.866870128 +0000 UTC m=+0.493335637 container cleanup 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:11.119 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:10 smithi079 bash[237688]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T08:36:11.119 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:11 smithi079 podman[237708]: 2024-09-17 08:36:11.084596935 +0000 UTC m=+0.599963409 container remove 97424a460a1eb03336367bc62d7e7e86b1da282f5ea05de608da4c7a291b2412 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T08:36:11.119 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:11 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:36:11.605 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:11 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T08:36:11.605 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:11 smithi079 systemd[1]: Stopped Ceph node-exporter.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:36:11.606 INFO:journalctl@ceph.node-exporter.a.smithi079.stdout:Sep 17 08:36:11 smithi079 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 4.937s CPU time. 2024-09-17T08:36:11.901 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T08:36:12.059 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:36:13.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:13 smithi173 systemd[1]: Stopping Ceph iscsi.iscsi.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:13.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:13 smithi173 ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a[62600]: debug Shutdown received 2024-09-17T08:36:23.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:23 smithi173 bash[170948]: time="2024-09-17T08:36:23Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-17T08:36:23.766 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:23 smithi173 podman[170948]: 2024-09-17 08:36:23.519126296 +0000 UTC m=+10.146413438 container died 13549ca30b542505749492fd5acb9d4cef711d62fdac5f5c271913792f04774c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS) 2024-09-17T08:36:24.165 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:23 smithi173 podman[170948]: 2024-09-17 08:36:23.906474581 +0000 UTC m=+10.533761765 container cleanup 13549ca30b542505749492fd5acb9d4cef711d62fdac5f5c271913792f04774c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-17T08:36:24.166 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:23 smithi173 bash[170948]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-17T08:36:24.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 podman[170971]: 2024-09-17 08:36:24.165949012 +0000 UTC m=+0.646576805 container remove 13549ca30b542505749492fd5acb9d4cef711d62fdac5f5c271913792f04774c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T08:36:24.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T08:36:24.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 podman[171018]: 2024-09-17 08:36:24.40602116 +0000 UTC m=+0.113069462 container died 8776a304171f82010a582e6fd73549700511aaf040b39a9b20d82c11d5a2a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T08:36:24.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 podman[171018]: 2024-09-17 08:36:24.759745793 +0000 UTC m=+0.466794101 container cleanup 8776a304171f82010a582e6fd73549700511aaf040b39a9b20d82c11d5a2a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T08:36:24.973 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 bash[171018]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-17T08:36:24.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-17T08:36:24.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 62406 (conmon) remains running after unit stopped. 2024-09-17T08:36:24.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 171036 (podman) remains running after unit stopped. 2024-09-17T08:36:24.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: Stopped Ceph iscsi.iscsi.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:36:24.974 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:24 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 3.244s CPU time. 2024-09-17T08:36:25.417 INFO:journalctl@ceph.iscsi.iscsi.a.smithi173.stdout:Sep 17 08:36:25 smithi173 podman[171036]: 2024-09-17 08:36:25.04486113 +0000 UTC m=+0.635415309 container remove 8776a304171f82010a582e6fd73549700511aaf040b39a9b20d82c11d5a2a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-17T08:36:25.417 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 systemd[1]: Stopping Ceph grafana.a for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:25.682 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 podman[171161]: 2024-09-17 08:36:25.427298469 +0000 UTC m=+0.114761241 container died 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, vcs-type=git, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc.) 2024-09-17T08:36:25.965 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 podman[171161]: 2024-09-17 08:36:25.755764097 +0000 UTC m=+0.443226855 container cleanup 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, vcs-type=git, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , description=Ceph Grafana Container, io.buildah.version=1.32.0, architecture=x86_64, distribution-scope=public, io.openshift.expose-services=, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, version=8.5, name=ubi8) 2024-09-17T08:36:25.965 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 bash[171161]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a 2024-09-17T08:36:26.223 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 podman[171179]: 2024-09-17 08:36:25.96520218 +0000 UTC m=+0.536636433 container remove 34d70f3a8d12a3011e21051d49a24dfa2dc3428db0678a743d9a94e24cb29ec0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-grafana-a, io.openshift.expose-services=, release=239.1651231664, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, maintainer=Paul Cuzner , architecture=x86_64, com.redhat.component=ubi8-container, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8) 2024-09-17T08:36:26.223 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:25 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:36:26.223 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:26 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-17T08:36:26.223 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:26 smithi173 systemd[1]: Stopped Ceph grafana.a for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:36:26.224 INFO:journalctl@ceph.grafana.a.smithi173.stdout:Sep 17 08:36:26 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@grafana.a.service: Consumed 13.669s CPU time. 2024-09-17T08:36:26.647 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:26 smithi173 systemd[1]: Stopping Ceph node-exporter.b for 66e8930a-74cb-11ef-bceb-c7b262605968... 2024-09-17T08:36:26.918 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:26 smithi173 podman[171303]: 2024-09-17 08:36:26.647901981 +0000 UTC m=+0.120703600 container died f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:36:27.223 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 podman[171303]: 2024-09-17 08:36:27.001399945 +0000 UTC m=+0.474201605 container cleanup f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:36:27.223 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 bash[171303]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T08:36:27.560 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 podman[171319]: 2024-09-17 08:36:27.260989491 +0000 UTC m=+0.616808882 container remove f7f248cafd20e494ad3eb6f781a220a643c35b4b19fd4bb94ce65f3dde9acc29 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66e8930a-74cb-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T08:36:27.560 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T08:36:27.560 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T08:36:27.561 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 systemd[1]: Stopped Ceph node-exporter.b for 66e8930a-74cb-11ef-bceb-c7b262605968. 2024-09-17T08:36:27.561 INFO:journalctl@ceph.node-exporter.b.smithi173.stdout:Sep 17 08:36:27 smithi173 systemd[1]: ceph-66e8930a-74cb-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 4.586s CPU time. 2024-09-17T08:36:28.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:36:28.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T08:36:28.120 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T08:36:28.122 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi079/crash 2024-09-17T08:36:28.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:36:28.181 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:36:28.181 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:36:28.183 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi173/crash 2024-09-17T08:36:28.183 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash -- . 2024-09-17T08:36:28.266 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T08:36:28.266 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-09-17T08:36:28.268 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T08:36:28.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-17T08:36:28.317 INFO:teuthology.orchestra.run.smithi079.stderr:grep: /var/log/ceph/66e8930a-74cb-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-17T08:36:28.319 INFO:tasks.cephadm:Compressing logs... 2024-09-17T08:36:28.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:36:28.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:36:28.344 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:36:28.351 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T08:44:03.901 INFO:tasks.cephadm:Archiving logs... 2024-09-17T08:44:03.901 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi079/log 2024-09-17T08:44:03.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:44:53.859 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi173/log 2024-09-17T08:44:53.861 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T08:45:22.470 INFO:tasks.cephadm:Removing cluster... 2024-09-17T08:45:22.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 --force 2024-09-17T08:45:22.659 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:45:23.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66e8930a-74cb-11ef-bceb-c7b262605968 --force 2024-09-17T08:45:23.498 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 66e8930a-74cb-11ef-bceb-c7b262605968 2024-09-17T08:45:23.987 INFO:tasks.cephadm:Removing cephadm ... 2024-09-17T08:45:23.987 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T08:45:24.021 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T08:45:24.036 INFO:tasks.cephadm:Teardown complete 2024-09-17T08:45:24.036 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T08:45:24.046 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T08:45:24.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T08:45:24.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T08:45:24.181 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T08:45:24.189 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T08:45:24.189 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:45:24.192 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T08:45:24.206 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:45:24.207 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +361us[ +382us] +/- 46ms 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -2117us[-2117us] +/- 52ms 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +2860us[+2860us] +/- 87ms 2024-09-17T08:45:24.227 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:45:24.230 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T08:45:24.230 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-17T08:45:24.231 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +392us[ +474us] +/- 46ms 2024-09-17T08:45:24.231 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -2095us[-2013us] +/- 52ms 2024-09-17T08:45:24.231 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +2891us[+2973us] +/- 86ms 2024-09-17T08:45:24.231 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T08:45:24.232 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T08:45:24.241 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T08:45:24.241 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T08:45:24.360 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:45:24.477 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T08:45:24.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:45:24.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:45:24.614 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T08:45:24.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:45:24.642 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T08:45:24.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T08:45:24.721 INFO:teuthology.task.internal:Duration was 2858.069227 seconds 2024-09-17T08:45:24.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T08:45:24.911 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T08:45:24.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:45:24.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T08:45:24.974 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:45:24.975 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T08:45:25.428 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T08:45:25.428 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-09-17T08:45:25.428 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:45:25.455 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T08:45:25.455 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T08:45:25.482 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T08:45:25.482 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:45:25.497 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T08:45:25.549 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T08:45:25.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:45:25.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T08:45:27.145 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T08:45:27.156 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T08:45:27.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:45:27.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T08:45:27.232 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T08:45:27.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:45:27.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:45:27.286 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-09-17T08:45:27.305 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-09-17T08:45:27.334 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:45:27.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:45:27.390 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T08:45:27.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T08:45:27.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T08:45:27.417 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T08:45:27.418 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi079 2024-09-17T08:45:27.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:45:27.502 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908191/remote/smithi173 2024-09-17T08:45:27.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T08:45:27.544 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T08:45:27.545 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:45:27.547 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T08:45:27.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T08:45:27.607 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T08:45:27.607 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T08:45:27.632 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T08:45:27.632 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:45:27.635 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T08:45:27.647 INFO:teuthology.orchestra.run.smithi079.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:45 /home/ubuntu/cephtest 2024-09-17T08:45:27.651 INFO:teuthology.orchestra.run.smithi173.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 08:45 /home/ubuntu/cephtest 2024-09-17T08:45:27.653 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T08:45:27.682 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T08:45:27.909 INFO:teuthology.run:Summary data: description: orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} duration: 2858.0692269802094 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T08:45:27.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T08:45:27.985 INFO:teuthology.run:pass